Shakespeare GPT-2
A GPT-2 model fine-tuned on Shakespeare's complete works to generate Shakespeare-style text.
Model Description
This model is a fine-tuned version of GPT-2 (124M parameters) trained on Shakespeare's complete works. It can generate text in Shakespeare's distinctive style, including dialogue, soliloquies, and dramatic prose.
Model Architecture
- Base Model: GPT-2 (124M parameters)
- Layers: 12
- Heads: 12
- Embedding Dimension: 768
- Context Length: 1024 tokens
- Total Parameters: ~124M
Training Details
- Dataset: Complete works of Shakespeare
- Training Steps: 100,000
- Batch Size: 4
- Sequence Length: 32
- Learning Rate: 3e-4
- Optimizer: AdamW
- Device: MPS/CUDA/CPU
Intended Use
This model is intended for:
- Generating Shakespeare-style text
- Creative writing assistance
- Educational purposes in literature
- Entertainment and artistic projects
Limitations
- May generate text that mimics but doesn't perfectly replicate Shakespeare's style
- Limited by training data to Shakespeare's vocabulary and themes
- Can produce anachronistic or inconsistent content
- Maximum context length of 1024 tokens
Training Data
The model was trained on Shakespeare's complete works, including:
- All plays (comedies, tragedies, histories)
- Sonnets and poems
- Total training tokens: [Insert number of tokens]
Performance
The model achieves:
- Training Loss: [Insert final training loss]
- Best Loss: [Insert best loss achieved]
Example Usage
python from transformers import GPT2LMHeadModel, GPT2Tokenizer Load model and tokenizer model_name = "your-username/shakespeare-gpt" tokenizer = GPT2Tokenizer.from_pretrained(model_name) model = GPT2LMHeadModel.from_pretrained(model_name) Generate text prompt = "To be, or not to be," input_ids = tokenizer.encode(prompt, return_tensors='pt') output = model.generate( input_ids, max_length=500, temperature=0.8, top_k=40, do_sample=True ) generated_text = tokenizer.decode(output[0], skip_special_tokens=True) print(generated_text)
Sample Outputs
Prompt: "To be, or not to be," Output: [Insert sample generation] Prompt: "Friends, Romans, countrymen," Output: [Insert sample generation]