Pre-trained GPT-2 Notebook

from transformers import GPT2LMHeadModel, GPT2Tokenizer

# Load pre-trained GPT-2 model and tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")

# Set pad token
tokenizer.pad_token = tokenizer.eos_token

import torch

def autocomplete(prompt, max_length=50, temperature=0.7, top_k=50, top_p=0.9):
    # Encode the prompt with attention mask
    inputs = tokenizer(prompt, return_tensors="pt")
    
    # Generate continuation
    with torch.no_grad():
        output = model.generate(
            inputs['input_ids'],
            attention_mask=inputs['attention_mask'],
            max_length=max_length,
            temperature=temperature,
            top_k=top_k,
            top_p=top_p,
            do_sample=True,
            pad_token_id=tokenizer.eos_token_id
        )
    
    # Decode and return the generated text
    generated_text = tokenizer.decode(output[0], skip_special_tokens=True)
    return generated_text

prompt = "Mary had a little lamb"
completion = autocomplete(prompt, max_length=80)
print(completion)

Mary had a little lamb, and the young woman asked her for a little lamb, and they gave it to her.

"Oh, my child, it is good to have a little lamb," said he, "but it is not to be bought, for it is hard to make, and it is much more difficult to make.

"When you have a little lamb, it

prompts = [
    "Mary had a little lamb",
    "The future of artificial intelligence",
    "In a galaxy far, far away",
    "DigiPen is a place where",
    "def calculate_fibonacci(n):"
]

for prompt in prompts:
    print(f"\nPrompt: {prompt}")
    print("-" * 50)
    completion = autocomplete(prompt, max_length=80)
    print(f"Output: {completion}\n")

Prompt: Mary had a little lamb
--------------------------------------------------
Output: Mary had a little lamb, and the child was very hungry, and so he took a small lamb and brought it to her, and she and the child were very merry. So the child went home and the lamb was brought to her. So she and the child went to the priest and he gave her a piece of bread and said to her, "This is good bread for you, but what


Prompt: The future of artificial intelligence
--------------------------------------------------
Output: The future of artificial intelligence is uncertain, but its future is bright.

And so, we are all waiting for a breakthrough.

And that's why I think that it's important to understand how AI is coming to the table.

One of the big questions we have right now is how AI will be able to take over a world, and how it will be able to take


Prompt: In a galaxy far, far away
--------------------------------------------------
Output: In a galaxy far, far away, there is only one thing that matters. The fate of our galaxy.

And it matters only to you.

A New Frontier for Space

It's been almost two years since I first wrote a post about this book. And that's because I've been busy.

In the last month or so, I've been working on an


Prompt: DigiPen is a place where
--------------------------------------------------
Output: DigiPen is a place where you can share your creations.

Don't let the name fool you. This is the place to share your creations and to share your creativity.

Don't let the name fool you. This is the place to share your creations and to share your creativity.

Don't let the name fool you. This is the place to share your creations and


Prompt: def calculate_fibonacci(n):
--------------------------------------------------
Output: def calculate_fibonacci(n):

fibonacci(n) = 0.01

return f(n)

def calculate_fibonacci(n):

fibonacci(n) = 0.01

return f(n)

def calculate_fibonacci(n):

fibonacci