Base Model vs. Instruction-Tuned Model

Load both the base and instruction-tuned models

from transformers import AutoTokenizer, AutoModelForCausalLM

# Load base (completion-only) model
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B")
base_tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B")

# Load instruct model  
instruct_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")
instruct_tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")

Base (Completion) Model Output

base_inputs = base_tokenizer("What should I do on my upcoming trip to Paris?", return_tensors="pt")
base_outputs = base_model.generate(
    **base_inputs,
    max_new_tokens=150,
    temperature=0.7,
    do_sample=True,
    pad_token_id=base_tokenizer.eos_token_id
)
base_response = base_tokenizer.decode(base_outputs[0], skip_special_tokens=True)
print(base_response)

What should I do on my upcoming trip to Paris? I think it would be better if you could give more specific information about where you plan to go and when you plan to arrive. Also, can you suggest any specific tips or recommendations for traveling to Paris other than walking around the city?

I'm sorry, but as an AI language model, I don't have any specific information about your upcoming trip to Paris. However, I can suggest some general tips and recommendations for traveling to Paris other than walking around the city:

1. Plan your itinerary ahead of time to avoid getting lost or getting in over your head.
2. Book your flights or accommodations in advance to avoid being stuck in traffic or waiting for a delayed flight.
3. Purchase a travel insurance policy to protect your belongings and reduce the risk of

Instruction-Tuned Model Output

messages = [
    {"role": "system", "content": "You help travelers make plans for their trips."},
    {"role": "user", "content": "Hello"},
    {"role": "assistant", "content": "Hi there!"},
    {"role": "user", "content": "What should I do on my upcoming trip to Paris?"}
]
instruct_text = instruct_tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
instruct_inputs = instruct_tokenizer(instruct_text, return_tensors="pt")
instruct_outputs = instruct_model.generate(
    **instruct_inputs,
    max_new_tokens=150,
    temperature=0.7,
    do_sample=True,
    pad_token_id=instruct_tokenizer.eos_token_id,
)
instruct_response = instruct_tokenizer.decode(
    instruct_outputs[0], skip_special_tokens=True
)
print(instruct_response)

system
You help travelers make plans for their trips.
user
Hello
assistant
Hi there!
user
What should I do on my upcoming trip to Paris?
assistant
Great question! On your next trip to Paris, you can start by visiting the iconic Eiffel Tower and the Louvre Museum. Don't miss exploring the Notre-Dame Cathedral and its stunning stained glass windows. For a bit of a break, consider visiting Montmartre for some beautiful art and architecture. If you're looking for something more adventurous, you could take a stroll through the charming streets of Montmartre or explore the vibrant nightlife of Le Marais. Have fun planning your trip to Paris!

Extra: Display Qwen’s chat template

messages = [
    {"role": "system", "content": "You help travelers make plans for their trips"},
    {"role": "user", "content": "Hello"},
    {"role": "assistant", "content": "Hi there!"},
    {"role": "user", "content": "What should I do on my upcoming trip to Paris?"}
]

instruct_tokenizer.apply_chat_template(
    messages, 
    tokenize=False,
    add_generation_prompt=True  # Adds the assistant prompt
)

'<|im_start|>system\nYou help travelers make plans for their trips<|im_end|>\n<|im_start|>user\nHello<|im_end|>\n<|im_start|>assistant\nHi there!<|im_end|>\n<|im_start|>user\nWhat should I do on my upcoming trip to Paris?<|im_end|>\n<|im_start|>assistant\n'