Analysis: Qwen 3.8's Alignment with GPT-5.5 Pro Reasoning Prefills

This report serves as version 1.1 of the "Reasoning prefills on a few open models" study, acting as a sequel to the original research and the Stolen Thoughts paper. The primary objective of this iteration was to rerun the experimental framework using GPT-5.5 Pro as the primary teacher model.

🛠️ Experimental Methodology

The researcher employed a specific technique to test if target models "follow" the reasoning path of a superior model.

The Process:

• Generate a standard response from the target model (Unprefilled).
• Generate a second response where the target model's reasoning channel is seeded with the first 1% of GPT-5.5 Pro's reasoning.
• Analyze the first 100 tokens of the target model's final answer.
• Calculate the overlap with the teacher's visible answer.

The scoring metric is defined as the average of the following: $\text{Score} = \frac{\text{Unigram Recall} + \text{Bigram Recall} + \text{Trigram Recall}}{3}$

# Conceptual logic for the prefill experiment
def evaluate_prefill(teacher_reasoning, target_model):
    prefill_seed = teacher_reasoning[:int(len(teacher_reasoning) * 0.01)]
    response = target_model.generate(prompt=prefill_seed)
    return calculate_source_recall(response[:100], teacher_answer)

📊 Results: General Model Comparison

The evaluation utilized a dataset of $n=45$ problems, split evenly between STEM, non-STEM, and synthetic puzzles.

🔍 Deep Dive: Qwen 3.8 Performance

When breaking down the results for Qwen3.8 A95B by category, the impact of the prefill becomes even more pronounced:

• STEM (15 problems): $19.26\% \rightarrow 46.24\%$ ($\Delta +26.99\text{ pp}$)
• Non-STEM (15 problems): $20.62\% \rightarrow 33.42\%$ ($\Delta +12.80\text{ pp}$)
• Puzzle (15 problems): $10.49\% \rightarrow 25.23\%$ ($\Delta +14.75\text{ pp}$)
• Total Average: $16.79\% \rightarrow 34.97\%$ ($\Delta +18.18\text{ pp}$)

Workflow Visualization

📝 Discussion and Conclusions

The most striking finding is that while Qwen showed negligible movement toward Opus 4.8 in previous tests, it shifted significantly toward GPT-5.5 Pro in this trial—particularly within the private synthetic puzzle set.

This suggests a strong possibility regarding the training data: Qwen learned from Opus. $\rightarrow$ Qwen likely learned from GPT-5.5 Pro (or a closely related GPT-family model).

Additionally, it is noted that Kimi K3 maintains the highest overall overlap with GPT-5.5 Pro regardless of the prefill (peaking at 35.65%), even though the prefill itself provided a relatively small boost of only $+4.54$ points.