Skip to content

What the base model's own report says about its training data

The contamination battery compares a post-cutoff window against a pre-cutoff control, so the cutoff is load-bearing and had never been sourced. Search returns "June 2024" for Qwen2.5-Coder with some confidence. It does not survive the primary source: in QwenLM/Qwen3 discussion 1093 a user asks whether the cutoff is June 2024, a maintainer pings two team members, neither answers, and a third participant offers June as a personal opinion. No Qwen maintainer states a date. It is a community guess with a citation-shaped shadow, and the battery must not rest on it.

The technical report does make a sourceable statement. Qwen2.5-Coder (arXiv:2409.12186, 3.1.1 Data Composition):

Source Code We collected public repositories from GitHub created before February 2024, spanning 92 programming languages. ... In addition to raw code, we also collected data from Pull Requests, Commits, Jupyter Notebooks, and Kaggle datasets

Three things follow, and two of them cut against the study.

  • That is a repository-creation filter, not a content cutoff. A repository created in 2011 and crawled in mid-2024 carries 2024 content. OpenStack and Qt are both far older than the filter, so it bounds nothing about how recent their content may be. The report gives no content cutoff, and the honest registered statement is that it is unstated.
  • Pull requests and commits are named as sources. The refinement targets are post-comment code, which is exactly what a commit contains, so the targets are plausibly exposed even though the review comments are not.
  • Gerrit is not GitHub. OpenStack reviews live on review.opendev.org and Qt's on codereview.qt-project.org. The inline review comments that make up the prompt side of every example are not on GitHub in any form. Only the code side has a plausible route into pretraining.

What this does not change. The post-cutoff window starts 2024-10-01 and the checkpoint was published 2024-09-17, so the post arm is clean by construction rather than by the report's word: that data did not exist when the model shipped. Only the control depends on the cutoff, and it depends on it in the safe direction, since a control that turns out not to be in training weakens the contrast rather than manufacturing one.

What it changes. The control moves back to a six-month window ending 2024-01, which sits comfortably inside any reading of the evidence, instead of the single month that ended one month before the stated repository boundary.