Skip to content

The medium is not visibly moving, at least not the part the probe reads

scripts/separability_over_time.py, datasets/results/style-drift.json. Two readings of whether the corpus drifts under the study, prompted by the naming-convention shift Xu et al. measure.

The cross-organization series is not available here, and that is worth stating. Holding content fixed needs a suffix both organizations carry, and the only one is Python, which Qt stops writing during the window: 39 to 122 examples a month in late 2024 against 0 to 31 through 2025, with two quarters at zero. The accuracies that come out (0.880, then 0.656, then 0.724) track that collapse, not the organizations' hands, and no other suffix reaches fifteen examples a month on both sides in more than two months. A time series of organizational separability is simply not measurable on this corpus.

Within each organization, its earliest quarter against its latest is at chance, on its own dominant suffix, whole changes held out together:

organization 2024-10 to 12 against 2025-08 to 10 changes accuracy
OpenStack (.py) ten months apart 369 0.552 [0.466, 0.602]
Qt (.cpp) ten months apart 986 0.510 [0.487, 0.572]

Both intervals cover 0.5. Over the span that separates RQ1's training window from its test window, the review text a classifier can read does not identify which end of the year it came from.

The code half is at chance too (style-drift-code.json). probe.code_shapes reads the refinement itself as convention shapes rather than identifiers, since identifiers name a project's subject matter while shapes are how it writes them: snake_case, camelCase, PascalCase, SCREAMING case, dunders, arrows, scope resolution, f-strings, braces, line-final semicolons, each occurrence one token. On the same quarters:

organization changes accuracy
OpenStack (.py) 308 0.511 [0.434, 0.562]
Qt (.cpp) 887 0.513 [0.453, 0.546]

So neither the reviewers' words nor the refinements' conventions identify which end of the year they came from. The drift Xu et al. measure is over three years and across 20,000 repositories with finer features than ten regular expressions; this says only that within one year, in these two organizations, on the medium this study actually trains on, the ground is not visibly moving. That is enough for the train-to-test transfer and not enough to contradict them.