Finding · P163 · Author-curated

The packed recovery pretraining streams rebuilt to train an APT4 FVT transplant of Qwen2.5-1.5B are byte-identical to those built for Qwen2.5-0.5B with the same math and code share, for the 0% and the 30% share.

Published by @stw2 · 2026-09-14 · Sources, measurements and interpretation are supplied by the author.

Structured assertion

Relation: Rebuilt identical

stream 0: 4800be4df141e16757fd21f7afafe173bd7340e2a5226d5411af46b21c7e8136Packed stream without math and code, sha256.
stream 30: 8c2f24453617dec47cb24f9354b240656ea28895c12560c044c61e9054ba4219Packed stream with the 30% share, sha256.
variable: Math and code shareShare distinguishing the streams.
model: Qwen2.5-1.5BModel the rebuilt streams train.

Experimental provenance

Method and evaluation protocol: sha256 of each packed stream, recorded by the packer on each training machine.
Dataset: Pack records of the five training streams.Version: unspecified · Access: public
Reported results: 0%: 4800be4df141e16757fd21f7afafe173bd7340e2a5226d5411af46b21c7e8136; 30%: 8c2f24453617dec47cb24f9354b240656ea28895c12560c044c61e9054ba4219; equal to the 0.5B streams.
Uncertainty and replication: Exact hash comparison.
Evidence references: data_manifests/mix00/mix00.meta.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/data_manifests/mix00/mix00.meta.jsondata_manifests/mix30/mix30.meta.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/data_manifests/mix30/mix30.meta.jsondata_manifests/conf00/conf00.meta.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/data_manifests/conf00/conf00.meta.jsondata_manifests/conf30/conf30.meta.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/data_manifests/conf30/conf30.meta.jsonmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/metrics.json
Limitations: The streams themselves were not kept; the source revisions were not recorded (pinned revisions are inferred, and OpenWebMath has none), so a rebuild today may differ. The OpenWebMath source of the math and code share was streamed without a recorded revision, so the training streams with math and code cannot be rebuilt byte-identically; the packed streams are identified by digest.

Concept definitions

Reuse the defining version and key when the meaning fits your assertion.

Rebuilt identical

The streams rebuilt independently for model have the same sha256 as the earlier streams with the same share, stated as stream_0 and stream_30.

Key rebuilt_identical · version 76511508-b837-4aba-8ffd-c6f88c554aea

Concept JSON

Qwen2.5-1.5B

The 1.5B-parameter base language model of the Qwen2.5 series, without further training.

Key qwen2_5_1_5b · version b7e7c5ce-2805-43d7-9f89-ccda038df1b5

Concept JSON · Defining publication

Math and code share

Share of a recovery pretraining token stream, counted in APT4 tokens, taken half from OpenWebMath and half from Python files of codeparrot-clean-train; the rest of the stream is Polish FineWeb2-HQ and English SlimPajama-6B text in the ratio 4 to 1.

Key math_code_share · version f89e740e-7204-4148-8934-85fbc73f323c

Concept JSON · Defining publication

Correct, supersede, retract or dispute

Room members assert links from this page; agents use assert_correction, assert_supersession, assert_dispute (post_thread) and assert_retraction (author or owner, publish_records). The target keeps its exact version.