Reanalysis of the released scores for 800 Polish documents from the 1.5B APT4 recovery-training confirmation finds a 30%-versus-0% mathematics/code cost of 0.015635 bits per byte. A paired document bootstrap gives a 95% interval of [0.011866, 0.018028], entirely below the 0.05 bits-per-byte bound.

Relation: Difference value

subject

APT4 FVT transplantModel family recovered.

base model

Qwen2.5-1.5BBase model of the transplant and reference of the gaps.

procedure

Continued pretrainingTraining applied.

variable

Math and code shareQuantity varied.

treatment value

30 percentMath and code share of the treatment.

control value

0 percentMath and code share of the control.

budget

1000000000 tokensRecovery pretraining tokens per arm.

metric

Pooled Polish bits per byteMetric compared.

value

0.01563485397787734 bits per byteTreatment minus control.

interval low

0.011866223713689613 bits per byteLower bound of the 95% bootstrap interval.

interval high

0.01802752032630769 bits per byteUpper bound of the 95% bootstrap interval.

Method and evaluation protocol

Computational reanalysis of published NPZ arrays, without model inference or training. Verify every input against the pinned upstream Git blob and Room SHA-256 where available. Require 200 finite per-document NLLs and positive integer byte counts for each of pl, pl_wiki_sci, pl_pes, pl_informal; require identical byte-count vectors across arms. Concatenate in that order. Calculate cost as sum(NLL_30)/ln(2)/sum(bytes_30) minus sum(NLL_00)/ln(2)/sum(bytes_00). Draw 2,000 paired bootstrap samples of 800 document indices with replacement, using a freshly initialized numpy.random.default_rng(20260707), and take linear 2.5th/97.5th percentiles. All analysis runs on local arm64 CPU with Python 3.12.13 and NumPy 1.26.3. Local design was committed before score download and computation; no Substrate attempt was registered for these historical local runs.

Dataset

Public per-document NLL and byte-count arrays for conf00_final and conf30_final, four Polish splits of 200 documents each. Underlying document text is restricted; this analysis uses public score arrays only.Version: stw2/tokenizer-science-tax commit 2457520fc798b19bee6893ac58e0357ec4c467d5, experiments/E06-math-code-rescue/results/perdoc · Access: public

Reported results

Cost 0.01563485397787734 bits per byte; paired 95% interval [0.011866223713689613, 0.01802752032630769]. Under E25's upper-bound rule <= 0.05, CONFIRMED for these fixed checkpoints and released evaluation scores. Original unpaired interval [-0.029218796616256135, 0.06033200376079382] is reproduced with the same point estimate. Original-RNG-position paired sensitivity interval [0.01189440246694698, 0.0180368912513535]; 20,000-resample corpus-stratified paired sensitivity [0.012062609874164515, 0.018017746484179514]. Independent math.fsum/count-weighted implementation agrees on all 2,000 primary replicates, and an offline rerun produces byte-identical deterministic deliverables.

Uncertainty and replication

95% percentile interval from a paired pooled-document bootstrap. Describes document sampling conditional on the supplied checkpoints, not training-run variance. The primary E25 analysis freshly seeds this endpoint; a sensitivity retains the original program's prior RNG consumption, with the same decision.

Evidence references

Local E6 reproduction and E25 results · restrictedLocal scitok-replication repository, commit abfdba89f94c4963ce424f9996f7916f4075fdfe, outputs/results.json; no public location registered.Deterministic rerun checks · restrictedLocal scitok-replication repository, commit abfdba89f94c4963ce424f9996f7916f4075fdfe, outputs/repeatability.json; no public location registered.perdoc/conf00_final__pl.npz · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/perdoc/conf00_final__pl.npzperdoc/conf00_final__pl_wiki_sci.npz · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/perdoc/conf00_final__pl_wiki_sci.npzperdoc/conf00_final__pl_pes.npz · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/perdoc/conf00_final__pl_pes.npzperdoc/conf00_final__pl_informal.npz · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/perdoc/conf00_final__pl_informal.npzperdoc/conf30_final__pl.npz · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/perdoc/conf30_final__pl.npzperdoc/conf30_final__pl_wiki_sci.npz · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/perdoc/conf30_final__pl_wiki_sci.npzperdoc/conf30_final__pl_pes.npz · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/perdoc/conf30_final__pl_pes.npzperdoc/conf30_final__pl_informal.npz · publichttps://github.com/stw2/tokenizer-science-tax/blob/2457520fc798b19bee6893ac58e0357ec4c467d5/experiments/E06-math-code-rescue/results/perdoc/conf30_final__pl_informal.npz

Limitations

Reproduces statistics from released scores, not model training, inference, tokenizer behavior or corpus construction. One original training run per arm. NPZs have row order and byte counts but no document hashes: exact byte-vector equality supports the reported pairing but cannot independently establish text identity. Raw text is restricted. Byte-weighted pooling emphasizes longer documents. Findings do not establish downstream task accuracy, all-Polish generalization, or elimination of a tokenizer science tax. The analysis code and new outputs are currently local, with restricted evidence references; no public reproduction-code commit is available.