At time zero, the mean init-quality spread of APT4 transplants of Qwen2.5-1.5B is 5.8107 bits per byte over formal domains and 2.4297 over prose domains, a ratio of 2.392.

Relation: Group mean comparison

metric

Init-quality spreadQuantity averaged over each group of domains.

model

APT4 transplant of Qwen2.5-1.5BModels measured.

subject domains

Formal domainsFirst group of domains.

subject value

5.8107 bits per byteMean over the first group.

comparator domains

Prose domainsSecond group of domains.

comparator value

2.4297 bits per byteMean over the second group.

ratio

2.392 ratioFirst mean divided by the second.

setting

time zero: no training after embedding initialisation; tokenizers loaded through a whitespace canary with the pristine APT4 tokenizer as fallbackTraining state and scoring condition.

Method and evaluation protocol

Mean over the three formal and the six prose domains of random-minus-FOCUS bits per byte, divided.

Dataset

First 200 documents of each of the nine primary evaluation domains: the Polish FineWeb2-HQ holdout, the English SlimPajama holdout, Polish reviews, Polish Wikipedia science articles, Polish PES examination questions, English arXiv abstracts, English LaTeX method sections, English Python code and math_clean statements.Version: unspecified · Access: restricted

Reported results

5.8107 / 2.4297 = 2.392; per-domain spreads from 1.5445 to 7.9292 with 95% paired bootstrap intervals in results/analysis.json.

Uncertainty and replication

Per-domain 95% paired document bootstrap intervals; no interval on the pooled means.

Evidence references

analysis.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/e7d4bb2df26254f88ecbf390a7cbf69ac210e0e3/experiments/E07-embedding-init/results/analysis.jsonmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/e7d4bb2df26254f88ecbf390a7cbf69ac210e0e3/experiments/E07-embedding-init/results/metrics.json

Limitations

The design text was first committed together with the stage-1 results. Init damage compares APT4 transplants with the base model under its own tokenizer at 2,048-token windows, which cover different byte spans, and fertility inflates absolute damage on English formal text. One build and one scoring run per initialisation at 1.5B.