At time zero, the formal-to-prose ratio of mean init damage of APT4 transplants of Qwen2.5-1.5B is 2.032 for random and 1.234 for FVT initialisation; both exceed 1.15.

Relation: Ratios against a threshold

metric

Init damageQuantity averaged over each group of domains.

model

APT4 transplant of Qwen2.5-1.5BModels measured.

subject domains

Formal domainsDomains whose mean is the numerator.

comparator domains

Prose domainsDomains whose mean is the denominator.

setting

time zero: no training after embedding initialisation; tokenizers loaded through a whitespace canary with the pristine APT4 tokenizer as fallbackTraining state and scoring condition.

random ratio

2.032 ratioRatio for random initialisation.

fvt ratio

1.234 ratioRatio for FVT initialisation.

threshold

1.15 ratioValue both ratios are compared with.

both exceed threshold

trueWhether both ratios exceed the threshold.

Method and evaluation protocol

Init damage per domain is transplant bits per byte minus base bits per byte on the first 200 documents; means over the three formal and the six prose domains are divided, per initialisation.

Dataset

First 200 documents of each of the nine primary evaluation domains: the Polish FineWeb2-HQ holdout, the English SlimPajama holdout, Polish reviews, Polish Wikipedia science articles, Polish PES examination questions, English arXiv abstracts, English LaTeX method sections, English Python code and math_clean statements.Version: unspecified · Access: restricted

Reported results

Random 7.7963 / 3.8367 = 2.032; FVT 2.0486 / 1.6604 = 1.234; threshold 1.15 for both.

Uncertainty and replication

Point values; the design's bootstrap intervals on pooled means were not computed.

Evidence references

analysis.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/e7d4bb2df26254f88ecbf390a7cbf69ac210e0e3/experiments/E07-embedding-init/results/analysis.jsonmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/e7d4bb2df26254f88ecbf390a7cbf69ac210e0e3/experiments/E07-embedding-init/results/metrics.json

Limitations

The design text was first committed together with the stage-1 results. Init damage compares APT4 transplants with the base model under its own tokenizer at 2,048-token windows, which cover different byte spans, and fertility inflates absolute damage on English formal text. One build and one scoring run per initialisation at 1.5B. The erratum amendment was written together with the re-scored rows.