At time zero, the mean FVT−FOCUS gap of APT4 transplants of Qwen2.5-1.5B is 0.063 bits per byte over formal domains and 0.2534 over prose domains.

Relation: Group mean comparison

metric

FVT−FOCUS gapQuantity averaged over each group of domains.

model

APT4 transplant of Qwen2.5-1.5BModels measured.

subject domains

Formal domainsFirst group of domains.

subject value

0.063 bits per byteMean over the first group.

comparator domains

Prose domainsSecond group of domains.

comparator value

0.2534 bits per byteMean over the second group.

setting

time zero: no training after embedding initialisation; tokenizers loaded through a whitespace canary with the pristine APT4 tokenizer as fallbackTraining state and scoring condition.

Method and evaluation protocol

Mean over the three formal and the six prose domains of FVT-minus-FOCUS bits per byte.

Dataset

First 200 documents of each of the nine primary evaluation domains: the Polish FineWeb2-HQ holdout, the English SlimPajama holdout, Polish reviews, Polish Wikipedia science articles, Polish PES examination questions, English arXiv abstracts, English LaTeX method sections, English Python code and math_clean statements.Version: unspecified · Access: restricted

Reported results

formal 0.063, prose 0.2534.

Uncertainty and replication

Point values; the design named a sign test, which was implemented as this comparison of means.

Evidence references

analysis.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/e7d4bb2df26254f88ecbf390a7cbf69ac210e0e3/experiments/E07-embedding-init/results/analysis.jsonmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/e7d4bb2df26254f88ecbf390a7cbf69ac210e0e3/experiments/E07-embedding-init/results/metrics.json

Limitations

The design text was first committed together with the stage-1 results. Init damage compares APT4 transplants with the base model under its own tokenizer at 2,048-token windows, which cover different byte spans, and fertility inflates absolute damage on English formal text. One build and one scoring run per initialisation at 1.5B.