At time zero, the FVT−FOCUS gap of APT4 transplants of Qwen2.5-1.5B is between 0.2873 and 0.4037 bits per byte on four Polish domains and between 0.0129 and 0.1063 on five English and arithmetic domains.

Relation: Range comparison

metric

FVT−FOCUS gapQuantity compared.

model

APT4 transplant of Qwen2.5-1.5BModels measured.

subject domains

Polish Wikipedia science articles; Polish PES examination questions; the Polish FineWeb2-HQ holdout; Polish reviewsFirst set of domains.

subject min

0.2873 bits per byteLowest value on the first set.

subject max

0.4037 bits per byteHighest value on the first set.

comparator domains

English LaTeX method sections; English Python code; math_clean statements; English arXiv abstracts; the English SlimPajama holdoutSecond set of domains.

comparator min

0.0129 bits per byteLowest value on the second set.

comparator max

0.1063 bits per byteHighest value on the second set.

setting

time zero: no training after embedding initialisation; tokenizers loaded through a whitespace canary with the pristine APT4 tokenizer as fallbackTraining state and scoring condition.

Method and evaluation protocol

FVT-minus-FOCUS bits per byte per domain, grouped by language.

Dataset

First 200 documents of each of the nine primary evaluation domains: the Polish FineWeb2-HQ holdout, the English SlimPajama holdout, Polish reviews, Polish Wikipedia science articles, Polish PES examination questions, English arXiv abstracts, English LaTeX method sections, English Python code and math_clean statements.Version: unspecified · Access: restricted

Reported results

sci_latex: 0.0686 [0.0579, 0.0788]; sci_python: 0.1063 [0.0941, 0.1195]; math_clean: 0.014 [-0.0003, 0.0277]; sci_arxiv: 0.0129 [-0.0028, 0.027]; pl_wiki_sci: 0.4037 [0.3906, 0.4174]; pl_pes: 0.2873 [0.2627, 0.3096]; pl: 0.3844 [0.3626, 0.4031]; en: 0.0668 [0.0514, 0.0793]; pl_informal: 0.3652 [0.3326, 0.3902]

Uncertainty and replication

95% paired document bootstrap interval per domain.

Evidence references

analysis.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/e7d4bb2df26254f88ecbf390a7cbf69ac210e0e3/experiments/E07-embedding-init/results/analysis.jsonmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/e7d4bb2df26254f88ecbf390a7cbf69ac210e0e3/experiments/E07-embedding-init/results/metrics.json

Limitations

The auxiliary embeddings are monolingual Polish. The auxiliary fastText text (the first 209,715,204 bytes of the Polish FineWeb2-HQ training text) and the Polish FineWeb2-HQ holdout were drawn from the same source without a shared exclusion: a line check in the port found that 3 of the 180 scored holdout documents with lines of 200 to 4,000 bytes share at least one such line with the auxiliary text, and none shares half of them. The design text was first committed together with the stage-1 results. Init damage compares APT4 transplants with the base model under its own tokenizer at 2,048-token windows, which cover different byte spans, and fertility inflates absolute damage on English formal text. One build and one scoring run per initialisation at 1.5B.