After 150M tokens of embeddings-only training, the FVT−FOCUS gap of APT4 transplants of Qwen2.5-1.5B is VANISHED on 5, SHRINKS on 2 and PERSISTS on 0 of 7 domains whose time-zero interval excludes zero.

Relation: Counts per persistence bin

metric

Bits per byteQuantity compared.

model

APT4 transplant of Qwen2.5-1.5BModels measured.

subject

Frequency-based Vocabulary TransferInitialisation whose value is the minuend.

comparator

FOCUSInitialisation whose value is subtracted.

domains

the English SlimPajama holdout; English LaTeX method sections; English Python code; Polish Wikipedia science articles; the Polish FineWeb2-HQ holdout; Polish reviews; Polish PES examination questionsDomains classified.

training tokens

150000000 tokensTokens of embeddings-only continued pretraining, nominal checkpoint size, before the value is measured.

count total

7 domainsDomains classified.

count vanished

5 domainsDomains in VANISHED.

count shrinks

2 domainsDomains in SHRINKS.

count persists

0 domainsDomains in PERSISTS.

Method and evaluation protocol

Count the pre-registered per-domain persistence bins; the design has no class verdict for this endpoint.

Dataset

First 200 documents of the evaluation domains, scored at the 150M-token checkpoints of embeddings-only training on the packed APT4 stream.Version: unspecified · Access: restricted

Reported results

en: VANISHED; sci_latex: VANISHED; sci_python: VANISHED; pl_wiki_sci: VANISHED; pl: VANISHED; pl_informal: SHRINKS; pl_pes: SHRINKS

Uncertainty and replication

Per-domain 95% paired bootstrap intervals.

Evidence references

analysis_s2.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/7d450d1f62209ff5ac5415543443d2a25db74b7e/experiments/E07-embedding-init/results/analysis_s2.jsonmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/7d450d1f62209ff5ac5415543443d2a25db74b7e/experiments/E07-embedding-init/results/metrics.json

Limitations

One training run per initialisation at 1.5B. The trainer's loss and gradient-norm logs, the smoke-run gate results, the transfer digests and the token stream's digest were not kept. Windows of 2,048 tokens.