After 150M tokens of embeddings-only training, the FVT−FOCUS gap of APT4 transplants of Qwen2.5-1.5B on Polish Wikipedia science articles is 0.0675 bits per byte (95% interval 0.0623 to 0.0722), 0.167 of its time-zero value 0.4037.

Relation: Persistence of a difference

metric

Bits per byteQuantity compared.

model

APT4 transplant of Qwen2.5-1.5BModels measured.

subject

Frequency-based Vocabulary TransferInitialisation of the model whose value is the minuend.

comparator

FOCUSInitialisation of the model whose value is subtracted.

evaluation text

Polish Wikipedia science articlesCorpus scored, first 200 documents.

training tokens

150000000 tokensTokens of embeddings-only continued pretraining, nominal checkpoint size, before the value is measured.

value

0.0675 bits per byteDifference after training.

interval low

0.0623 bits per byteLower bound of the 95% interval.

interval high

0.0722 bits per byteUpper bound of the 95% interval.

baseline value

0.4037 bits per byteDifference at time zero.

ratio

0.167 ratioDifference after training divided by the difference at time zero.

bin

VANISHEDPre-registered class: VANISHED, SHRINKS or PERSISTS.

Method and evaluation protocol

Paired document bootstrap (1000 resamples, seed 20260705) of FVT-minus-FOCUS bits per byte at the 150M-token checkpoints, divided by the time-zero difference.

Dataset

First 200 documents of the evaluation domains, scored at the 150M-token checkpoints of embeddings-only training on the packed APT4 stream.Version: unspecified · Access: restricted

Reported results

0.0675 [0.0623, 0.0722]; time zero 0.4037; ratio 0.167; VANISHED.

Uncertainty and replication

95% paired document bootstrap interval on the 150M-token value; none on the ratio.

Evidence references

analysis_s2.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/7d450d1f62209ff5ac5415543443d2a25db74b7e/experiments/E07-embedding-init/results/analysis_s2.jsonmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/7d450d1f62209ff5ac5415543443d2a25db74b7e/experiments/E07-embedding-init/results/metrics.json

Limitations

One training run per initialisation at 1.5B. The trainer's loss and gradient-norm logs, the smoke-run gate results, the transfer digests and the token stream's digest were not kept. Windows of 2,048 tokens.