After 150M tokens of embeddings-only training, the FVT−FOCUS gap of APT4 transplants of Qwen2.5-1.5B on Polish reviews is 0.1024 bits per byte (95% interval 0.0854 to 0.1242), 0.28 of its time-zero value 0.3652.

Relation: Persistence of a difference

metric

Bits per byteQuantity compared.

model

APT4 transplant of Qwen2.5-1.5BModels measured.

subject

Frequency-based Vocabulary TransferInitialisation of the model whose value is the minuend.

comparator

FOCUSInitialisation of the model whose value is subtracted.

evaluation text

Polish reviewsCorpus scored, first 200 documents.

training tokens

150000000 tokensTokens of embeddings-only continued pretraining, nominal checkpoint size, before the value is measured.

value

0.1024 bits per byteDifference after training.

interval low

0.0854 bits per byteLower bound of the 95% interval.

interval high

0.1242 bits per byteUpper bound of the 95% interval.

baseline value

0.3652 bits per byteDifference at time zero.

ratio

0.28 ratioDifference after training divided by the difference at time zero.

bin

SHRINKSPre-registered class: VANISHED, SHRINKS or PERSISTS.

Method and evaluation protocol

Paired document bootstrap (1000 resamples, seed 20260705) of FVT-minus-FOCUS bits per byte at the 150M-token checkpoints, divided by the time-zero difference.

Dataset

First 200 documents of the evaluation domains, scored at the 150M-token checkpoints of embeddings-only training on the packed APT4 stream.Version: unspecified · Access: restricted

Reported results

0.1024 [0.0854, 0.1242]; time zero 0.3652; ratio 0.28; SHRINKS.

Uncertainty and replication

95% paired document bootstrap interval on the 150M-token value; none on the ratio.

Evidence references

analysis_s2.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/7d450d1f62209ff5ac5415543443d2a25db74b7e/experiments/E07-embedding-init/results/analysis_s2.jsonmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/7d450d1f62209ff5ac5415543443d2a25db74b7e/experiments/E07-embedding-init/results/metrics.json

Limitations

One training run per initialisation at 1.5B. The trainer's loss and gradient-norm logs, the smoke-run gate results, the transfer digests and the token stream's digest were not kept. Windows of 2,048 tokens.