At time zero, the mean cross-entropy of APT4 transplants of Qwen2.5-1.5B on a 40-token Polish sentence is 8.8109 nats for FOCUS, 10.3366 for FVT and 13.7575 for random initialisation.

Relation: Values by initialisation

metric

mean next-token cross-entropy in natsQuantity measured.

model

APT4 transplant of Qwen2.5-1.5BModels measured.

evaluation text

one Polish sentence of 40 APT4 tokens about tokenization and scientific abilityText measured.

setting

time zero: no training after embedding initialisationTraining state of the models.

focus value

8.8109 natsValue for FOCUS initialisation.

fvt value

10.3366 natsValue for FVT initialisation.

random value

13.7575 natsValue for random initialisation.

Method and evaluation protocol

Forward pass of each freshly built transplant on one sentence during the build, with the APT4 tokenizer loaded from its own directory.

Dataset

One Polish sentence fixed in scripts/02_build_inits.py.Version: unspecified · Access: public

Reported results

FOCUS 8.8109, FVT 10.3366, random 13.7575.

Uncertainty and replication

Single sentence; no interval.

Evidence references

build_stats/focus.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/884186c96aedece6ef61da4e12e239ef5df929a5/experiments/E07-embedding-init/results/build_stats/focus.jsonbuild_stats/fvt.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/884186c96aedece6ef61da4e12e239ef5df929a5/experiments/E07-embedding-init/results/build_stats/fvt.jsonbuild_stats/random.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/884186c96aedece6ef61da4e12e239ef5df929a5/experiments/E07-embedding-init/results/build_stats/random.json

Limitations

One sentence; a sanity check of the builds, not an endpoint. The build statistics were copied from the restricted model directories into the repository in the port.