Finding

Sign in with GitHub
← Publications

Finding · P229 · Author-curated

On the English preamble, the fertility tax against the Mistral-derived tokenizer is 1.2108 for the science-slice 32k tokenizer, 1.2108 with whitespace pieces, 1.0711 for the balanced 32k tokenizer, 1.4828 for the Polish-only 32k tokenizer and 1.5466 for APT4.

Published by @stw2 · 2026-09-14 · Sources, measurements and interpretation are supplied by the author.

Structured assertion

Relation: Values per tokenizer

metric
Fertility taxQuantity measured.
reference
Mistral-derived tokenizerTokenizer each value is relative to.
value science slice
1.2108 ratioValue for the science-slice 32k tokenizer.
value science slice whitespace
1.2108 ratioValue for the science-slice 32k tokenizer with whitespace pieces.
value balanced
1.0711 ratioValue for the balanced 32k tokenizer.
value polish only
1.4828 ratioValue for the Polish-only 32k tokenizer.
value apt4
1.5466 ratioValue for APT4.
evaluation text
Constitution preambleText measured.
language
EnglishLanguage of the text.

Experimental provenance

Method and evaluation protocol
Tokens of each tokenizer on the English preamble protocol text divided by the Mistral-derived tokenizer's tokens.
Dataset
Official English translation of the preamble of the Constitution of the Republic of Poland, protocol text.Version: unspecified · Access: public
Reported results
e9-sci 1.2108; e9-sci-ws 1.2108; e9-bal 1.0711; e9-pl 1.4828; apt4 1.5466
Uncertainty and replication
Single text; no interval.
Limitations
One short formal text is the only general-English reference measured; the balanced arm's comparison is exploratory.

Paper citations

arXiv:2604.10799v1 →Revision supplied by author

Associated through an explicitly referenced cited claim.

Concept definitions

Reuse the defining version and key when the meaning fits your assertion.

Values per tokenizer

The metric, relative to the reference tokenizer on the evaluation text, takes value_science_slice for the science-slice 32k tokenizer, value_science_slice_whitespace for the science-slice 32k tokenizer with whitespace pieces, value_balanced for the balanced 32k tokenizer, value_polish_only for the Polish-only 32k tokenizer and value_apt4 for APT4.

Key values_per_tokenizer · version c212f70e-53ad-49af-8989-afba67519c2f

Concept JSON · Defining publication

Constitution preamble

Preamble of the Constitution of the Republic of Poland, in Polish and in its official English translation.

Key constitution_preamble · version 86acabda-0237-47be-8e26-a81500c184aa

Concept JSON · Defining publication

Fertility tax

Tokens per word of the subject tokenizer divided by tokens per word of the comparator tokenizer on the same text.

Key fertility_tax · version 1de64025-ce26-4aa5-a17b-0ebb58006700

Concept JSON · Defining publication

Mistral-derived tokenizer

Tokenizer of the original Bielik v3 models, derived from Mistral's.

Key mistral_tokenizer · version bd3d8329-e626-4805-9fff-f83f146769a7

Concept JSON · Defining publication

Exact references

extends

eed1981a-09e9-4aa9-bd57-3db32c7ba4a0

APT4's and the Mistral-derived tokenizer's values on the same text.