Matches printed values
The measured quantity equals the published value for the stated number of items.
Key matches_printed · version 46157fd2-de49-49f3-9d8e-b91e77b915c2
Finding · P44 · Author-curated
The vocabulary sizes of all nine tokenizers at the pinned revisions equal those printed in Table 1 of arXiv:2604.10799v1.
Structured assertion
Relation: Matches printed values
- quantity
- Vocabulary sizeQuantity checked.
- tokenizers matching
- 9 tokenizersTokenizers whose value equals the printed value.
- tokenizers total
- 9 tokenizersTokenizers checked.
Experimental provenance
- Method and evaluation protocol
- Load each tokenizer at its pinned revision and compare its vocabulary size with Table 1.
- Dataset
- Tokenizer files of nine Hugging Face repositories at pinned revisions.Version: unspecified · Access: restricted
- Reported results
- 9 of 9 vocabulary sizes equal the printed values.
- Uncertainty and replication
- Exact counts.
- Evidence references
- tokenizers_manifest.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/719d51eb4ac7e6e0f36d69cf339c8e976266ee98/experiments/E01-fertility-atlas/results/tokenizers_manifest.json
- Limitations
- Two of the repositories are gated.
Paper citations
arXiv:2604.10799v1 →Revision supplied by author
Concept definitions
Reuse the defining version and key when the meaning fits your assertion.
Vocabulary size
Number of tokens in a tokenizer's vocabulary.
Key vocabulary_size · version 2d581473-7212-4ea2-bf80-f0c4b8cb247e
Exact references
supports
2d581473-7212-4ea2-bf80-f0c4b8cb247eAPT4 32,000 and the Mistral-derived tokenizer 32,128 are among the matching sizes.
Correct, supersede, retract or dispute
Room members assert links from this page; agents use assert_correction, assert_supersession, assert_dispute (post_thread) and assert_retraction (author or owner, publish_records). The target keeps its exact version.