Execution attempt · A77 · planned
Builds the English, Polish and other vocabulary partitions of both tokenizers from five corpora and checks golden pieces.
Pinned source and configuration
https://github.com/stw2/tokenizer-science-tax @ 6a20c5c41c509137b0de80a80ec4fd3d3243ba56
Reference checked 2026-09-14 13:12 UTC. Later commits, branches or plan changes do not retarget this attempt.
- Command
- python scripts/01_build_lexicons.py
- Working directory
- experiments/E08-latent-pivot
- Configuration paths
- None
- Parameters
- min_count 5; rate_ratio 10; Polish-diacritic override; 150 traces per condition for the label mix; seed 20260704 for the audit samples
- Environment
- 3.12.13; tokenizers through mlx_lm.utils.load_tokenizer; CPU; Apple silicon, 128 GB unified memory
- Output directory
- Not recorded
Inputs
Materials the registrant named when registering this attempt, by content identity; obtainability is derived from their location reports. Nothing is fetched or verified.
- corpus en-gsm8k · reference corpus · M30 en-gsm8k.jsonl · raw output · Download
- corpus en-arxiv-abstracts · reference corpus · M27 en-arxiv-abstracts.jsonl · raw output · Download
- corpus pl-wiki-science · reference corpus · M32 pl-wiki-science.jsonl · raw output · Download
- corpus pl-informal · reference corpus · M33 pl-informal.jsonl · raw output · Ask the reporter
- corpus pl-science-pes · reference corpus · M31 pl-science-pes.jsonl · raw output · Ask the reporter
- MLX tokenizer.json bielik-pl-11b-v3.0-instruct · tokenizer · M394 MLX tokenizer.json of Bielik-PL-11B-v3.0-Instruct · tokenizer · Ask the reporter
- MLX tokenizer.json bielik-11b-v3.0-instruct · tokenizer · M395 MLX tokenizer.json of Bielik-11B-v3.0-Instruct · tokenizer · Ask the reporter
- traces bielik-pl-11b-v3.0-instruct · evaluation data · M99 raw/run_bielik-pl-11b-v3.0-instruct.jsonl · raw output · Ask the reporter
- traces bielik-11b-v3.0-instruct · evaluation data · M100 raw/run_bielik-11b-v3.0-instruct.jsonl · raw output · Ask the reporter
Delivered events
Registered
#1Builds the English, Polish and other vocabulary partitions of both tokenizers from five corpora and checks golden pieces.
Started
#2Started.
Succeeded
#3Completed with exit code 0.
Exit code 0.
- lexicon_bielik-pl-11b-v3.0-instruct.json · https://github.com/stw2/tokenizer-science-tax/blob/6a20c5c41c509137b0de80a80ec4fd3d3243ba56/experiments/E08-latent-pivot/data/lexicon_bielik-pl-11b-v3.0-instruct.json · public · sha256 81f559e815cb… · 96712 bytes · M398
- lexicon_bielik-pl-11b-v3.0-instruct_audit.txt · https://github.com/stw2/tokenizer-science-tax/blob/6a20c5c41c509137b0de80a80ec4fd3d3243ba56/experiments/E08-latent-pivot/data/lexicon_bielik-pl-11b-v3.0-instruct_audit.txt · public · sha256 a02ee690501c… · 1841 bytes · M399
- lexicon_bielik-11b-v3.0-instruct.json · https://github.com/stw2/tokenizer-science-tax/blob/6a20c5c41c509137b0de80a80ec4fd3d3243ba56/experiments/E08-latent-pivot/data/lexicon_bielik-11b-v3.0-instruct.json · public · sha256 e902a73015b2… · 97095 bytes · M400
- lexicon_bielik-11b-v3.0-instruct_audit.txt · https://github.com/stw2/tokenizer-science-tax/blob/6a20c5c41c509137b0de80a80ec4fd3d3243ba56/experiments/E08-latent-pivot/data/lexicon_bielik-11b-v3.0-instruct_audit.txt · public · sha256 4e3dd9dc0862… · 1685 bytes · M401
- lexicons_report.json · https://github.com/stw2/tokenizer-science-tax/blob/6a20c5c41c509137b0de80a80ec4fd3d3243ba56/experiments/E08-latent-pivot/data/lexicons_report.json · public · sha256 a262abef4ba5… · 1167 bytes · M402
Report an event
The registrant’s capture tool normally delivers start and outcome events. Reporting here is the same author report with the browser as the reported time; it does not observe the process.
This attempt has a delivered outcome. A new execution is a new attempt.