On 766 of 800 Polish STEM questions with Polish chain-of-thought, the mean pivot excess of Bielik-11B-v3.0-Instruct is 0.6899, with a 95% bootstrap interval from 0.6858 to 0.6938 (bootstrap p-value 0 with 2,000 resamples).

Relation: Estimate with interval

metric

Pivot excessQuantity estimated.

subject

Bielik-11B-v3.0-InstructModel measured.

evaluation items

Polish STEM question setQuestion set.

items

766 questionsQuestions used.

value

0.6899 shareMean over the questions used.

interval low

0.6858 shareLower bound of the 95% bootstrap interval.

interval high

0.6938 shareUpper bound of the 95% bootstrap interval.

p value

0 p-valueTwo-sided bootstrap p-value.

setting

teacher-forced logit lens of answers with Polish chain-of-thought; two-sided stratified paired item bootstrap, 2000 resamplesMeasurement and test.

Method and evaluation protocol

Teacher-forced forward pass of each answer through the MLX bf16 model with a custom loop over the 50 blocks; after each block the residual stream at generated positions is decoded through the final RMSNorm and output head with an fp32 softmax; masses on English- and Polish-labelled pieces are pooled over reasoning positions whose next token is language-labelled. Answers below 0.95 teacher-forcing agreement, with a failed round-trip or without language positions are excluded. Per question, band English share minus the layer-50 share; mean over the questions usable in both models.

Dataset

Greedy answers of both models to 800 Polish STEM questions (250 GSM8K-PL, 300 LLMzSzŁ STEM, 250 PES) with Polish chain-of-thought, and their graded outcomes; questions usable in both models after the lens exclusions.Version: unspecified · Access: restricted

Reported results

Pivot excess 0.6899 [0.6858, 0.6938], bootstrap p-value 0 with 2,000 resamples; mean band English share 0.708, mean layer-50 English share 0.0181.

Uncertainty and replication

95% stratified paired item bootstrap interval (2000 resamples, strata benchmark, seed 20260704); not part of the Holm family. A p-value of 0 means that no bootstrap resample reached zero; with 2000 resamples the smallest non-zero value is 0.001.

Evidence references

analysis.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/a9246a60690bf1627e4cc56095f64e3307508bb2/experiments/E08-latent-pivot/results/analysis.jsonmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/a9246a60690bf1627e4cc56095f64e3307508bb2/experiments/E08-latent-pivot/results/metrics.json

Limitations

Logit lens, not a tuned lens: early-layer decodings are noisy. The vocabulary partitions label 16163 Polish and 1289 English pieces for APT4 and 1665 and 6116 for the Mistral-derived tokenizer, and were built from re-serialised MLX tokenizer files without an equivalence check. The design was written after the lens run started. The Polish-model lens file combines records from two invocations separated by an unrecorded rewrite of the lens script. The design registered A1 for the transplant only; this value is descriptive.