On Polish STEM questions, the mean band English share of Bielik-PL-11B-v3.0-Instruct is 0.258 for its answers with Polish chain-of-thought and 0.3671 for its answers with English chain-of-thought.

Relation: Metric comparison

metric

Band English shareQuantity compared.

subject

Polish chain-of-thoughtPrompting condition of the first value.

subject value

0.258 shareMean band English share under the subject condition.

comparator

English chain-of-thoughtPrompting condition of the second value.

comparator value

0.3671 shareMean band English share under the comparator condition.

setting

Bielik-PL-11B-v3.0-InstructModel both values are measured on.

scope

Polish chain-of-thought: 766 of 800 Polish STEM questions usable in both models; English chain-of-thought: 767 usable answers of this modelAnswers averaged.

Method and evaluation protocol

Teacher-forced forward pass of each answer through the MLX bf16 model with a custom loop over the 50 blocks; after each block the residual stream at generated positions is decoded through the final RMSNorm and output head with an fp32 softmax; masses on English- and Polish-labelled pieces are pooled over reasoning positions whose next token is language-labelled. Answers below 0.95 teacher-forcing agreement, with a failed round-trip or without language positions are excluded. Band English share per answer, averaged per condition.

Dataset

Greedy answers of both models to 800 Polish STEM questions (250 GSM8K-PL, 300 LLMzSzŁ STEM, 250 PES) with Polish chain-of-thought, and their graded outcomes; questions usable in both models after the lens exclusions.Version: unspecified · Access: restricted

Reported results

0.258 against 0.3671; the report reads the ratio as 70%.

Uncertainty and replication

Point values without intervals.

Evidence references

analysis.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/a9246a60690bf1627e4cc56095f64e3307508bb2/experiments/E08-latent-pivot/results/analysis.jsonmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/a9246a60690bf1627e4cc56095f64e3307508bb2/experiments/E08-latent-pivot/results/metrics.json

Limitations

Logit lens, not a tuned lens: early-layer decodings are noisy. The vocabulary partitions label 16163 Polish and 1289 English pieces for APT4 and 1665 and 6116 for the Mistral-derived tokenizer, and were built from re-serialised MLX tokenizer files without an equivalence check. The design was written after the lens run started. The Polish-model lens file combines records from two invocations separated by an unrecorded rewrite of the lens script. The two means average different sets of answers.