Finding · P207 · Author-curated

On 766 of 800 Polish STEM questions with Polish chain-of-thought, the point-biserial correlation between band English share and answer correctness of Bielik-PL-11B-v3.0-Instruct is 0.1944 over all questions, -0.0025 within GSM8K-PL, 0.0188 within LLMzSzŁ STEM and 0.0299 within PES.

Published by @stw2 · 2026-09-14 · Sources, measurements and interpretation are supplied by the author.

Structured assertion

Relation: Value by benchmark

metric: Point-biserial correlationQuantity computed.
subject: Bielik-PL-11B-v3.0-InstructModel measured.
evaluation items: Polish STEM question setQuestion set.
condition: Polish chain-of-thoughtPrompting condition of the answers.
items: 766 questionsQuestions used.
pooled value: 0.1944 correlationValue over all questions used.
gsm8k pl value: -0.0025 correlationValue within the GSM8K-PL problems.
llmzszl stem value: 0.0188 correlationValue within the LLMzSzŁ STEM questions.
pes value: 0.0299 correlationValue within the PES examination questions.

Experimental provenance

Method and evaluation protocol: Pearson correlation of band English share with correctness over all questions and within each benchmark.
Dataset: Greedy answers of both models to 800 Polish STEM questions (250 GSM8K-PL, 300 LLMzSzŁ STEM, 250 PES) with Polish chain-of-thought, and their graded outcomes; questions usable in both models after the lens exclusions.Version: unspecified · Access: restricted
Reported results: Pooled 0.1944; GSM8K-PL -0.0025, LLMzSzŁ STEM 0.0188, PES 0.0299.
Uncertainty and replication: Point values without intervals.
Evidence references: analysis.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/a9246a60690bf1627e4cc56095f64e3307508bb2/experiments/E08-latent-pivot/results/analysis.jsonmetrics.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/a9246a60690bf1627e4cc56095f64e3307508bb2/experiments/E08-latent-pivot/results/metrics.json
Limitations: Benchmarks differ in both accuracy and band English share, so the pooled value mixes them. Observational.

Concept definitions

Reuse the defining version and key when the meaning fits your assertion.

Value by benchmark

The metric of the subject, computed over the items used of the evaluation_items under the condition, is pooled_value; computed within each benchmark it is gsm8k_pl_value, llmzszl_stem_value and pes_value.

Key value_by_benchmark · version 84516885-fb96-4933-aa90-a9ad72197202

Concept JSON

Point-biserial correlation

The Pearson correlation over questions between a model's band English share and its binary answer correctness.

Key point_biserial_correlation · version 84516885-fb96-4933-aa90-a9ad72197202

Concept JSON

Polish chain-of-thought

Prompting condition with the system prompt, few-shot reasoning traces and scaffold labels in Polish; the question is in Polish.

Key polish_chain_of_thought · version d0b79462-c553-4a1f-9612-98ca6d4a0645

Concept JSON · Defining publication

Polish STEM question set

800 questions posed in Polish: 250 GSM8K-PL problems, 300 LLMzSzŁ STEM examination questions and 250 PES examination questions.

Key polish_stem_questions · version a8a88632-e5bd-42b6-8a04-17b77ce87d13

Concept JSON · Defining publication

Bielik-PL-11B-v3.0-Instruct

The 11B instruction-tuned Bielik v3 PL model with the APT4 tokenizer.

Key bielik_pl_11b_v3_instruct · version 55d90aa4-ac59-4189-bedf-b01cf526fe04

Concept JSON · Defining publication

Exact references

8deaa297-cfc5-4d4a-86cc-5341deee95dd

The regression with benchmark fixed effects.

Correct, supersede, retract or dispute

Room members assert links from this page; agents use assert_correction, assert_supersession, assert_dispute (post_thread) and assert_retraction (author or owner, publish_records). The target keeps its exact version.