Attributed research checkpoint
E3: the English chain-of-thought gain of Bielik-PL-11B-v3.0-Instruct minus that of Bielik-11B-v3.0-Instruct on 800 Polish STEM questions is -0.0225 (95% interval -0.055 to 0.0112), inconclusive under the pre-registered rule.
Assignment, status and accepted plans remain authoritative on each experiment.
Reported state
- Reported progress
- E3 completed with three retrospective plan versions: the design, the GSM8K-PL token cap raised to 1280 before generation, and regeneration of capped answers at 1280 tokens after an interim grading, both amendments reconstructed. 3,200 greedy answers graded and analysed; the per-benchmark and sensitivity differences are inconclusive as well.
- Open issues
- The pre-registered H2 analyses (gain against the Polish token surplus over benchmark and model combinations, item-level association) and the comparison with E2's GSM8K bridge were not run. The smoke gate's outputs were not kept. The interim grading and the answers before regeneration were overwritten. The grading and analysis scripts were last edited after generation began.
- Suggested next action
- E4: effective context for scientific text on the same model pair.
- Access needs
- Both models are gated on Hugging Face; the LLMzSzŁ and PES item sets, the exemplars and the complete generations are restricted materials, rebuilt from pinned dataset revisions; generation needs Apple silicon as written.
Exact references
Exact record · P76
a8a88632-e5bd-42b6-8a04-17b77ce87d13Primary test: the 95% interval includes zero, which the pre-registered rule reads as inconclusive.
Exact record · P77
080dd7b3-7a21-419e-bf4e-c3770f5cdde6Secondary test on one benchmark: the 95% interval includes zero.
Exact record · P78
4c39157e-9b47-4096-b640-a79bf90a103dSecondary test on one benchmark: the 95% interval includes zero.
Exact record · P79
a164a223-a32f-4fdf-a694-c1d5cda9b77dSecondary test on one benchmark: the 95% interval includes zero.
Exact record · P80
d0d57a25-5d4f-467b-8753-4fbff8780904Sensitivity rerun of the primary test: the 95% interval includes zero.
Exact record · P81
ff9de3a3-46c9-4ec8-8367-70f0f2d374eeSensitivity rerun of the primary test: the 95% interval includes zero.
Exact record · P82
07d4e6b0-719c-43ec-b75d-ff7bac9cd488One of the two gains whose difference the primary test estimates.
Exact record · P83
5696028e-6f29-44d8-a073-8dcd5d4eba07One of the two gains whose difference the primary test estimates.
Exact record · P84
d0b79462-c553-4a1f-9612-98ca6d4a0645Accuracy of the pair on translated GSM8K with Polish reasoning.
Exact record · P85
5907d582-517e-4a59-bdc6-c58cf7a7fe7dInput to H2, whose pre-registered test was not run.
Exact record · P86
f0577f2f-d667-4197-a5f1-ea423c4cb866Input to H2, whose pre-registered test was not run.
Exact record · P89
95f8ccf1-de11-482c-99fa-d5240c8f6eddInstrument check: the truncation asymmetry that prompted the regeneration.
Experiment · E3
Open experiment →Accepted plan
Exact plan →Accepted plan
Exact plan →Accepted plan
Exact plan →