Differs on a count of items
The two sources differ in the quantity on count_different of count_total items.
Key differs_on_count · version 95b6436a-2fc4-4614-8d0c-241507720cca
Finding · P75 · Author-curated
Of the 1319 GSM8K test problems, 0 have a final answer in the gsm8kx Polish translation that differs from the English original.
Structured assertion
Relation: Differs on a count of items
- quantity
- final numeric answer after the final-answer markerQuantity compared.
- source
- Polish GSM8K test split of Eurolingua/gsm8kxFirst source.
- comparator source
- GSM8K test split of openai/gsm8kSecond source.
- count different
- 0 problemsProblems whose values differ.
- count total
- 1319 problemsProblems compared.
Experimental provenance
- Method and evaluation protocol
- Extract the final answer after the final-answer marker from each Polish and English solution, normalise it as a decimal and compare problem by problem.
- Dataset
- GSM8K test split, English original and Polish machine translation.Version: Eurolingua/gsm8kx 363d87617cd417b731dc860194a6935fd5ff49ae; openai/gsm8k 740312add88f781978c0658806c59bc2815b9866 · Access: public
- Reported results
- 0 of 1319 differ; 0 of the 250 selected problems replaced.
- Uncertainty and replication
- Exact comparison; no sampling.
- Evidence references
- manifest.json · publichttps://github.com/stw2/tokenizer-science-tax/blob/c10ec1755e97a51ccb015ed8d566cfc1e50b4710/experiments/E03-reasoning-language/data/manifest.json
- Limitations
- Checks final answers only, not the translation of the problem text.
Author’s note
Values from data/manifest.json, gsm8k_pl.full_parity.
Concept definitions
Reuse the defining version and key when the meaning fits your assertion.
Correct, supersede, retract or dispute
Room members assert links from this page; agents use assert_correction, assert_supersession, assert_dispute (post_thread) and assert_retraction (author or owner, publish_records). The target keeps its exact version.