Execution attempt

Sign in with GitHub
← Experiment E11 · Does Qwen2.5-1.5B score the same multiple-choice content lower in Polish than in English, and do Polish-heavy continued pretraining and the APT4 transplant change that gap?

Execution attempt · A115 · planned

Scores all 2,400 English and Polish items with the continued-pretraining checkpoint that has APT4 by FVT.

Succeeded · Registered by @stw2 via agent. Reported and received times are kept apart; no computation or result is verified.

Pinned source and configuration

https://github.com/stw2/tokenizer-science-tax @ 3c128813c117fd41b46d3a9949c2c8751134df27

Reference checked 2026-09-14 13:18 UTC. Later commits, branches or plan changes do not retarget this attempt.

Command
python scripts/02_probe_paired_mcq.py --model ../E05-control-arm/ckpts/armB/ckpt_00500M --tag armB_500M --out results/scores_armB_500M.jsonl --dump-prompts
Working directory
experiments/E11-cross-language-access
Configuration paths
None
Parameters
all items; tag armB_500M
Environment
3.12.13; PyTorch, MPS, bf16; Apple silicon, 128 GB unified memory
Output directory
Not recorded

Inputs

Materials the registrant named when registering this attempt, by content identity; obtainability is derived from their location reports. Nothing is fetched or verified.

Delivered events

  1. Registered

    #1

    Scores all 2,400 English and Polish items with the continued-pretraining checkpoint that has APT4 by FVT.

    reported · received · @stw2 via agent · posted to the Thread

  2. Started

    #2

    Started.

    reported · received · @stw2 via agent · attempt only

  3. Succeeded

    #3

    Completed with exit code 0.

    Exit code 0.

    • scores_armB_500M.jsonl · https://github.com/stw2/tokenizer-science-tax/blob/5e8117ffb3f168331ee63f1032cd40a62c6296bd/experiments/E11-cross-language-access/results/scores_armB_500M.jsonl · public · sha256 b8452c8be5e5… · 782025 bytes · M507
    • scores_armB_500M.jsonl as run · Archived private repository, experiments/E11/results/scores_armB_500M.jsonl at commit bee818e; the public copy differs only in the relative checkpoint path in the model field · unavailable · sha256 f652446c637a… · 861225 bytes · M508
    • prompt_dump_armB_500M_belebele_en.json · https://github.com/stw2/tokenizer-science-tax/blob/5e8117ffb3f168331ee63f1032cd40a62c6296bd/experiments/E11-cross-language-access/results/prompt_dumps/prompt_dump_armB_500M_belebele_en.json · public · sha256 1139fe67bc03… · 1045 bytes · M509
    • prompt_dump_armB_500M_belebele_pl.json · https://github.com/stw2/tokenizer-science-tax/blob/5e8117ffb3f168331ee63f1032cd40a62c6296bd/experiments/E11-cross-language-access/results/prompt_dumps/prompt_dump_armB_500M_belebele_pl.json · public · sha256 25fa59d3a33a… · 1145 bytes · M510
    • prompt_dump_armB_500M_mmlu_en.json · https://github.com/stw2/tokenizer-science-tax/blob/5e8117ffb3f168331ee63f1032cd40a62c6296bd/experiments/E11-cross-language-access/results/prompt_dumps/prompt_dump_armB_500M_mmlu_en.json · public · sha256 91bab88b57f5… · 1141 bytes · M511
    • prompt_dump_armB_500M_mmlu_pl.json · https://github.com/stw2/tokenizer-science-tax/blob/5e8117ffb3f168331ee63f1032cd40a62c6296bd/experiments/E11-cross-language-access/results/prompt_dumps/prompt_dump_armB_500M_mmlu_pl.json · public · sha256 af5417af176f… · 1231 bytes · M512
    • full_run.log · https://github.com/stw2/tokenizer-science-tax/blob/5e8117ffb3f168331ee63f1032cd40a62c6296bd/experiments/E11-cross-language-access/results/full_run.log · public · sha256 ca4390954444… · 966 bytes · M513
    • full_run.log as run · Archived private repository, experiments/E11/results/full_run.log at commit bee818e; console output of the three scoring runs; the public copy differs only in the relative checkpoint path of the loader line · unavailable · sha256 490ed1c5d34f… · 999 bytes · M514

    reported · received · @stw2 via agent · posted to the Thread

Report an event

The registrant’s capture tool normally delivers start and outcome events. Reporting here is the same author report with the browser as the reported time; it does not observe the process.

This attempt has a delivered outcome. A new execution is a new attempt.