Execution attempt

Sign in with GitHub
← Experiment E5 · How much of the change from an untouched model to the same model after an APT4 transplant and continued pretraining is attributable to the continued pretraining rather than to the tokenizer replacement, when the original-tokenizer model receives identical continued pretraining?

Execution attempt · A51 · planned

Writes flat headline values from the analysis, the scoring rows and the transplant statistics.

Succeeded · Registered by @stw2 via agent. Reported and received times are kept apart; no computation or result is verified.

Pinned source and configuration

https://github.com/stw2/tokenizer-science-tax @ 9a57aa40f4e774136f51b6730c33266d59094a45

Reference checked 2026-09-14 13:12 UTC. Later commits, branches or plan changes do not retarget this attempt.

Command
python scripts/10_metrics.py
Working directory
experiments/E05-control-arm
Configuration paths
None
Parameters
none
Environment
3.14.5; Python standard library; Apple silicon, 128 GB unified memory
Output directory
Not recorded

Inputs

Materials the registrant named when registering this attempt, by content identity; obtainability is derived from their location reports. Nothing is fetched or verified.

Latest reported metrics

Values the registrant’s tool attached to its latest progress or outcome event. Substrate stores them; it verifies no measurement.

mcq_n_pes
250
bpb_base_en
0.8294031390820282
bpb_base_pl
1.2042606606653605
gap_100m_en
0.2371281667636993
gap_100m_pl
0.09570088576958091
gap_200m_en
0.18493013298866012
gap_200m_pl
0.046511438107843306
gap_300m_en
0.1594311606941441
gap_300m_pl
0.03045944400101641
gap_400m_en
0.14416074412071267
gap_400m_pl
0.020108477186116303
gap_500m_en
0.132755540323775
gap_500m_pl
0.014747514693972286
bpb_fvt_t0_en
2.635149611696946
bpb_fvt_t0_pl
2.9274600687097636
fvt_pieces_fvt
31741
bpb_base_pl_pes
1.4070743632872356
digits_acc_base
0.5867
digits_n_pooled
900
fvt_pieces_byte
256
gap_500m_pl_pes
0.037111492420940584
fvt_pieces_empty
0
mcq_acc_pes_base
0.248
bpb_arm_a_100m_en
0.8438114905133021

and 162 more in the attempt JSON.

Delivered events

  1. Registered

    #1

    Writes flat headline values from the analysis, the scoring rows and the transplant statistics.

    reported · received · @stw2 via agent · posted to the Thread

  2. Started

    #2

    Started.

    reported · received · @stw2 via agent · attempt only

  3. Succeeded

    #3

    Completed with exit code 0.

    Exit code 0.

    mcq_n_pes=250 · bpb_base_en=0.8294031390820282 · bpb_base_pl=1.2042606606653605 · gap_100m_en=0.2371281667636993 · gap_100m_pl=0.09570088576958091 · gap_200m_en=0.18493013298866012 · gap_200m_pl=0.046511438107843306 · gap_300m_en=0.1594311606941441 · …

    • metrics.json · https://github.com/stw2/tokenizer-science-tax/blob/9a57aa40f4e774136f51b6730c33266d59094a45/experiments/E05-control-arm/results/metrics.json · public · sha256 7cf09ebd3eae… · values 13fb471e5c8f… · 8501 bytes · M188

    reported · received · @stw2 via agent · posted to the Thread

Report an event

The registrant’s capture tool normally delivers start and outcome events. Reporting here is the same author report with the browser as the reported time; it does not observe the process.

This attempt has a delivered outcome. A new execution is a new attempt.