Execution attempt

Sign in with GitHub
← Experiment E3 · Where do OpenAI's current frontier models stand on ZendoBench 1.0.0 dev when they play through the Codex CLI with every tool off, and how do they play: how much information do their experiments gain, and how many rule classes are still alive when they submit?

Execution attempt · A15 · planned

E3 arm gpt-5.6-terra-codex-high: all 460 dev games through the Codex CLI with no tools, run per the accepted plan; launched after the arm before it by scripts/01_all.sh

Succeeded · Registered by @stw2 via agent. Reported and received times are kept apart; no computation or result is verified.

Pinned source and configuration

https://github.com/stw2/zendo-lab @ 9939c141fc598a90e9d05b4735ce584dc8277bdd

Reference checked 2026-10-06 21:45 UTC. Later commits, branches or plan changes do not retarget this attempt.

Command
bash experiments/E03-codex-frontier/scripts/01_run.sh gpt-5.6-terra-codex-high
Working directory
.
Configuration paths
experiments/E03-codex-frontier/scripts/sandbox.sb experiments/E03-codex-frontier/scripts/codex-cli/package-lock.json
Parameters
model=gpt-5.6-terra; backend=Codex CLI 0.160.0 (codex exec, no tools), 12 calls in flight; hardware=provider; sampling=reasoning effort high, summary detailed (Codex defaults: temperature 1.0, top_p 0.98, verbosity low); output cap=none
Environment
macOS 26.5 on Apple M4 Max 128 GB (harness only); Python 3.13 via uv; ZendoBench 1.0.0; Codex CLI 0.160.0 (codex exec, no tools) under sandbox-exec, a new empty folder per call; ChatGPT sign-in in a Codex home of its own; the model served by OpenAI
Output directory
Not recorded

Inputs

Materials the registrant named when registering this attempt, by content identity; obtainability is derived from their location reports. Nothing is fetched or verified.

Latest reported metrics

Values the registrant’s tool attached to its latest progress or outcome event. Substrate stores them; it verifies no measurement.

T1.wins
29
T2.wins
51
T3.wins
112
T4.wins
2
T5.wins
21
T6.wins
12
T1.scored
30
T2.scored
60
T3.scored
150
T4.scored
60
T5.scored
100
T6.scored
60
T1.win_rate
0.9666666666666667
T2.win_rate
0.85
T3.win_rate
0.7466666666666667
T4.win_rate
0.03333333333333333
T5.win_rate
0.21000000000000002
T6.win_rate
0.19999999999999998
diagnose.eig
0.5178182409126847
diagnose.zero
0.36342229199372056
headline_items
430
headline_t2_t6
0.40800000000000003
headline_ci_low
0.36940016025271794
diagnose.p_first
0.45003741580031387

and 11 more in the attempt JSON.

Delivered events

  1. Registered

    #1

    E3 arm gpt-5.6-terra-codex-high: all 460 dev games through the Codex CLI with no tools, run per the accepted plan; launched after the arm before it by scripts/01_all.sh

    reported · received · @stw2 via agent · posted to the Thread

  2. Started

    #2

    Launched 2026-10-07T04:26:30Z at 664e563 by scripts/01_all.sh, after the arm before it and this arm's connection check (exit 0): bash experiments/E03-codex-frontier/scripts/01_run.sh gpt-5.6-terra-codex-high.

    reported · received · @stw2 via agent · attempt only

  3. Succeeded

    #3

    Exit 0; 430/430 headline games finished, score --verify verified every game. T2-T6 headline 40.8 [36.9, 45.1]; seed-only 4.7; 11.1 experiments per game at 0.52 bits of expected information; median 3.5 classes alive at the first submission. Isolation flags: none. Run files and call traces restricted (sha256 given); summaries public at c8ecc7f. Exit codes by part: [0].

    Exit code 0. Duration 13044 s.

    T1.wins=29 · T2.wins=51 · T3.wins=112 · T4.wins=2 · T5.wins=21 · T6.wins=12 · T1.scored=30 · T2.scored=60 · …

    • gpt-5.6-terra-codex-high.jsonl · experiments/E03-codex-frontier/results/runs/gpt-5.6-terra-codex-high.jsonl (run file; held by the Room owner; not public; ask the reporter) · restricted · sha256 e693314e990a… · 14122119 bytes · M60
    • gpt-5.6-terra-codex-high.traces.jsonl.gz · experiments/E03-codex-frontier/results/runs/traces/gpt-5.6-terra-codex-high.traces.jsonl.gz (call traces; held by the Room owner; not public; ask the reporter) · restricted · sha256 8cb5c29ae306… · 52411244 bytes · M61
    • gpt-5.6-terra-codex-high.score.json · https://github.com/stw2/zendo-lab/blob/c8ecc7f7605238c72c5836bd4730814377cf8906/experiments/E03-codex-frontier/results/scores/gpt-5.6-terra-codex-high.score.json · public · sha256 5912acf46815… · 28195 bytes · M62
    • gpt-5.6-terra-codex-high.diagnose.md · https://github.com/stw2/zendo-lab/blob/c8ecc7f7605238c72c5836bd4730814377cf8906/experiments/E03-codex-frontier/results/scores/gpt-5.6-terra-codex-high.diagnose.md · public · sha256 556e008ef2a5… · 1643 bytes · M63
    • gpt-5.6-terra-codex-high.files.json · https://github.com/stw2/zendo-lab/blob/c8ecc7f7605238c72c5836bd4730814377cf8906/experiments/E03-codex-frontier/results/scores/gpt-5.6-terra-codex-high.files.json · public · sha256 6f61ef39033d… · 154 bytes · M64

    reported · received · @stw2 via agent · posted to the Thread

Report an event

The registrant’s capture tool normally delivers start and outcome events. Reporting here is the same author report with the browser as the reported time; it does not observe the process.

This attempt has a delivered outcome. A new execution is a new attempt.