Execution attempt

Sign in with GitHub
← Experiment E3 · Where do OpenAI's current frontier models stand on ZendoBench 1.0.0 dev when they play through the Codex CLI with every tool off, and how do they play: how much information do their experiments gain, and how many rule classes are still alive when they submit?

Execution attempt · A16 · planned

E3 arm gpt-6-astra-codex-high: all 460 dev games through the Codex CLI with no tools, run per the accepted plan; launched after the arm before it by scripts/01_all.sh

Succeeded · Registered by @stw2 via agent. Reported and received times are kept apart; no computation or result is verified.

Pinned source and configuration

https://github.com/stw2/zendo-lab @ 9939c141fc598a90e9d05b4735ce584dc8277bdd

Reference checked 2026-10-06 21:45 UTC. Later commits, branches or plan changes do not retarget this attempt.

Command
bash experiments/E03-codex-frontier/scripts/01_run.sh gpt-6-astra-codex-high
Working directory
.
Configuration paths
experiments/E03-codex-frontier/scripts/sandbox.sb experiments/E03-codex-frontier/scripts/codex-cli/package-lock.json
Parameters
model=gpt-6-astra; backend=Codex CLI 0.160.0 (codex exec, no tools), 12 calls in flight; hardware=provider; sampling=reasoning effort high, summary detailed (Codex defaults: temperature 1.0, top_p 0.98, verbosity low); output cap=none
Environment
macOS 26.5 on Apple M4 Max 128 GB (harness only); Python 3.13 via uv; ZendoBench 1.0.0; Codex CLI 0.160.0 (codex exec, no tools) under sandbox-exec, a new empty folder per call; ChatGPT sign-in in a Codex home of its own; the model served by OpenAI
Output directory
Not recorded

Inputs

Materials the registrant named when registering this attempt, by content identity; obtainability is derived from their location reports. Nothing is fetched or verified.

Latest reported metrics

Values the registrant’s tool attached to its latest progress or outcome event. Substrate stores them; it verifies no measurement.

T1.wins
30
T2.wins
59
T3.wins
150
T4.wins
49
T5.wins
93
T6.wins
59
T1.scored
30
T2.scored
60
T3.scored
150
T4.scored
60
T5.scored
100
T6.scored
60
T1.win_rate
1
T2.win_rate
0.9833333333333333
T3.win_rate
1
T4.win_rate
0.8166666666666667
T5.win_rate
0.93
T6.win_rate
0.9833333333333333
diagnose.eig
0.3910052758057136
diagnose.zero
0.5443359127235824
headline_items
430
headline_t2_t6
0.9426666666666668
headline_ci_low
0.902650253076997
diagnose.p_first
0.7005303313834651

and 11 more in the attempt JSON.

Delivered events

  1. Registered

    #1

    E3 arm gpt-6-astra-codex-high: all 460 dev games through the Codex CLI with no tools, run per the accepted plan; launched after the arm before it by scripts/01_all.sh

    reported · received · @stw2 via agent · posted to the Thread

  2. Started

    #2

    Launched 2026-10-07T08:09:44Z at 408e426 by scripts/01_all.sh, after the arm before it and this arm's connection check (exit 0): bash experiments/E03-codex-frontier/scripts/01_run.sh gpt-6-astra-codex-high.

    reported · received · @stw2 via agent · attempt only

  3. Succeeded

    #3

    Exit 0; 430/430 headline games finished, score --verify verified every game. T2-T6 headline 94.3 [90.3, 96.3]; seed-only 4.7; 17.1 experiments per game at 0.39 bits of expected information; median 1.0 classes alive at the first submission. Isolation flags: none. Run files and call traces restricted (sha256 given); summaries public at fb00498. Exit codes by part: [0].

    Exit code 0. Duration 9512 s.

    T1.wins=30 · T2.wins=59 · T3.wins=150 · T4.wins=49 · T5.wins=93 · T6.wins=59 · T1.scored=30 · T2.scored=60 · …

    • gpt-6-astra-codex-high.jsonl · experiments/E03-codex-frontier/results/runs/gpt-6-astra-codex-high.jsonl (run file; held by the Room owner; not public; ask the reporter) · restricted · sha256 0b199c688b79… · 19006933 bytes · M65
    • gpt-6-astra-codex-high.traces.jsonl.gz · experiments/E03-codex-frontier/results/runs/traces/gpt-6-astra-codex-high.traces.jsonl.gz (call traces; held by the Room owner; not public; ask the reporter) · restricted · sha256 1a9ec066f338… · 41669765 bytes · M66
    • gpt-6-astra-codex-high.score.json · https://github.com/stw2/zendo-lab/blob/fb00498ec60bac1ee5f351f7a3bef01aae224ab4/experiments/E03-codex-frontier/results/scores/gpt-6-astra-codex-high.score.json · public · sha256 9b923a744359… · 26836 bytes · M67
    • gpt-6-astra-codex-high.diagnose.md · https://github.com/stw2/zendo-lab/blob/fb00498ec60bac1ee5f351f7a3bef01aae224ab4/experiments/E03-codex-frontier/results/scores/gpt-6-astra-codex-high.diagnose.md · public · sha256 1811997a5840… · 1610 bytes · M68
    • gpt-6-astra-codex-high.files.json · https://github.com/stw2/zendo-lab/blob/fb00498ec60bac1ee5f351f7a3bef01aae224ab4/experiments/E03-codex-frontier/results/scores/gpt-6-astra-codex-high.files.json · public · sha256 5b29f3d82c2f… · 152 bytes · M69

    reported · received · @stw2 via agent · posted to the Thread

Report an event

The registrant’s capture tool normally delivers start and outcome events. Reporting here is the same author report with the browser as the reported time; it does not observe the process.

This attempt has a delivered outcome. A new execution is a new attempt.