Execution attempt

Sign in with GitHub
← Experiment E4 · Where do Claude Haiku, Sonnet, Opus and Fable stand on ZendoBench 1.0.0 dev at high reasoning effort through a tool-free Claude Code harness matched as closely as possible to E3, and how do they experiment and decide to submit?

Execution attempt · A21 · rerun

Haiku high-effort baseline after A17 temporary-rate-limit preflight failure. No prior model/game outcome. Connection check, six-game smoke, then 460 dev games; unchanged settings.

Preflight failed · Registered by @stw2 via agent. Reported and received times are kept apart; no computation or result is verified.

Pinned source and configuration

https://github.com/stw2/zendo-lab @ c60e22fcc4dbda7a32d7d7b3ef1f2cfc079619d9

Reference checked 2026-10-08 09:45 UTC. Later commits, branches or plan changes do not retarget this attempt.

Command
bash experiments/E04-claude-baselines/scripts/01_run.sh claude-haiku-5-5-claude-high
Working directory
.
Configuration paths
experiments/E04-claude-baselines/DESIGN.md
Parameters
{"reasoning_effort": "high", "thinking": "adaptive", "max_tokens": 128000, "sampling": "provider defaults", "manifest": "dev", "games": 460, "batch": 12, "player": "model", "backend": "function", "seed": "manifest episode seeds; model unseeded", "model": "claude-haiku-5-5"}
Environment
Claude Code 2.1.293; macOS; Apple M4 Max 128 GB local orchestrator; hosted Anthropic inference, provider hardware unknown; isolated Claude subscription login.
Output directory
Not recorded

Inputs

Materials the registrant named when registering this attempt, by content identity; obtainability is derived from their location reports. Nothing is fetched or verified.

Delivered events

  1. Registered

    #1

    Haiku high-effort baseline after A17 temporary-rate-limit preflight failure. No prior model/game outcome. Connection check, six-game smoke, then 460 dev games; unchanged settings.

    reported · received · @stw2 via agent · posted to the Thread

  2. Preflight failed before launch

    #2

    A21 exact-prompt preflight did not pass: five guarded tries and direct exact-prompt control returned HTTP 429. Standard CLI wrapper and minimal SDK identity controls answered OK. Thus ordinary Claude access works; the observed rejection depends on request configuration. Final identical retry stopped. No benchmark games. Amend protocol to retain the fixed SDK identity sentence and register new source before games.

    Exit code 130. Signal SIGINT. Error: exact_prompt_configuration_rejected; minimal SDK identity succeeds

    • a21-haiku-connection.jsonl.gz · experiments/E04-claude-baselines/results/runs/preflight/a21-haiku-connection.jsonl.gz (private diagnostic transcript held by owner; not public; ask reporter) · restricted · sha256 d051cf402bd8… · 6754 bytes · M75
    • a21-haiku-direct-control.jsonl.gz · experiments/E04-claude-baselines/results/runs/preflight/a21-haiku-direct-control.jsonl.gz (private diagnostic transcript held by owner; not public; ask reporter) · restricted · sha256 cc0f73f2f43a… · 1873 bytes · M76
    • a21-haiku-identity-control.jsonl.gz · experiments/E04-claude-baselines/results/runs/preflight/a21-haiku-identity-control.jsonl.gz (private diagnostic transcript held by owner; not public; ask reporter) · restricted · sha256 2fc365e95aa7… · 2558 bytes · M77
    • a21-haiku-standard-prompt-control.jsonl.gz · experiments/E04-claude-baselines/results/runs/preflight/a21-haiku-standard-prompt-control.jsonl.gz (private diagnostic transcript held by owner; not public; ask reporter) · restricted · sha256 6fb9a8af9cd1… · 2493 bytes · M78

    reported · received · @stw2 via agent · posted to the Thread

Report an event

The registrant’s capture tool normally delivers start and outcome events. Reporting here is the same author report with the browser as the reported time; it does not observe the process.

This attempt has a delivered outcome. A new execution is a new attempt.