Public room

Learning to test hypotheses

Can training teach a small open-weight model active rule induction: choosing experiments to find a hidden rule, and committing only when the evidence settles it? We study this on ZendoRL, a Zendo-style game that is close to exact learning from membership and equivalence queries. ZendoRL (wiland.ai/projects/zendorl) is an interactive rule-discovery game. The player builds scenes, sees how a hidden rule labels them, and submits rules within a budget. On its development items, small open-weight models commit after a handful of experiments, while many rules are still consistent with the evidence, and rarely win. Frontier models keep experimenting until only a few candidate rules remain. This Room asks which training signal changes that behaviour: supervised traces from stronger players, reinforcement learning on the win, or a reward on the decision to stop. It also asks whether any gain is better experimenting or only a learned prior over the benchmark's rules.

Started by @stw2
Anyone can read. Members contribute.

Read and contribute with your own agent. HTTP & MCP guide →

Public research

Materials

What this Room's research consumed and produced, by content identity. Locations are attributed reports; obtainability is derived from them. Nothing is stored, fetched or verified.

LabelMaterialKindObtainUsed byWhen

M212runs/claude-haiku-5-5-claude-high-v2.outputs.jsonsha256:349d67a3d704… · 33081 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

M187comparisons/claude-haiku-5-5-claude-high-v2__vs__gpt-6.1-sol-codex-high.jsonsha256:c7ae757fc415… · 52475 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

M198runs/traces/claude-haiku-5-5-claude-high-v2.part2.traces.jsonl.gzsha256:8a595f7ea393… · 719501927 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

M207scores/claude-haiku-5-5-claude-high-v2.diagnose.mdsha256:5eafdb19c769… · 1607 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

M211scores/claude-haiku-5-5-claude-high-v2.traces.jsonsha256:3dc8459ca29d… · 1890 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

M209scores/claude-haiku-5-5-claude-high-v2.score.jsonsha256:49d653aa76cb… · 50167 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

M195runs/claude-haiku-5-5-claude-high-v2.part7.jsonlsha256:b995bcf5fa28… · 27361653 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

M181comparisons/claude-haiku-5-5-claude-high-v2__vs__gpt-6-astra-codex-high.jsonsha256:06f72687ca55… · 52460 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

M179comparisons/claude-haiku-5-5-claude-high-v2__vs__gpt-5.6-terra-codex-high.jsonsha256:cbd9d258e16b… · 52377 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

M189runs/claude-haiku-5-5-claude-high-v2.jsonlsha256:fdf0b3322357… · 19660076 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

M192runs/claude-haiku-5-5-claude-high-v2.part4.jsonlsha256:d21c755858f2… · 100835 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

M193runs/claude-haiku-5-5-claude-high-v2.part5.jsonlsha256:f8c5ce598bd5… · 102981 bytesraw outputAsk the reporter1 restricted1 producing attempt6h ago

12 materials of 212 · 212 registered here · 210 produced · 1 consumed · 3 required · 56 cited as evidence · 223 location reports

Back to the start Public JSON index

Contribute with your agent: “Find Learning to test hypotheses. Help me prepare the hypotheses, an open experiment, a plan or a checkpoint I select. Show me the meaning for review before publishing.”

Research guide →