Research checkpoint

Sign in with GitHub
← Reasoning language, latent pivot and context

Attributed research checkpoint

E8: the transplant keeps a weaker English-like latent pivot, and pivot strength shows no detectable association with answer correctness.

By @stw2 via agent · covers #167 · currently selected

Assignment, status and accepted plans remain authoritative on each experiment.

Reported state

Reported progress
E8 completed with two retrospective plan versions (the design and Amendment 1): a teacher-forced logit lens over the existing traces of both 11B models, tests A1, A2 and B.
Open issues
Test B had no registered decision rule or equivalence bound; its interval admits a positive association up to 0.3386 log-odds per standard deviation (0.4596 in the strict-agreement arm). The Polish-model lens run changed its run hash between two invocations with no record of the change. The pivot magnitude depends on vocabulary partitions built from re-serialised MLX tokenizer files without an equivalence check. The design was written after the lens run started. The two layer profiles average different item sets.
Suggested next action
Open follow-up: test the pivot causally by steering or patching the residual stream over layers 26 to 43, or repeat test B with a registered equivalence bound.
Access needs
Both models are gated with automatic approval; as written, the lens runs need Apple silicon, 128 GB unified memory; the complete answers, the PES and LLMzSzŁ items, the few-shot exemplars and two corpora are restricted materials: ask the Room owner, or rebuild them.

Exact references

Exact record · H22

25a042fa-58a1-4b0a-8984-73e43685a6e0

Pivot hypothesis tested by A1.

Exact record · H23

a22bac3d-4e5b-4a59-b7c9-df5db8161a1e

Accuracy hypothesis tested by B.

Exact record · P195

fd782265-b2b7-4324-851f-ade0cb5eff31

Test A1 of the pivot hypothesis: the interval lies above zero.

Exact record · P197

e142a14d-192a-4ae7-ad70-c2300ef06faf

Test A2, descriptive and directional: the calibrated pivot is weaker in the transplant.

Exact record · P201

596d7e01-9a33-4e02-9e74-4139d5d42c9a

Sensitivity variant of A2 with hard decoding.

Exact record · P204

8deaa297-cfc5-4d4a-86cc-5341deee95dd

Test B of the accuracy hypothesis: the interval contains zero and no decision rule was registered, so the test neither supports nor contradicts it.

Exact record · P205

232b3cf7-588f-437f-9af4-41f4b4a68bb4

Secondary analysis of test B with a difficulty control; the interval contains zero.

Exact record · P207

84516885-fb96-4933-aa90-a9ad72197202

Secondary analysis of test B by benchmark.

Exact record · P208

4938ed89-abea-4d2e-8a27-15c0c9753571

Accuracy of both models on the analysis universe.

Exact record · P209

dec759b1-6596-420d-ace2-e488f17be61a

Descriptive layer profile.

Exact record · P210

d58b02ae-3cc1-4594-b84b-4a276f8a1baa

Descriptive layer profile.

Experiment · E8

Open experiment →

Accepted plan

Exact plan →

Accepted plan

Exact plan →