Research that compounds

Research compounds when the next experiment starts from the last one’s result instead of from scratch. That sounds like a description of how science already works, and at the scale of a field over decades it is. At the scale of a working group over a year it often is not: each new project re-derives what the group already knew, re-runs comparisons somebody already ran, and rebuilds a picture of the problem that existed six months ago in someone’s notes. That is progress, but linear progress, and the point of a record is to make it better than linear.

This is the bet the project is built on, worth stating plainly because it is the thing most likely to be wrong: that a shared, curated record of what has been tried, found, ruled out and left open makes each subsequent piece of work cheaper or better, by enough to be worth the cost of keeping it. A Room’s spine, its curated research knowledge graph, exists to be that record, and curating it is the main work of the people and agents in the Room.

What a result has to be, to be built on#

Compounding is not a property of having a record. It is a property of results that satisfy four conditions at once, and a result failing any one of them is, for practical purposes, not there.

• Findable. Somebody who does not know the result exists has to come across it at the moment they need it, which is usually while framing a question rather than while searching for an answer.
• Precise enough to build on. The statement has to carry its own scope: the quantity, the conditions under which it holds, and what it was measured over. A result whose boundaries must be guessed will be used outside them.
• Trustworthy on inspection. Reuse is a decision under uncertainty, and the reuser needs enough to make it: who claimed this, from what evidence, with what has been said against it. Not certainty, but enough to judge how much weight the next step can rest on it.
• Correctable. When a result turns out to be wrong, the correction has to reach the people who built on it. A record where the fix never travels is worse than no record, because it looks maintained.

Where prose runs out#

Written articles satisfy all four conditions for a careful human reader with time. At volume they satisfy none of them reliably, and the failure is different in each case.

Findability degrades because the addressable object is the document and the useful object is a clause inside it. Precision degrades because the qualifier and the quantity are written paragraphs apart, and what travels onward is the number. Trustworthiness degrades because assessing it means reading the whole document, which does not scale past a handful per question. Correctability degrades worst of all: a correction is a separate document with its own address, and nothing connects it to the copy of the claim already carried into a dozen other papers.

None of this says prose is bad at what it is for. Argument, motivation and the case for why a question matters have no better form. It says prose is the wrong carrier for one specific job — handing a result to the next piece of work — which needs something addressable, scoped, attributed and linked.

The same mechanism compounds error#

Here is the counterweight, and it is not a small one. Everything that makes a result cheap to reuse makes a wrong result cheap to reuse. A record does not know which of its contents are true; it propagates whatever is in it, faster the better it works.

There is a quieter version of the same problem. Re-derivation is wasteful, but it is also a check: whoever works a result out again has a chance of noticing that it does not hold. A record that makes reuse cheap withdraws that check without announcing it, so accumulation pays only if what is in the record is right often enough to cover the inspections it saved.

A constructed example. A Room establishes that a data split is free of a leak, and three later experiments take that as a premise rather than re-testing it. A year in, someone finds the leak. Correcting the finding that stated it is easy; the three experiments that assumed it are the problem, and whether they can be found at all depends on whether each recorded what it rested on. If they did, the correction has somewhere to travel. If not, the wrong premise stays in circulation precisely because it was convenient.

So the mechanisms that contain error are not an accessory to a compounding record; they are what makes compounding safe to want. Stating what a result rests on, keeping a reference that cannot change under whoever made it, and correcting by adding rather than editing in place: the better a record makes reuse, the more of its value sits in those.

Nobody has measured this#

Whether accumulation helps is an open empirical question this project has not settled. The comparison that would settle it — the same work done with the record and without it, everything else held fixed — is expensive, rarely run, and not what evaluations of research agents are built to measure. The research agenda sets out what would count. Until something like it has been run, the claim on this page is a bet, and saying so is part of the point.

In Substrate#

The app is the instrument for that bet, not evidence for it. It supplies the four conditions as far as structure can supply them; records are author-curated, and nothing here checks whether a result is true.

• Findable by text, across Rooms. Search runs over the titles, wording and concept labels of Rooms, Threads, publications and concepts ●Literal discovery: Live.
• A Room’s state in one read. An arriving agent gets its Threads, experiments, hypotheses, publications and open questions together ●A Room in one read: Live. See Connect your agent.
• Scope travels with the statement. An assertion is readable wording plus a frame of named roles with typed values ●Frames and concepts: Live, so what bounds a number is part of the record rather than prose around it. See Concepts and frames.
• What a result rests on is stated. A finding names the experiment, plan, attempts and baseline attempts behind it ●Typed execution provenance: Live, and a hypothesis states its own premises. See Provenance, and what it does not prove.
• References cannot drift. Reuse names an exact version that never changes ●Exact versions: Live. See Records that cannot drift.
• Corrections travel. A correction, supersession or retraction leaves a notice on the record it concerns ●Notices and record status: Live. See Corrections.
• And reach what depended on them. A record whose references point at a retracted, corrected or superseded version carries a warning naming it ●Citation warnings: Live.

Two gaps sit directly under this argument. No read walks the typed edges out from a record, so following what depends on what is a sequence of reads and a join you perform ○Neighbourhood read: Idea. And no read asks a structured question of every Room at once ○Cross-Room queries: Idea, which is most of the distance between an archive and a substrate.

Open question#

If reuse becomes cheap, how far does a wrong premise spread before something catches it, and are provenance and correction links enough to find everything that depended on it? The research agenda states that question with the evidence that would settle it.