How to read this site
This site publishes the results of one research project, an experiment at a time. Its pages mix prose that a person wrote with numbers that a program generated, and they mix the author's own sentences with sentences a language model drafted. This page says which is which, in what order an experiment page presents them, and what a reader who wants to check the work should look at first.
It is about the site rather than the research. What the settings behind a run actually do is the capabilities page, and the terms used throughout are defined there.
Who wrote what
Prose on this site sets in one of two faces, and the face is a statement about who put the words in that order. The sans-serif face is the author writing in his own words. The serif face, with a dotted bar down the left margin where the passage runs a paragraph or more, was drafted by a language model. Some of it the author has read closely and some of it he has not, and the mark does not distinguish the two, so a marked passage is prose that no one has necessarily checked. Any page carrying one opens with a line that says so and names the models that did the drafting.
The dots are what to look for, because they are not the only bar in the margin. An experiment page draws a solid colored bar down the side of each of its labeled blocks, one color per kind, and that bar is structure rather than authorship: it says a stated intent or a measured result starts here, and says nothing about who wrote it. Where a drafted passage fills such a block, which is the usual case, the two bars run side by side, the solid one outside and the dotted one in.
The mark applies to the passage, not to the page, so a section can hold both faces and often does. A model-drafted paragraph followed by a sentence the author added himself shows the change of hands at the sentence.
What the mark records is authorship of the sentence, and it is deliberately not a claim about quality. The author decides what to run, what to publish and what each experiment means, and he is answerable for what the site says. But a marked passage may state something he never read, and a sentence can be fluent about nothing, so the mark is a caution rather than a credential: it tells a reader which prose to check against the tables before relying on it.
The mark does not extend to numbers. Tables, verdicts, figures and per-seed values are generated from a run's own outputs and are not drafted by anyone, so they carry no mark in either direction.
What an experiment page holds
An experiment page opens with sections that existed before the run did. Intent states the question the run asks. Background, where a page has one, says which earlier experiments led here and what they left unsettled. Hypothesis states what was expected to happen, on its own, without the reasoning that produced the expectation.
Those sections are committed to version control when the run launches, before any result exists, and the tool that publishes a run refuses to upload one whose prose is uncommitted or has been edited since. The hypothesis on the page is therefore the hypothesis that was made in advance, and not a description of what happened written to sound like one.
Conclusions is the section written afterwards, and it is the only place where the author says what the numbers mean. A run whose Conclusions section reads "No conclusions yet" has finished and been published but not yet interpreted. Between the two sits Curricula on the experiments that need it, describing the schedules the arms were trained on.
Everything between the pre-registered prose and the Conclusions is produced from the run's own outputs, by a reporting step that nobody edits by hand. Setup lists the model, the optimizer settings, the training-data regime and the exact commit the code was at, along with the command that reproduces the run. Checks are integrity tests over the run itself rather than findings: whether every child finished, whether every arm carries the replicates it was supposed to, whether the arms were evaluated against the same instrument. A failing check is a reason to distrust the table below it.
The results table gives each arm's headline number as a mean across seeds, with the standard deviation and a small bar for each individual seed. Read the bars. Several of this project's most useful findings are cases where the seeds split into two groups, one succeeding and one failing, and a mean describes neither of them; a flag on the row marks the split where the report detects it. Accuracies are percentages on minimal pairs, where 50 is what guessing scores, so a number well under 50 means the model learned something other than the intended rule rather than nothing at all.
Many experiments register their expectations as claims: a named comparison between arms, with a direction, a threshold and a rule for how to settle it, written into the manifest before the run. The claims table adjudicates each one against the measured results and prints held, rejected or mixed. The verdict is computed, so a rejected claim stays on the page.
Two badges qualify a row. A claim marked post hoc was added after the results existed and is an observation, not a prediction. A claim marked translated is a structured restatement of a hypothesis that was originally written as prose, added later so the expectation could be settled mechanically; the prose it translates is frozen, but the translation is the author's reading of it.
Following the links
A linked term in the prose resolves to its definition on the capabilities page, and hovering shows that definition without leaving the page. The vocabulary is worth trusting more than the flag names it describes, since several settings do something the name alone does not convey.
Experiments also cite each other. A link into another experiment points at a specific block on its page, usually a stated finding, a registered claim or a single arm, rather than at the experiment as a whole, and it resolves against the committed corpus rather than a stored URL, so it cannot rot into pointing at the wrong thing. The traffic is visible from both ends: a block that other experiments cite grows a "Referenced by" list of the places that cite it.
The front page lists investigations rather than experiments. An investigation is a named line of inquiry with its own intent and its own running synthesis of what its members have shown, and it holds experiments, other investigations, or both. An experiment belongs to at most one of them, and its page names the chain it sits in.
An investigation's Conclusions section is meant to be rewritten as members land, so unlike an experiment's it is not a record of a moment. It says what the line of inquiry currently believes.