name: anaphor_n60
investigation: anaphor-binding
intent: |
  \begin{intent}[The anaphor sweep again, at six times the replication]
  \label{anaphor-n60-intent}
  \begin{llm}[claude-opus-5]
  This experiment repeats the anaphor \@{sweep} with 60 \@{replicates} at every
  \@[α]{alpha} instead of 10, and changes nothing else: the same
  \@{anaphor-task} grammar, the same α grid from 0.0 to 3.0 in steps of 0.1,
  the same 256-wide two-layer four-head \@{model} trained for 1200
  \@[iterations]{step} from seed 42, the same \@{waypoints} at 300, 600, 900
  and 1200, and a matching \@{oneshot} \@{arm} at 60 replicates. It is 1,860
  models in the Zipfian arm and 60 in the oneshot arm.

  The first ten \@{seeds} at each α are the ones the earlier run used, because
  the seed derivation depends only on the α grid and the base \@{seed}, both
  unchanged. That makes the earlier run a strict subset of this one and its ten
  replicates a free consistency check, and it is also why the pre-registered
  test below is restricted to the fifty that are new.

  The point of the extra replication is one quantity: how often a training run
  ends up with the binding rule at all. The earlier sweep could not measure
  that to better than ±0.16, which is wider than any difference worth arguing
  about. At 60 replicates the standard error is 0.065, and pooling five
  neighbouring α values brings it to 0.029.
  \end{llm}
  \end{intent}
background: |
  \begin{background}[An answer the first run could not give]
  \label{anaphor-n60-background}
  \begin{llm}[claude-opus-5]
  The first anaphor sweep settled one thing and failed to settle another.
  It settled that accuracy collapses above α of about 2.3
  (\@[the plateau and its cliff]{anaphor-plateau-and-cliff}), and that the
  collapse is \@[erosion]{erosion} rather than a failure to acquire the rule,
  since the high-α and plateau \@{seeds} are indistinguishable at 300
  \@[steps]{step} and separate only afterwards. What it could not settle is
  whether anything happens inside the plateau. The best single α there stood
  at 8 of 10 against 110 of 220 for the rest, which is p = 0.103, and the mean
  curve put its maximum at a third α again. Ten replicates cannot tell a real
  rise of 0.15 from noise; the test had about 0.10 power.

  The reason a proportion is the quantity of interest, rather than the mean
  accuracy the reports lead with, is that the outcome is close to
  all-or-nothing per seed (\@[two basins, not a
  spread]{anaphor-all-or-nothing}). A \@{seed} finishes either holding the rule
  or not holding it, so a mean over seeds is a mixture weight wearing the units
  of an accuracy. Both summaries describe the same measurements and this run
  reports both.
  \end{llm}
  \end{background}
hypothesis: |
  \begin{hypothesis}[The cliff replicates; the interior is the open question]
  \label{anaphor-n60-hypothesis}
  \begin{llm}[claude-opus-5]
  The collapse above α of 2.3 should replicate at full strength. Inside the
  plateau the registered expectation is that there is no interior structure to
  find, and that the apparent rise near α of 1.0 to 1.2 in the first run was
  the noise its power predicted.

  Three summaries of the same 1,920 measurements are reported, and the analysis
  is fixed here before the run so that it cannot be chosen afterwards to suit
  the answer. The first is mean \@{unseen-mismatch} at 1200 \@[steps]{step},
  which is what every other experiment in this corpus reports and is kept for
  comparability. The second is the proportion of replicates finishing above 90,
  which is the quantity the two-basin structure makes meaningful; the threshold
  is 90 and results at 80 and 95 are reported alongside it so the choice can be
  seen not to matter. The third is a logistic regression of that same
  above-90 outcome on α across the plateau, with α entering smoothly, tested by
  likelihood ratio against a flat fit. The regression is the primary test,
  because it uses every replicate, needs no threshold on α, and asks the
  question directly: is the plateau flat.

  The confirmatory test runs on the fifty replicates that are new at each α.
  The first ten repeat the seeds that generated the hypothesis being tested,
  and including them would let the data vouch for itself. All sixty are used
  for estimation and for the published curve.

  If the regression finds no curvature the plateau is flat, α does not
  influence whether the \@{reflexive} finds its subject anywhere below 2.3, and
  the only thing concentration does to this dependency is destroy it at the
  top. If it finds curvature, the location and height of the maximum become
  measurable for the first time, and the comparison against the verb task's
  peak at 1.2 becomes a real comparison rather than an argument about noise.
  \end{llm}
  \end{hypothesis}
seed: 42
base_args: [--task, anaphor]
runs:
  - label: zipfian
    args: [--parameter-range, "0.0,3.0", --step, "0.1", --n-iterations, "60",
           --max-iter, "1200", --waypoints, "300", "600", "900", "1200"]
  - label: oneshot
    args: [--oneshot, --n-iterations, "60",
           --max-iter, "1200", --waypoints, "300", "600", "900", "1200"]
compare:
  metric: unseen_mismatch
