name: anaphor
investigation: anaphor-binding
intent: |
  \begin{intent}[Does Figure 2's shape survive moving the dependency]
  \label{anaphor-intent}
  \begin{llm}[claude-opus-5]
  This experiment runs the paper's \@{sweep} again with one thing changed:
  what has to agree. The \@{anaphor-task} keeps the \@{sentence} space, the
  \@[Zipfian]{alpha} sampling over noun-verb \@{pairings}, the held-out
  \@{unseen-pairings} and the \@{model} exactly as the reproduction had
  them, and moves \@{number} off the verb. Verbs are regular past-tense forms
  that never inflect, and every sentence ends in a \@{reflexive},
  \texttt{himself} or \texttt{themselves}, that must agree with the subject.
  Each \@{minimal-pair} flips that reflexive rather than the verb.

  Every noun in a training sentence carries the subject's number, so the
  training data never distinguishes binding the subject from binding the first
  noun or the nearest one; the \@{mismatch} conditions are what separate them.
  The Zipfian \@{arm} sweeps α from 0.0 to 3.0 in steps of 0.1 with ten
  \@{seeds} at each value, and the \@{oneshot} arm adds ten more at the
  deterministic 1:1 limit. Both train the 256-wide, two-layer, four-head model
  for 1200 \@[iterations]{step} from seed 42, and both record
  \@{waypoints} at 300, 600, 900 and 1200 \@[steps]{step}, so a dependency
  that turns out to be easy shows its \@{saturation} in the trajectory rather
  than hiding behind a ceiling at the end.

  One difference from the reproduction is worth stating before any number is
  read. A \@{waypoint} \@{run} reports the weights at a fixed
  \@[budget]{budget}, where the reproduction reported the checkpoint with the
  lowest validation \@{loss}; across that run's 320 \@{replicates} those two
  coincided 275 times, and the other 45 stopped at 900 or 600
  \@[steps]{step}. The α axis is estimated the same way at every grid point
  here, so the shape of the curve is internally consistent, but a peak
  location compared against the reproduction's is compared across two
  estimators.
  \end{llm}
  \end{intent}
background: |
  \begin{background}[What the reproduction established]
  \label{anaphor-background}
  \begin{llm}[claude-opus-5]
  The reproduction found the paper's figure intact: generalization to
  \@{unseen-pairings} on the hardest condition peaks at an intermediate
  \@[α]{alpha} rather than climbing as the \@{pairings} grow more repetitive
  (\@[the intermediate peak]{paper-intermediate-peak}), and the
  \@{oneshot} limit, whose pairings are perfectly consistent, generalizes
  worse than the best Zipfian setting
  (\@[oneshot memorizes]{paper-oneshot-memorizes}). The reading offered for
  that shape is \@{collocational-bootstrapping}: repetition gives the
  \@{model} a reason to represent the subject, and variation stops it from
  solving the task by memorizing pairs.

  What the reproduction cannot say is whether the thing repetition builds is
  positional or lexical. Every measurement in it was taken on the same verb
  whose partner distribution α controls, so a subject representation and a
  well-learned partner list predict the same curve. Separating them needs a
  dependency that requires the subject and never appears in the α
  distribution.
  \end{llm}
  \end{background}
hypothesis: |
  \begin{hypothesis}[An interior optimum, displaced upward]
  \label{anaphor-hypothesis}
  \begin{llm}[claude-opus-5]
  The registered expectation is that \@{unseen-mismatch} accuracy peaks at an
  interior \@[α]{alpha} well above 1.4: locating a subject with no agreement
  task sharpening the representation should need more concentration per verb
  than the paper's setting supplied, so the optimum moves up rather than
  landing on 1.4 again. \@{seen-match} should sit near ceiling at every α; if
  it does not, nothing else in the run is interpretable.

  Two other outcomes are worth naming in advance because they say different
  things. A flat \@{unseen-mismatch} curve with \@{seen-match} at ceiling is
  a clean negative: concentrating noun-verb \@{pairings} does not build a
  subject representation the \@{reflexive} can reach, and the mechanism is
  lexical. A curve that rises monotonically to α = 3.0 would say the
  high-concentration failure mode does not bite here, which would in turn
  mean the \@{seen-unseen-pairings} axis is carrying no signal at the tested
  slot and the run has two live conditions rather than four.
  \end{llm}
  \end{hypothesis}
seed: 42
base_args: [--task, anaphor]
runs:
  - label: zipfian
    args: [--parameter-range, "0.0,3.0", --step, "0.1", --n-iterations, "10",
           --max-iter, "1200", --waypoints, "300", "600", "900", "1200"]
  - label: oneshot
    args: [--oneshot, --n-iterations, "10",
           --max-iter, "1200", --waypoints, "300", "600", "900", "1200"]
compare:
  metric: unseen_mismatch
