imagining syntax

← mirror_v1

s3_e3

done · peak unseen_mismatch 76.6

α schedule

α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50 α 3.0 ×50

direction
constant
α range
3.0–3.0
stages
24

Figures

schedule.png
schedule.png
trajectory.png
trajectory.png

Results

accuracy by training iteration, mean ± sd over seeds · schedule ends at α = 3.0

seen_* probed at reference α = 1.4 · unseen_* = held-out pairings, uniform (α-independent) · 1000 pairs per condition

iterationseen_matchseen_mismatchunseen_matchunseen_mismatch
50 51.3 ± 2.249.6 ± 0.850.0 ± 1.750.3 ± 1.6
100 56.5 ± 6.653.8 ± 4.253.7 ± 5.850.7 ± 2.5
150 76.1 ± 14.074.2 ± 13.168.9 ± 14.364.5 ± 11.9
200 89.4 ± 3.487.3 ± 3.977.6 ± 5.874.1 ± 6.6
250 90.1 ± 2.089.0 ± 2.076.2 ± 3.873.2 ± 3.8
300 92.2 ± 1.490.7 ± 1.177.6 ± 3.274.2 ± 3.5
350 92.1 ± 2.390.5 ± 1.977.0 ± 5.772.9 ± 4.9
400 91.8 ± 1.989.8 ± 1.876.8 ± 5.372.0 ± 4.6
450 92.9 ± 1.690.9 ± 1.678.4 ± 4.072.6 ± 4.1
500 93.9 ± 1.391.6 ± 1.378.6 ± 3.273.6 ± 3.4
550 93.3 ± 1.591.4 ± 1.379.0 ± 4.474.5 ± 4.7
600 93.3 ± 1.491.0 ± 1.479.0 ± 4.473.1 ± 4.9
650 93.9 ± 1.191.6 ± 1.179.9 ± 4.974.3 ± 6.1
700 93.8 ± 1.291.6 ± 1.578.9 ± 3.773.7 ± 4.6
750 93.8 ± 0.891.8 ± 1.080.3 ± 3.173.8 ± 4.5
800 93.8 ± 1.091.9 ± 1.279.2 ± 3.974.0 ± 4.5
850 93.8 ± 0.992.2 ± 1.480.3 ± 2.775.0 ± 3.9
900 94.5 ± 1.192.2 ± 1.481.6 ± 3.874.7 ± 5.3
950 94.3 ± 1.392.0 ± 1.280.6 ± 4.374.8 ± 4.4
1000 94.2 ± 0.892.2 ± 1.280.9 ± 2.775.1 ± 3.0
1050 94.5 ± 0.992.5 ± 1.782.1 ± 2.675.9 ± 4.6
1100 94.7 ± 0.692.5 ± 1.581.7 ± 3.675.4 ± 4.7
1150 94.8 ± 1.192.5 ± 1.582.7 ± 4.376.6 ± 5.5
1200 94.7 ± 0.992.6 ± 1.082.0 ± 3.175.2 ± 5.2

Downloads