zipfian

done · peak unseen_mismatch 97.7

Configuration

n_layer: 2
n_head: 4
n_embd: 256
base_seed: 42

Figures

Results

at final waypoint (t = 1200), mean ± sd over seeds

seen_* generated at each grid point's own training α · unseen_* = held-out pairings, uniform (α-independent) · 1000 pairs per condition

α	seen_match	seen_mismatch	unseen_match	unseen_mismatch
0.0	100.0 ± 0.0	52.5 ± 28.8	100.0 ± 0.0	53.1 ± 29.0
0.1	100.0 ± 0.0	66.4 ± 41.4	100.0 ± 0.0	66.2 ± 40.7
0.2	100.0 ± 0.0	85.4 ± 18.1	100.0 ± 0.0	84.8 ± 19.1
0.3	100.0 ± 0.0	79.8 ± 25.8	100.0 ± 0.0	79.9 ± 25.1
0.4	100.0 ± 0.0	84.7 ± 24.9	100.0 ± 0.0	84.3 ± 25.2
0.5	100.0 ± 0.0	68.9 ± 31.7	100.0 ± 0.0	69.0 ± 31.5
0.6	100.0 ± 0.0	80.3 ± 26.9	100.0 ± 0.0	80.8 ± 26.1
0.7	100.0 ± 0.0	84.2 ± 18.1	100.0 ± 0.0	84.4 ± 17.7
0.8	100.0 ± 0.0	82.8 ± 20.4	100.0 ± 0.0	82.2 ± 21.1
0.9	100.0 ± 0.0	68.3 ± 33.8	100.0 ± 0.0	69.2 ± 33.1
1.0	100.0 ± 0.0	76.3 ± 25.6	100.0 ± 0.0	76.5 ± 25.8
1.1	100.0 ± 0.0	81.5 ± 30.4	100.0 ± 0.0	81.2 ± 30.1
1.2	100.0 ± 0.0	85.4 ± 30.0	100.0 ± 0.0	85.4 ± 29.8
1.3	100.0 ± 0.0	71.7 ± 26.8	100.0 ± 0.0	71.8 ± 26.7
1.4	100.0 ± 0.0	75.3 ± 27.4	100.0 ± 0.0	75.3 ± 27.3
1.5	100.0 ± 0.0	79.2 ± 22.6	100.0 ± 0.0	78.9 ± 22.5
1.6	100.0 ± 0.0	85.8 ± 17.3	100.0 ± 0.0	85.7 ± 16.9
1.7	100.0 ± 0.0	66.9 ± 30.3	100.0 ± 0.0	66.6 ± 31.1
1.8	100.0 ± 0.0	74.4 ± 26.7	100.0 ± 0.0	74.1 ± 26.9
1.9	100.0 ± 0.0	82.0 ± 28.5	100.0 ± 0.0	81.7 ± 28.5
2.0	100.0 ± 0.0	82.3 ± 19.3	100.0 ± 0.0	82.2 ± 21.5
2.1	100.0 ± 0.0	75.0 ± 22.6	100.0 ± 0.0	75.2 ± 22.2
2.2	100.0 ± 0.0	75.6 ± 20.3	100.0 ± 0.0	76.0 ± 21.3
2.3	100.0 ± 0.0	70.9 ± 25.0	100.0 ± 0.0	70.9 ± 25.5
2.4	100.0 ± 0.0	65.6 ± 26.0	100.0 ± 0.0	66.1 ± 25.7
2.5	100.0 ± 0.0	51.5 ± 28.1	100.0 ± 0.0	51.5 ± 29.4
2.6	100.0 ± 0.0	62.0 ± 25.0	100.0 ± 0.0	61.4 ± 25.4
2.7	100.0 ± 0.0	59.5 ± 21.3	100.0 ± 0.0	59.7 ± 19.8
2.8	100.0 ± 0.0	59.3 ± 19.1	100.0 ± 0.0	58.4 ± 19.4
2.9	100.0 ± 0.0	52.1 ± 20.4	100.0 ± 0.0	50.3 ± 20.5
3.0	100.0 ± 0.0	55.8 ± 23.4	100.0 ± 0.0	55.1 ± 23.3

zipfian

Configuration

Figures

Results

Downloads