zipfian

done · peak unseen_mismatch 87.5

Configuration

n_layer: 2
n_head: 4
n_embd: 256
base_seed: 42

Figures

Results

at final waypoint (t = 1200), mean ± sd over seeds

seen_* generated at each grid point's own training α · unseen_* = held-out pairings, uniform (α-independent) · 1000 pairs per condition

α	seen_match	seen_mismatch	unseen_match	unseen_mismatch
0.0	100.0 ± 0.0	75.2 ± 27.8	100.0 ± 0.0	75.4 ± 27.9
0.1	100.0 ± 0.0	72.7 ± 32.6	100.0 ± 0.0	72.7 ± 32.3
0.2	100.0 ± 0.0	76.0 ± 30.2	100.0 ± 0.0	76.1 ± 30.1
0.3	100.0 ± 0.0	76.4 ± 27.8	100.0 ± 0.0	76.1 ± 28.0
0.4	100.0 ± 0.0	79.0 ± 28.4	100.0 ± 0.0	78.8 ± 28.5
0.5	100.0 ± 0.0	69.9 ± 29.8	100.0 ± 0.0	70.1 ± 29.8
0.6	100.0 ± 0.0	76.7 ± 29.7	100.0 ± 0.0	76.9 ± 29.3
0.7	100.0 ± 0.0	74.3 ± 26.3	100.0 ± 0.0	74.6 ± 26.1
0.8	100.0 ± 0.0	79.6 ± 24.5	100.0 ± 0.0	79.8 ± 24.3
0.9	100.0 ± 0.0	72.2 ± 28.3	100.0 ± 0.0	72.2 ± 28.3
1.0	100.0 ± 0.0	75.5 ± 30.2	100.0 ± 0.0	75.9 ± 30.0
1.1	100.0 ± 0.0	74.7 ± 28.9	100.0 ± 0.0	74.3 ± 29.0
1.2	100.0 ± 0.0	81.4 ± 26.6	100.0 ± 0.0	81.3 ± 26.7
1.3	100.0 ± 0.0	76.9 ± 26.1	100.0 ± 0.0	77.0 ± 26.0
1.4	100.0 ± 0.0	76.4 ± 27.6	100.0 ± 0.0	76.2 ± 27.6
1.5	100.0 ± 0.0	80.6 ± 23.5	100.0 ± 0.0	80.6 ± 23.8
1.6	100.0 ± 0.0	73.7 ± 25.3	100.0 ± 0.0	73.5 ± 25.6
1.7	100.0 ± 0.0	70.7 ± 25.8	100.0 ± 0.0	70.6 ± 26.3
1.8	100.0 ± 0.0	73.4 ± 25.9	100.0 ± 0.0	73.6 ± 25.9
1.9	100.0 ± 0.0	70.7 ± 27.6	100.0 ± 0.0	70.6 ± 27.5
2.0	100.0 ± 0.0	75.3 ± 25.3	100.0 ± 0.0	75.1 ± 26.2
2.1	100.0 ± 0.0	70.4 ± 27.7	100.0 ± 0.0	70.1 ± 28.2
2.2	100.0 ± 0.0	66.0 ± 24.3	100.0 ± 0.0	65.9 ± 25.2
2.3	100.0 ± 0.0	66.6 ± 22.1	100.0 ± 0.0	66.3 ± 22.6
2.4	100.0 ± 0.0	68.3 ± 22.3	100.0 ± 0.0	68.7 ± 22.8
2.5	100.0 ± 0.0	62.1 ± 24.8	100.0 ± 0.0	62.1 ± 25.8
2.6	100.0 ± 0.0	56.4 ± 26.5	100.0 ± 0.0	55.9 ± 27.0
2.7	100.0 ± 0.0	58.7 ± 22.4	100.0 ± 0.0	57.8 ± 22.5
2.8	100.0 ± 0.0	53.8 ± 21.0	100.0 ± 0.0	53.5 ± 21.3
2.9	100.0 ± 0.0	55.4 ± 21.3	100.0 ± 0.0	55.1 ± 22.4
3.0	100.0 ± 0.0	57.0 ± 23.3	100.0 ± 0.0	56.5 ± 23.4

zipfian

Configuration

Figures

Results

Downloads