7. Postmortem and results

We set a goal on 10-06: reach #10 on the competition server. We didn’t. Our best recorded rank was #16, the last one we recorded was #24, and we have no record of a final end-of-competition rank. This post is the scoreboard and what we make of it.

The main lesson is the plainest one. Throwing more tokens at the problem does win: our rating rose about linearly with the tokens we put in. The stall after 10-06 fits that, since rounds shrank from six workers to three or four agents and one session stopped at its weekly usage limit.

The ladder

The server rating is Elo-like and reset to 1500 when the 1.2.3 queen rules shipped on 10-01. These are the points we wrote down, with the bot that was live (bots are described in what the factory made).

Date	Rank	Rating	Live bot
10-01	#72 of 947	~1767	arena-search-7 (v9)
10-01		1797	gst-giant-2 (v10), then reset to 1500
10-04	#35	1832	the teammate’s queen-ring17
10-04		1977	ring-plus-4’s window (from 1831)
10-05	#22 → #26	2035 → 2012	ring-sched-8, then ring-sched-17
10-06	#31 → #28 → #22 → #16	2032 → 2038 → 2099 → 2139	ring-contest-1, then ring-sprint-1
10-07	#19 → #23	2146 → 2131	ring-sprint-1 (#10 was 2270)
10-07/08	#28 → #29 → #27	2112 → 2104 → 2120	ring-sprint-1, then ring-field-3
10-08	#35	2095	ring-field-3
10-09	#29 → #24	2150 → 2194	ring-sealed-2, ring-shield-3, ring-shield-10

Two caveats sit under that table. First, the rating ran about 90 above our measured, rating-adjusted performance: at #16, ring-sprint-1 performed at 2046. Second, several jumps were not the bot. ring-sealed-2’s +78 came from opponents’ ratings drifting up; on 17 common opponents it won exactly as many games as the previous bot’s rates predicted. By our own measurements the real gains all came between 10-04 and 10-06 (see what the factory made). After that the ladder mostly wandered between 2100 and 2200. #1 (aAah) was at 2526 on 10-07.

The sprint tournament

On 10-01 the server ran a sprint tournament: 591 teams, best of 7, on 7 new maps (Australia, Islands, Maze, Stripes, Tower Defense, Around UNSW, weakhold). It ran under the old 1.2.2 rules, and our live bot was v11 (gst-search-4). We were seeded #67, won 5 matches, and lost the round of 16 3-4. Over the tournament we went 31-11 in games, worst on Tower Defense (1-4). Cutlery beat PPP 4-3 in the final.

A strategist went through 140 tournament replays afterwards. Winners covered the map faster: the winner had visited more of the board in 94 of 112 games, and our heads had covered 23% by r100, the lowest of the 17 teams measured. On Tower Defense we never left the home box. Winners also started consolidating 50-100 rounds before us.

The numbers

The arena played 291,668 games between 680 submissions, 441 of them from the agents. Before it, the local league had played 170,039 matches between 206 variants. The agents left 545 worker briefs and 78 strategist files behind.

Commit volume traced the method. It was heaviest in the first two days, with six workers on 20-minute tasks in the local league; it fell from 10-01 to 10-03 while long-running teams worked and the arena was built; it rose again from 10-04 with loss analysis, strategists and short fresh workers.

What worked

• Real games as the signal. Every durable gain was found by analysing server losses, then making one change. The ring line’s climb (#35 to #16 in two days) came out of that loop.
• Short workers with one change each, judged externally. Lines climbed by stacking single changes that beat their parent: gst went 15 → 23 → 27 → … → 131, ring-plus 1 → 4 → 7 → 9 → 16 → 19 → 21.
• Taking a teammate’s bot seriously. queen-ring17 was better on the server than anything the agents had made, and building on it was the biggest single step.
• Fixing the measurement before trusting it. Two bugs made early results worthless: the sim seeded pearls differently from the judge, and a compiler flag made native builds diverge from the wasm judge. Finding them changed which bots we believed in.
• Fresh maps as a check. Without generated maps we would have shipped the centre rush as a general gain.

What didn’t

• The arena drifted from the server. Its server suite had 2 fixed seeds per map and its pool was our own bots, so queen-guard-7 and ring-plus-4 looked better at home than on the ladder.
• Consolidation. Every attempt to feed one champion was neutral or worse in every line. The gap was always economy before r300.
• Long-running teams. In round 9 their contexts grew and they drifted; about 15 verdicts landed inside noise.
• Per-round merges. Bundling a round’s changes into one draft mixed effects; we reversed it on 10-06.
• The work after 10-06. From 10-07 to 10-08 the ladder stalled around 2100 and we reset the orchestrator to its 10-06 instructions. Most of the ring-field, ring-sealed and ring-shield changes measured as flat on the server.

What we would do differently

• Judge on the server, per map and rating-adjusted, from the first day, and treat our own arena as a filter only. That became the written rule only on 10-04, after queen-guard-7.
• Put non-family bots in the arena pool early. spar-field-2 beat its parent 67-45 but did worse against non-ring bots; a pool of our own relatives rewards beating our own habits.
• Spend more of the effort on the mid-game army, which the eval crawl pointed at on 10-05.
• Test every opening on generated map families before reading an arena gain.

What’s left

The arena is kept as a static snapshot where any game re-runs in the browser, and the code is public under Battlecode-UnderTheC: syeyoung-tokens for the bots, the orchestrator instructions and the generation reports, tokenz-arena for the arena and the eval model, and syeyoung-rl for the RL attempt.