How it works

Why Pong

Pong is the Atari game chat models handle worst: below random in the Atari-GPT benchmark, mostly because deciding takes them longer than the game gives them. A live loop is the one job where speed is the whole point, so it is the fair place to see what a fast answer is worth. Alpie Seed is 169Pi's 9B model; with its thinking switched off it reads the state and answers the question directly.

The rules

• Same serve, same rules, same question in every lane.
• One decision moves the ball one step. 8 steps cross the court.
• The model's paddle moves at most 12 units per decision, which is enough reach over a crossing to get anywhere the ball can go.
• The left paddle is a script, not a model. It always returns the ball and it takes no time to decide, so the only difference between lanes is the model on the right.
• When you play, a step never finishes in under 260 ms, so a fast model is a game rather than a reflex test. The floor paces the ball, not the clock: the number beside the court is still Alpie Seed 9B's real round trip.

What the model sees

{"court":{"w":160,"h":100},"ball":{"x":80,"y":50,"vx":19,"vy":4.4},"paddle":{"y":50,"h":20},"interceptY":67.4,"dir":"toward"}

The line-up

• Alpie Seed 9B on 169Pi: the model this site is about, with its thinking switched off.
• Sarvam 105B on Sarvam: the other Indian model family, about 10B parameters active per answer, thinking off.
• Ministral 3 8B on Mistral: the Ministral 3 size closest to Seed's 9B; does not think.
• Qwen 3.8 27B on Cloudflare: three times Seed's size, thinking off.
• Gemini 3.5 Flash-Lite on Google: Google's newest small Gemini, at its lowest reasoning setting; billed per call.
• Nova 2 Lite on Amazon Bedrock: Amazon's small Nova on Bedrock, which does not reason unless asked; size not published; billed per call.

Every lane is asked through its own provider's API with structured output, a JSON schema that allows only up, down or stay, at temperature 0 and the lowest reasoning setting that API accepts. Nobody gets a retry. In the comparison table, Alpie Core's API ignores the schema and writes its reasoning out, so its move is read from its final answer. On Amazon Bedrock the schema is held by a tool call the model must make, since Nova 2 Lite refuses Bedrock's schema output, and Grok 4.7 refuses any temperature, so it is sent none. The number on a lane is the round trip of that one call, timed on the server, and nothing else. A recorded lane plays on past five points, so every lane plays the whole recording; it stops only if its API refuses three calls in a row, as Sarvam's rate limit has done in recordings, more than once.

The rivals are chosen to be fair to Alpie Seed: about its size or larger where the lab publishes a size (Google and Amazon do not, for Gemini 3.5 Flash-Lite and Nova 2 Lite), one per lab, and answering without thinking first, or at the lowest reasoning setting their API accepts. The first rivals are free to start on with no credit card, as Alpie is; the frontier labs' small models that follow are billed per call, beginning with Gemini 3.5 Flash-Lite, which Google's free tier held to 15 requests a minute when a lane makes about 60, then Amazon's Nova 2 Lite on Bedrock. gpt-oss was dropped from the race because its API cannot switch reasoning off, and Grok 4.7 cannot either, so it is in the table below rather than the race, as are Sarvam 105B and both Alpie models with thinking on. Ministral 3 8B is served on Bedrock too, but races on Mistral's own API, as each rival races on its own lab's; the table has both rows.

Every lane that can think races with thinking off through the same switch: the chat template's enable_thinking: false, sent to Alpie Seed, Sarvam and Qwen alike. Gemini cannot turn thinking off at all, so it runs at its lowest setting, Nova 2 Lite reasons only when asked to, and Ministral 3 does not think at all, so neither is sent a switch. Sarvam's reasoning_effort setting cannot turn thinking off either: its docs say null turns reasoning off, but null, “low” (its lowest named level) and leaving it out all thought before answering when tried, so the race does not use it; the table shows “low” as Sarvam's thinking-on row. Alpie Core, in the table only since 26 October 2026, ignores the switch and thinks anyway, which is what its rows are there to show.

Alpie Seed and Alpie Core are called on 169Pi's test endpoint; the rivals are called on their providers' public APIs, free tiers for all but the billed lanes. The two are not necessarily provisioned the same way, and the numbers here do not separate the model from the service in front of it.

More models, same question

The race shows a few models at once. This is the wider check: on 2026-10-27 the same question went to more models, one call per decision, sequential, 30 real game states each, the same states in the same order for every row, including Sarvam 105B and both Alpie models with thinking on as well as off, and Grok 4.7, which cannot stop reasoning. The rows that reason first are run a few states per request, because the Alpie ones take 12 to 15 s a call, and the parts are added up; they still answer the same 30. The first row is the baseline.

Measured from a Vercel function in iad1, each model through its own API, so every latency includes the network from there. The other models answered 386 of 419 correctly between them; Alpie Seed 9B answered all 30 correctly. Alpie Seed 9B with thinking on got no answer to one call (first error: “No object generated: the model did not return a response.”), which is not counted.

A few slow calls set the p95 of 3 rows (Qwen 3.8 27B: 2 of 30 calls took over three times its median, the longest 8.5 s; Gemma 4 26B-A4B: 6 of 30 calls took over three times its median, the longest 11.6 s; Sarvam 105B with thinking on, reasoning effort low (its lowest): 2 of 30 calls took over three times its median, the longest 15.1 s). Nothing is retried and calls go one at a time, so a slow or refused call is the provider's service as much as the model, and from outside we cannot tell which.

Other Indian models

Sarvam 105B is the only other Indian model in the race, because it is the only one we found served free on a provider's own API. Two more are still served, both by Featherless through the Hugging Face router, which charges for every call, so they are measured here rather than raced. On 2026-10-27 they were asked the same question as every lane, with the same schema, at temperature 0 and with no retries, over 30 game states, beside Alpie Seed and Sarvam 105B.

Sarvam-M is Sarvam's post-training of Mistral Small 3.1 24B. Featherless rejects the enable_thinking switch the race sends, so it runs with no switch; it answered without thinking first. Fathom-R1-14B is Fractal's reasoning model, built on DeepSeek-R1-Distill-Qwen-14B, which always thinks unless something stops it: here the schema does. Allowed to think, it ran out of tokens before it answered on both states tried, so it has no thinking row. Featherless files both models' replies under reasoning instead of content; a finished reply there is read as the answer, for every lane alike.

Measured from a Vercel function in iad1, each model through its own API, so every latency includes the network from there. The rows are comparable with each other; the race and the table above are measured from the same place, in separate runs. The other models answered 54 of 90 correctly between them; Alpie Seed 9B answered all 30 correctly.

Not in the table, because they could not be put the question:

• Sarvam 30B (Sarvam): retired from Sarvam's API; open weights with no other host found.
• Krutrim-2 12B (Ola Krutrim): no longer served, as Krutrim Cloud's catalogue now holds only other companies' models.
• Param2 17B-A2.4B (BharatGen): open weights with no hosted API found, under BharatGen's non-commercial licence.
• Zia LLM (Zoho): announced at 1.3B, 2.6B and 7B, with no public API found.
• Soket, Tech Mahindra (Indus), Genloop, CoRover (BharatGPT) and Hanooman: no public chat API or hosted model found.
• Inya VoiceOS (Gnani): a 5B voice-to-voice research preview; Gnani's released models work on speech, not text chat.

Where Ably comes in

POST /api/game leaves a worker running on Vercel. It joins the channel pong:game:<id> as an ordinary member, announces itself in presence as the agent, runs the physics and publishes a state snapshot after every decision. Your browser is another member of the same channel: present as a player, publishing input messages carrying one move, drawing what the channel says rather than a simulation of its own.

That is why the watch link works. Anyone who opens it attaches to the same channel with rewind, so the current state arrives immediately instead of at the next decision, and the number of people watching is that presence set, counted.

Read the code

It is all public, and it is meant to be read. Start with these, in order.

1. 1lib/game/types.ts · the contract: the court, one decision = one tick, and the bytes a model is given
2. 2lib/game/engine.ts · the whole game as pure functions, including the paddle that never misses
3. 3lib/decide/prompt.ts · the one question every model is asked, and its answers
4. 4lib/decide/endpoints.ts · how each lane reaches its model's own API, with no key in the code
5. 5lib/decide/llm.ts · the one timed call: structured output, or a move read from a prose answer
6. 6lib/worker/game-worker.ts · the agent as a member of the channel: the decision loop and the wire
7. 7lib/worker/referee.ts · when a game stops, with no clock of its own
8. 8app/api/game/route.ts · one HTTP request, one function, one game
9. 9lib/ably/hooks.ts · everything a browser does: subscribe, be present, publish input

Play against Alpie · Back to the lanes