Bicycle Test

The Bicycle Test gives AI models one short instruction, to make an SVG of a bicycle, and shows what comes back. The instruction contains a typo, and the typo is kept. Every model received the same text. Each drawing on the page is the SVG as the model returned it, with the model's name, its setting, token count, cost and number of attempts beside it.

The outputs are not graded or ranked. Each is one sample from one model at one setting, so the page puts them side by side and leaves the looking to the reader. Results come in three rounds, July, September and October 2026, newest first.

Make an SVG of a bicyle.

How it was run

The July round used the Google API with Gemini 2.5, 3.1 and 3.5 models, each at two settings for how much the model thinks before answering. The September round sent five models through OpenRouter, which routed each request to the provider named on its card: Claude Fable 5.1, GPT-6 Astra, Gemini 3.8 Flash, GLM 5.3 and GLM 5.3 Flash. Four of them ran at high reasoning; GLM 5.3 is shown at low. The October round called Claude Haiku, Sonnet and Opus 5.5 through the Anthropic API and three Meta models, Muse Spark 1.3, Wonder and Wonder Full, through the Muse command-line tool, all at high effort.

Failed attempts are kept and counted on the cards. In July many requests were retried because of rate limits and output limits, and three model and setting combinations never returned a finished SVG: Gemini 2.5 Flash-Lite at high, Gemini 3.1 Pro preview at high and Gemini 3.5 Flash at high. Those cards say so instead of showing a drawing. In September the first high-reasoning request to GLM 5.3 used up its 16,384 output tokens without an SVG, and a later high-reasoning retry was stopped once the low-reasoning request had produced one. In October the first Wonder Full request used a model name that does not exist, and the second attempt returned the SVG.

Costs are recorded as each route reports them. July costs are calculated from token counts and recorded prices. September costs are the charges OpenRouter reported. The Claude costs in October are estimated from standard API prices, and the Muse tool reported no token counts or charges, so those cards show none. Because each card is a single run, nothing here ranks the models.