The "pelican benchmark" by Simon Willison arbitrates Kimi K3

An SVG illustration test acts as a quick progress gauge: Simon Willison publishes his reading of K3 on July 16 and 17.

The fact

Simon Willison publishes on July 16, 2026, "Kimi K3, and what we can still learn from the pelican benchmark" and follows up with a complementary post on July 17 ("Quoting Kimi K3" - the model refuses to leak its system prompt). The pelican benchmark - asking the model to draw a pelican in SVG - is used as a simple smoke test: to condense into an image what a model can reason outside of its usual distribution.

Our reading

The pelican benchmark is not a scientific test. Its value: it allows for quick sorting, by eye, without a scoring grid. After a year of discredited synthetic benchmarks, the return to readable visual tests is significant - hype fatigue favors what can be judged without a PDF document. What Willison retains for K3 in this format is a model that speaks but keeps its guardrails (refusal to leak its system prompt), which aligns with Nikkei's reading of the same day on American competitive pressure.

To watch

Independent tests on code and agent benchmarks in the coming week; Anthropic and OpenAI's reaction to pricing.