One absurd frog prompt is exposing how differently every major LLM handles a bizarrely specific visual instruction

Claude Opus 5 out-reasoned 13 rivals on one absurd frog prompt that standard benchmarks miss

Claude Opus 5 produced anatomically annotated vector graphics for a prompt asking AI models to generate an SVG of a frog with a Habsburg jaw, while most of the 13 other models in the same benchmark produced frogs whose jaw additions were disconnected blobs that made no anatomical sense. The benchmark, called Frogs and hosted at frogs.vaguespac.es, gives each of 14 models three attempts per month. In August 2026, all 42 runs produced SVG output 1. Every model can write working vector-graphics code. The gap is semantic: combining a concrete object with a culturally specific anatomical modifier into output that actually depicts both.

The prompt layers demands that no standard benchmark tests together. A model must know what a Habsburg jaw is (mandibular prognathism, a protruding lower jaw linked to generations of royal inbreeding), translate that deformity onto a frog's face, and implement it as functional SVG 2. A Hacker News discussion that brought the benchmark wider attention drew a direct contrast with the well-known "pelican riding a bicycle" SVG test, arguing that the frog prompt is harder because it hinges on an anatomical feature that models may or may not connect to royalty 2.

What the SVG source code reveals about model reasoning

The most revealing artifact on the Frogs site is not the rendered images but the code comments embedded in the SVG markup. Claude Opus 5's three submissions include annotations that go well beyond labeling geometric shapes. One run contains a comment reading "HABSBURG JAW: massive protruding mandible." Another describes the upper lip as "recessed, tucked behind the jaw" and the lower teeth as "protruding over the upper lip" 1. A third run adds "droopy regal eyelids," a detail no frog anatomy requires 1. The benchmark site itself notes that these annotations offer "anatomical interpretation beyond plain labeling" 1.

That distinction matters because it separates two very different operations. A model that labels a shape "circle" is annotating geometry. A model that labels a shape "massive protruding mandible" is demonstrating it parsed the compound instruction and reasoned about how the requested deformity would distort a frog's face. It also has a Hugging Face dataset and a GitHub harness, which means every run is archived as structured data rather than screenshot impressions 1.

The benchmark creator, posting on Hacker News, identified a pattern visible only when you compare all 14 models side by side: seven of fourteen models silently imported royalty into a prompt that named only an anatomical feature 2. The prompt never mentions kings, crowns, or dynasties. It says "Habsburg jaw," which is a medical term. Yet half the field drew regal styling. Only two of those seven models showed any awareness they were extrapolating beyond the literal request 2.

That ratio is the benchmark's sharpest finding. One hundred percent of runs produced SVG. Fourteen percent of models demonstrated that they recognized their own inference when they made it. The 86-point gap between those two numbers is the difference between a model that can generate code and one that understands what it is coding. Neither the benchmark site nor the Hacker News discussion frames the gap in those terms.

Why this prompt catches what standard evaluations cannot

The prompt works because it stacks four capabilities into a single instruction, and a model must handle all four to produce coherent output:

• Code generation: every run must produce syntactically valid SVG that renders without errors 1
• Visual reasoning: the model must spatially arrange body parts so the jaw, mouth, and teeth sit in anatomically plausible positions 1
• Historical knowledge: the model must retrieve what mandibular prognathism is and connect it to the Habsburg dynasty 2
• Conceptual synthesis: the model must merge a frog's anatomy with a human deformity into something that reads as both at once 1

Standard benchmarks isolate each of these. A code-generation suite tests syntax. A knowledge exam tests recall. An art-generation test checks rendering. None ask a model to do all four simultaneously, which is exactly what makes the failure modes so visible. A commenter on the Hacker News thread observed that many models drew the frog face correctly and added a protruding shape for the jaw, but the two elements were not connected in any way that made sense 2. The model parsed the words, assembled the parts, and never reconciled them.

Commenters ranked Claude Opus 5 first among the 14 models, followed by Kimi K3 and Grok 4.5 2. The benchmark creator separately named Gemini 3.6 Flash as a personal favorite for aesthetic quality 2. Those rankings are subjective. But they point to a structural divide: the models that parsed the compound concept produced richer SVGs with anatomically specific annotations. The ones that parsed the words individually satisfied the syntax of the prompt without grasping what it asked them to draw.

For developers choosing models for multi-constraint instruction following, the practical signal is this. Benchmarks that test one capability at a time will not tell you whether a model can hold four constraints in working memory and satisfy them simultaneously. The Frogs benchmark is informal by academic standards, run by a single person with a GitHub harness and a strange prompt. But the capability gap it surfaces is the one that matters when you ask a model to do something its training data never explicitly covered. If you are building a product that depends on a model translating a compound, real-world instruction into working output, test it on something nobody has asked it to do before. The models that look identical on standard benchmarks will not look identical there.