The Mystery of the Generic AI Pelican

The Emperor Pelican Has No Clothes Published: October 14, 2026

Have you ever wondered why AI-generated pelicans always seem to follow the same boring pattern?

When users input a specific prompt—for instance, Generate an SVG of a pelican riding a bicycle—the results are often strikingly similar. Based on runs by Simon Willison (using the highest reasoning settings available for SVG output), there is a glaring discrepancy between what LLMs do produce and what they could produce.

This phenomenon is a symptom of LLM diversity collapse, a critical issue in generative AI that remains largely ignored.

---

🎨 The Design Creativity Bench

To quantify this lack of variety, research was conducted into "diversity collapse" specifically within the realm of UI design. This led to the creation of the Design Creativity Bench, a framework designed to test whether AI can actually "think" creatively or if it is simply mimicking a narrow set of patterns.

The Research Objective

The goal was to determine if frontier models could move beyond basic functionality and achieve true design innovation.

• Evaluate 16 frontier AI models.
• Test across 80 open-ended UI tasks.
• Adapt tasks to two different product domains.
• Compare results against professional human designers.

How Creativity is Measured

Unlike standard benchmarks that look at a single quality metric, this study utilizes a three-pronged approach to define creativity:

1. Originality: The degree to which a model's output differs from other models given the same prompt.
2. Creative Range: The ability of the model to pivot its design logic when the product domain changes.
3. Appropriateness: The extent to which the UI actually fulfills the functional needs of the brief.

Mathematically, the gap in performance can be viewed as: $\text{Creativity Gap} = \text{Score}_{\text{Human}} - \text{Score}_{\text{Model}}$

---

📊 The Results: Human vs. Machine

The data indicates that while AI can build a functional site, it fails to innovate. The following table highlights the top performers:

Key Finding: Even the most advanced model, GPT-6 Astra, falls significantly short of human designers, scoring only 65.5 compared to the human benchmark of 92.3.

---

🛠️ Evaluation Workflow

The following diagram illustrates how the Design Creativity Bench processes model outputs:

🔗 Resources & Further Reading

For those interested in the technical implementation or the full data set, the team has made the following resources available:

• Full Leaderboard: kombai.com/benchmark/
• Academic Paper: arxiv.org/abs/2610.11539
• Dataset: HuggingFace/Kombai/design-creativity-bench
• Source Code: github.com/kombai-io/design-creativity-bench

# Example of how one might track diversity collapse
def calculate_diversity(outputs):
    unique_patterns = set([out.pattern for out in outputs])
    return len(unique_patterns) / len(outputs)

# Result for LLM Pelicans: 0.12 (Low Diversity)
# Result for Human Designers: 0.89 (High Diversity)