Stop Classifying—Start Hallucinating

August 10th, 2026

Using Large Language Models (LLMs) to categorize search queries or products has become groundbreaking boring. However, the real challenge remains: how do you force an LLM to stick to a strict, legal vocabulary of categories, colors, or brands that your database actually supports?

The Problem: The "Legal" Vocabulary Struggle

Consider the Wayfair WANDS e-commerce dataset. If a user searches for a "wood coffee table," you need to map that query to a specific category. The problem is that the taxonomy is massive, containing hundreds of options such as:

• Furniture / Office Furniture / Desks
• Furniture / Living Room Furniture / Coffee Tables End Tables / Coffee Tables
• Furniture / Living Room Furniture / Coffee Tables End Tables / End Side Tables
• Décor Pillows / Decorative Pillows Blankets / Throw Pillows
• Furniture / Bedroom Furniture / Dressers Chests

The Traditional Approach: Structured Outputs

The standard move is to use structured outputs, forcing the model to choose from a Literal list of allowed values. In Pydantic, it looks like this:

from typing import Literal
from pydantic import BaseModel, Field

# Imagine this list contains 500+ real taxonomy strings
FullyQualifiedClassifications = Literal[ 
    'Furniture / Bedroom Furniture / Beds Headboards / Beds', 
    'Furniture / Living Room Furniture / Chairs Seating / Accent Chairs', 
    'Rugs / Area Rugs', 
    # ... 497 more entries
]

class QueryClassification(BaseModel): 
    """ 
    Structured representation of a search query for furniture e-commerce.
    """ 
    classifications: list[FullyQualifiedClassifications] = Field( 
        description="A possible classification for the product." 
    )

response = client.responses.parse( 
    model="gpt-5.4-mini", 
    input="Classify the query: brown coffee table", 
    text_format=QueryClassification, 
) 
print(response.output_parsed.message) 
# Result: Furniture / Living Room Furniture / Coffee Tables End Tables / Coffee Tables

While this works, it is inefficient. Sending a massive schema of 500+ literals in every request is expensive and eventually hits the model's context limit.

---

The Alternative: The "Hallucination" Pattern

Instead of restricting the LLM, why not let it be creative? You can use a "dumb" (cheaper) model to simply invent a plausible category.

The Hallucination Prompt: "Your task is to create novel, never seen before, furniture, home goods, or hardware classification that best fit a search query.

Examples of classifications:

• Furniture / Living Room Furniture / Coffee Tables End Tables / Coffee Tables
• Décor Pillows / Decorative Pillows Blankets / Throw Pillows
• Kitchen Tabletop / Kitchen Organization / Food Storage Canisters

Query to generate classification for: brown coffee table"

By doing this, we stop sending the legal list. The LLM might return something like: Furniture / Living Room / Tables / Coffee

At first glance, this is useful useless because that exact string doesn't exist in your database. But here is the trick.

The Technical Bridge: Vector Embeddings

You can map these "hallucinations" to real categories using semantic similarity.

1. Pre-compute: Create an in-memory set of embeddings for all real Wayfair classifications using a model like MiniLM.
2. Embed: Generate an embedding for the LLM's fake category.
3. Match: Use a dot product to find the closest real neighbor.

The mathematical similarity is calculated as: $\text{similarity} = \mathbf{v}_{\text{fake}} \cdot \mathbf{v}_{\text{real}}$

The Workflow Logic

Implementation Checklist

• Generate embeddings for the entire legal taxonomy.
• Prompt a small LLM to "invent" a category.
• Convert the invention into a vector.
• Perform a nearest-neighbor search.

Comparison: Structured vs. Hallucinated

The bottom line: You don't need to ship your entire schema to the LLM every time. Just let it imagine the answer, then use math to find the truth.

---

Upcoming events: Vectors Week — Join me for a series of events focusing on vector retrieval, hybrid search, and the architecture of custom vector databases.

Doug Turnbull [Twitter] | [LinkedIn] | [Newsletter] | [Bsky]