🚀 Launch HN: Context.dev (YC S26)

The Unified API for Extracting Structured Web Data for AI Agents

Context.dev provides a powerful Web Scraping and Crawl API specifically engineered for AI agents. Their latest update introduces Monitors, allowing users to track any website for real-time changes.

🛠️ Core Capabilities & Features

Context.dev simplifies the process of turning the messy web into clean, usable data.

• Scraping & Crawling: Retrieve content in markdown, HTML, or via sitemaps. Perform targeted searches or comprehensive full-site crawls.
• Structured Extraction: Use JSON schemas to pull specific data points from any webpage.
• Brand Intelligence:
  • Visuals: Logos, colors, fonts, and full styleguides.
  • Identity: Company descriptions, social media links, and physical addresses.
  • Assets: High-quality images, logo CDNs, and screenshots.
• Classification & Identification:
  • Industry tagging via NAICS and SIC codes.
  • Transaction string matching to identify brands (with optional MCC and location hints).

---

🎯 Use Cases: From AI to Product UX

The platform serves two primary audiences: those building autonomous agents and those enhancing product features.

🤖 For AI Agents

• LLM Empowerment: Provide models with real-time web context.
• Generative AI: Ground RAG (Retrieval-Augmented Generation) in the most current content available.

🎨 For Product Experiences

• Seamless Onboarding: Use autofill capabilities to reduce friction.
• Dynamic Branding: Implement programmatic theming and automated brand kit generation.
• Data Enrichment: Automatically populate and enhance company profiles.

---

🔄 The Evolution of Web Data Acquisition

The transition to Context.dev represents a shift in how developers handle web context. Mathematically, the process can be viewed as: $\text{Raw Website} \xrightarrow{\text{Context.dev API}} \text{Structured JSON/Markdown}$

---

📈 Proven Success Stories

Mintlify: Integrated Context.dev in under 10 minutes to build a tool that converts any GitHub repository URL into a fully branded documentation site.

SiteGPT: Migrated from Firecrawl to Context.dev to scrape entire websites, creating the foundational knowledge bases for their customer support bots.

Sourcely: Leverages the API to crawl academic journals and PDFs at scale, citing superior speed, accuracy, and quality compared to other market alternatives.

---

⚙️ Technical Implementation

Agentic Setup

You can configure the platform manually via the dashboard or let an AI agent handle the integration.

• Sign up for an account.
• Retrieve API key via context.dev/auth.md.
• Follow the docs.context.dev/agent-quickstart guide.
• Recommended: Paste a single line into your coding agent to automate the entire signup and integration process.

Code Example: Structured Extraction

Using zod to define the data shape for the /web/extract endpoint:

import ContextDev from 'context.dev';
import { z } from 'zod';

const contextDevClient = new ContextDev({ 
  apiKey: process.env['CONTEXT_DEV_API_KEY'] 
});

// Define the desired data structure
const schema = z.object({
  pricing_tiers: z.array(z.object({
    name: z.string(),
    price: z.number(),
    billing_cycle: z.enum(['monthly', 'yearly', 'one_time', 'usage_based']),
  })),
});

// Convert Zod schema to JSON Schema and execute
const result = await contextDevClient.extract({
  url: 'https://example.com/pricing',
  schema: schema.toJSONSchema(),
});

console.log(result.data.pricing_tiers);

---

🗺️ Data Flow Architecture

Connect your agent to the web today. From simple scraping to complex brand extraction, Context.dev is the single provider for all your web enrichment needs.