Keenable SELECT: Web Search via SQL

Keenable SELECT is an advanced agentic system designed to treat the entire internet as a queryable database, allowing users to perform complex web research using SQL syntax.

The system generates comprehensive research reports where every output is backed by a transparent "trajectory"—a complete log of every query executed, the resulting tool outputs, and the final data sets.

🛠️ How the Process Works

When a user submits a complex request, such as: "Which AI researchers moved between frontier labs since 2025? For each move list the researcher, the lab they left, where they went and the month," the agent doesn't just "browse"; it executes a structured query.

The Logic Flow

Example SQL Execution

The agent translates the natural language request into a specialized SQL statement:

SELECT 
    SEM_EXTRACT(content, 'researcher'), 
    SEM_EXTRACT(content, 'left lab'), 
    SEM_EXTRACT(content, 'joined lab'), 
    SEM_EXTRACT(content, 'move month') 
FROM WEB_SEARCH(8 diverse queries) 
WHERE SEM_MATCH(content, 'named researcher moving between frontier labs, 2025+')

In terms of computational cost, the system optimizes the process: $\text{Total Cost} = (\text{Cheap SQL Filters}) + \sum (\text{Expensive LLM Extractions per Row})$

---

📚 Research Gallery

Below is a collection of reports generated by Keenable SELECT. These are not simple summaries; they are data-driven trajectories.

---

⚙️ The Underlying System

Keenable SELECT operates as an MCP server featuring a primary tool called select.

Core Architecture

• DuckDB Integration: Executes read-only SELECT statements on live web data.
• Hybrid Execution: Semantic operators run outside DuckDB; their results are injected back into the row set for final SQL processing.
• Efficiency: Can scan >1,000 pages and filter them via WHERE clauses without LLM overhead.
• State Management: Every result set is assigned a unique id, allowing subsequent queries to reference previous data.
• Reporting: A dedicated server model converts these rows into shareable HTML reports.

Semantic Operators Reference

Note: Operators support dynamic arguments, such as WEB_SEARCH(name || ' founding year').

---

🤖 The Dual-Agent Framework

Every report is the result of a collaboration between two distinct agents:

1. The Research Agent

This is a standard tool-loop LLM equipped with the select tool.

• Behavior: It iteratively writes and executes queries until the research goal is met.
• Transparency: It streams all tool calls and results as real-time events.
• Continuity: Follow-up questions are handled by referencing the stored conversation transcript.

2. The Report Agent

Once data is gathered, this agent handles the presentation via the generate_html_report function.

• Environment: Operates in a sandboxed Python session.
• Data Handling: It accesses result sets as dataframes, ensuring data integrity by avoiding manual retyping by the LLM.
• Quality Control:
  1. Renders a draft.
  2. Analyzes screenshots and JavaScript error counts.
  3. Iteratively fixes the document within a set budget before final publication.