logo
Kurashizu Blogwhere ideas flow
  • Home
  • News
  • Blog
  • About
  • Home
  • News
  • Blog
  • About

© 2026 Kurashizu. All rights reserved.

Admin·New Post·Service Status
Powered by Cloudflare·31.10.26
← Back to news

Agent memory as a file format

Original article
calpaterson.com|126 points|60 comments|by ingve|Aug 31, 2026

Agent Memory as a File Format

August 2026

[Sound of a floppy disk being inserted] "WOW—I actually know the corporate VLAN configuration!"

Most benchmarks for AI models begin with a completely empty context window. While this is done to ensure a "level playing field" for testing, it is fundamentally unrealistic. In the real world, an agent should never start from zero; it should begin its task with every piece of relevant information already at its fingertips.

Why Current Memory Architectures Fail

Most existing attempts at agent memory are, frankly, adequate rubbish. We can generally categorize these failing systems into three buckets:

System TypePrimary ApproachThe Fatal Flaw
The Platform Lock-inProprietary harnesses built by the LLM provider.Designed to move users from "API" to "Platform" business models; focuses on user-centric data rather than world-centric utility.
The Over-EngineeredComplex stacks (e.g., pgvector + Neo4j + a dedicated "memory" LLM).Too complex to manage and actually confuses the model; fails to evolve as the model frontier advances.
The High ModernistRationalist graphs and logical propositions.Strips context away, leaving the agent with "distilled facts" that are isolated and meaningless.

The common thread here is that these systems treat memory as a process (a pipeline) rather than data.

"Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious." — Brooks

Introducing: The memoryfield Format

Instead of a pipeline, memory should be a portable data format. I propose the .memoryfield format, which is essentially a structured archive:

my-memories.memoryfield.zip
├── carbon-fibre-woks.md
├── finnish-bureaucracy-tips.md
├── [... many more md files ...]
├── wec-2026-season-notes.md
└── nomic-embed-text-v1.5.sqlite3

What exactly is a memoryfield?

  • Markdown "pages" for human and AI readability.
  • YAML frontmatter for structured metadata.
  • SQLite vector index (optional) to enable efficient semantic search.

Design Decision 1: Prioritize Prose over "Chunks"

RAG (Retrieval-Augmented Generation) is often nightmarishly complex because it tries to force-feed an AI documents written by humans (like massive, messy PDFs) that weren't designed for machine consumption.

However, when an agent creates a memory, it is already an AI. It can write in the format it likes best: Markdown. There is no need to chunk, summarize, or mechanically process the text. The agent simply writes the prose directly.

Example of a memoryfield page:

---
title: Carbon Fibre Woks
created: '2026-03-01T09:00:00Z'
updated: '2026-08-22T14:30:00Z'
uuid: 6aa615f0-486f-48a7-a210-ba4f5ff18c8b
summary: Thermal properties of carbon fibre cookware
---
Carbon fibre woks conduct heat evenly, but...

Note: To ensure the page fits within a vector embedding, there is a soft limit of ≈8kb\approx 8\text{kb}≈8kb (roughly 2,0002,0002,000 tokens). This is actually a feature, not a bug. At ≈1,300\approx 1,300≈1,300 words, it's the length of a standard magazine article. If more detail is needed, the agent simply creates more pages.


Design Decision 2: Semantic Jumps vs. Graph Walking

Some previous attempts, such as Karpathy wikis, relied on hyperlinked Markdown files (similar to Obsidian). The theory was that the agent would "walk" the knowledge graph to find information.

A complex, beautiful knowledge graph that looks impressive but is useless to an LLM

In reality, agents hate traversing graphs. It is slow, unreliable, and confusing.

The "Graph Walk" Bottleneck

The logic for walking a graph looks like this:

If the required information is NNN steps deep, the agent must perform N+1N+1N+1 serial tool calls. Because each call takes several seconds, the latency is brutal.

Mathematically, the cost of retrieval is: Latency=(N+1)×Tool Call Time\text{Latency} = (N + 1) \times \text{Tool Call Time}Latency=(N+1)×Tool Call Time

Furthermore, this forces the agent to engage in 1990s-style SEO hacking. It must ensure every page title and link caption is "snappy" and "accurate" so that it knows which link to click. This kills the ability to record implicit lore or ambient side-details—the very things that make a large corpus useful. If a page isn't titled perfectly, the agent simply misses the information.