SCM: Screen Memories

Deep AI-Powered Visual & Auditory Retrieval for macOS

SCM is a sophisticated local search engine designed specifically for macOS. It allows users to index every single photograph and every individual frame of video within any designated folder, turning a chaotic library into a searchable database of memories.

The Core Philosophy: Search should mirror human thought. Instead of remembering a filename, you describe a memory in plain English, and a local vision model handles the retrieval.

---

🚀 What Sets SCM Apart?

Unlike standard file searches, SCM dives deep into the content of your media:

• Granular Video Search: It doesn't just find the video file; it segments scenes and embeds them, meaning you land on the exact shot not just the file.
• Multimodal Text Extraction: It utilizes OCR for visible text and OpenAI's Whisper for spoken dialogue, treating each as a distinct search mode.
• Persistent Workspaces: Users can save specific queries as tabs. Additionally, specialized tabs for Email and Screenshots can be toggled on or off.
• Autonomous Maintenance: The system features watched folders for automatic imports, uses content hashing to prevent duplicates during renames, and re-embeds data in the background when models are updated.

Search Architecture Overview

---

🔍 Five Dimensions of Search

Technical Deep Dives

1. File Retrieval

The system begins with a keyword pre-pass on filenames before the vision model takes over. Results are calculated using cosine similarity between image embeddings: $\text{similarity} = \cos(\theta) = \frac{\mathbf{A} \cdot \mathbf{B}}{\|\mathbf{A}\| \|\mathbf{B}\|}$ To ensure quality, SCM employs an honesty floor (calibrated per model) and a diversity filter to remove near-duplicates. For CJK (Chinese, Japanese, Korean) languages, it utilizes overlapping bigrams (e.g., "台北車站" matches both 台北 and 車站).

2. Scene Detection

To prevent "grid flooding" with irrelevant data, a noise gate is applied. Each video is limited to a maximum of 3 representative scenes.

3. OCR & Dialogue

• OCR: Operates independently of the AI engine, allowing for literal text matching even while offline.
• Dialogue: Uses three tiers of retrieval:
  1. Exact line: A contiguous phrase.
  2. Exact words: All words within one utterance or an $\le 8\text{s}$ window.
  3. Words spoken: Any occurrence within the same video.

4. The "Ask" Feature (LLM)

This is an opt-in feature using a llama.cpp sidecar. It doesn't hallucinate from the web; it answers based only on the evidence already extracted (OCR, dialogue, filenames).

• Commands: Use /screenshots, /videos, or /email to filter the corpus.
• Control: Use Stop to halt generation.

Default Model:

• Qwen3 1.7B: $\approx 1.1\text{GB}$ (Optimized for 8GB RAM Macs).

---

🛠 Installation & Requirements

Prerequisites

• OS: macOS (Electron-builder package; tray features are macOS-exclusive).
• Runtime: Bun (used as the package manager and runner).
• Dependencies: bun install to fetch Node modules.
• Storage: Initial model download (CLIP) is $\approx 435\text{MB}$; thereafter, it is fully offline.

Setup Steps

• Option A: Full Trust (Homebrew)

  brew tap allenv0/scm
  brew trust allenv0/scm
  brew install --cask allenv0/scm/scm
  

• Option B: Least Privilege (Cask Only)

  brew tap allenv0/scm
  brew trust --cask allenv0/scm/scm
  brew install --cask scm
  

• Updating

  brew upgrade --cask allenv0/scm/scm
  

---

💻 Development Guide

For those looking to build from source, use the following bun commands:

bun run dev $\rightarrow$ Builds renderer $\rightarrow$ Launches Electron. bun start $\rightarrow$ Launches Electron without rebuilding. bun run build $\rightarrow$ Compiles React renderer into dist/.

Packaging the App

To create a DMG or ZIP installer:

# Signed version (requires keychain identity)
bun run dist 

# Unsigned version
bun run dist:unsigned

Build Process Flow:

1. vite build $\rightarrow$ Compiles src/ to dist/.
2. Bundles dist/ with main.js, preload.js, main-lib/, and indexer/.
3. Outputs installers to dist-app/ (e.g., SCM-0.2.4.dmg).