Moonshine Micro: Compact Voice Interfaces for Embedded Systems

Moonshine Voice is an open-source AI framework designed for developers creating real-time voice-driven applications and agents. Its specialized variant, moonshine/micro, is engineered specifically for Digital Signal Processors (DSPs) and microcontrollers.

The project utilizes the Raspberry Pi RP2350 as its primary reference platform—a chip that is remarkably affordable, retailing for approximately 80 cents.

Core Capabilities

Moonshine Micro integrates three primary voice functions into a tiny footprint, capable of operating in as little as $470\text{ KB}$ of RAM:

VAD: Voice-Activity Detection.
Command Recognition: Speech-to-Text (STT).
Neural Speech Synthesis: Text-to-Speech (TTS).

System Workflow

The following diagram illustrates the sequential processing pipeline used in the demo:

Resource Budget & Performance

The system is optimized for extreme resource constraints. The following data represents the requirements for the RP2350 demo:

Component	Flash Memory	SRAM (Peak Arena)	Compute Load
VAD	$\approx 89\text{ KiB}$	$\approx 36\text{ KiB}$	$\approx 0.8\text{ MMAC/frame}$ ( $\approx 25\text{ MMAC/s}$ )
STT (SpellingCNN)	$\approx 1.3\text{ MiB}$	$\approx 346\text{ KiB}$	$\approx 36\text{ MMAC/s}$
TTS (Neural Diphone)	$\approx 1.8\text{ MiB}$	$\approx 340\text{ KiB}$	$\approx 37\text{ MMAC typical}$ ( $\approx 65\text{ MMAC/s out}$ )
Total Pipeline	$\approx 3.6\text{ MiB}$	$\approx 468\text{ KiB}$	Latency: $0.7\text{--}1.0\text{ s}$ (Classify + Speak)

Technical Note on Memory: The SRAM figure is ~~not additive~~. Because the VAD, STT, and neural TTS modules run sequentially, they share a single TensorFlow Lite Micro (TFLM) arena of $\approx 384\text{ KiB}$ . The total provisioned RAM is $\approx 468\text{ KiB}$ out of the $520\text{ KiB}$ available on the RP2350 (with $\approx 491\text{ KiB}$ used by wifi hardware).

Definitions:

MAC: Multiply-Accumulate operation.
MMAC/s: Millions of multiply-accumulates per second during active processing.

Implementation Details

The libraries for VAD, STT, and TTS are modular and can be used independently. All neural computations are powered by the TensorFlow Lite Micro library.

Repository Structure

The project is organized as follows:

moonshine/micro/
├── examples/           # RP2350 implementation demos
├── g2p/                # Grapheme-to-Phoneme logic
├── klatt-tts/          # TTS implementation
├── models/             # SpellingCNN and TinyVadCNN models
├── stt/                # Speech-to-Text core
├── vad/                # Voice Activity Detection core
└── third-party/        # External dependencies

Integration Checklist

Configure CMakeLists.txt
Import pico_sdk_import.cmake
Set up WiFi connection via the provided voice example
Deploy to RP2350 hardware

Licensing and Legal

The software is distributed under a permissive license to encourage commercial adoption.

Core Source Code: Licensed under the MIT License.
Models: The SpellingCNN and TinyVadCNN found in the models/ directory are also MIT Licensed.
External Code: Anything located in third-party/ is subject to the specific licenses of its original project (refer to the LICENSE file within each respective subfolder).

Documentation Links

For further details, refer to the following sections:

Voice Activity Detection
Speech to Text
Custom Word Recognition
Neural Text to Speech
Wifi Setup Example