Whistle Synth
Whistle Synth: A New Mac App
Jeff Kaufman has released a unique program that allows users to control a synthesizer via whistling. The application is currently available on the Mac App Store, where it is provided as an open-source tool with no advertisements and zero cost.
🛠️ Hardware Requirements
To get the best experience, you will need:
- An external microphone (Highly recommended).
- Wired headphones (Essential to minimize audio lag).
🚀 Technical Evolution via AI
Kaufman leveraged Claude to overhaul the app's implementation, resulting in a more stable, lower-latency experience with a wider variety of instrument voices.
1. Reducing Latency
The latency reduction was achieved through a clever feedback loop: Kaufman pointed the microphone directly into the headphones, allowing Claude to measure the round-trip time. By implementing the callback API and other optimizations, the performance improved drastically:
| Metric | Old Implementation | New Implementation |
|---|---|---|
| Round Trip Latency | 80ms | 5ms |
2. Eliminating Stutters
To stop the audio from stuttering, the program's architecture was refactored. The logic was split into two distinct components: a detector and a synthesizer.
Interestingly, this structure mirrors a design Kaufman had created five years prior but had abandoned for a more complex, yet limiting, approach.
🎹 Sound Design & Iteration
The development of the "voices" was a collaborative effort between the author and the AI.
The Bass Voices: Claude prototyped various bass sounds. One standout is the octaveless bass. In this mode, as the pitch increases, the harmonic mix shifts to ensure the perceived octave remains constant.
The Treble Struggle: While bass sounds were easier to perfect, treble voices proved difficult.
- Successes: One usable treble voice.
- Failures:
Physical modeling of a flute(Never sounded right).Tine piano / Rhodes(Unsuccessful).Various synth leads(Unsuccessful).
"I think a lot of [the drawbar's success] is that the rotating speaker (Leslie) adds a lot of life that would normally be missing for a sustained note."
Kaufman also experimented with an accordion sound, which is "decent" but not yet perfect.
The "Fifth Down" Feature
Inspired by Ugo Conti, a setting was added that plays a fifth below the whistled note. If you whistle a , the synth plays a . Mathematically, this means the whistle provides a multiple of the third harmonic: This allows users to play in keys that might otherwise fall outside their natural whistling range.
🍎 The App Store Journey
Distributing via the Mac App Store was a first for Kaufman, primarily due to Apple's strict review process. He used a combination of AI tools to navigate this:
- Icon Design: Gemini created an initial version (too busy).
- Icon Scaling: Claude attempted to create scalable icons.
- Developer Account: Paid the membership fee and verified identity.
- Support: Fixed a typo where Apple read his name as
"Kaufhan". - Marketing: Created screenshots and a demo video with Claude's help.
Icon Degradation Process:
- High Res: Full detail.
- 128x128: The wave pattern disappears.
- 32x32: Becomes a "garish green whistle" impression.
💭 Final Reflections
Kaufman views this project as a prime example of AI acting as a complement to human creativity.
"I was never going to get around to doing any of this if I needed to write it all myself."
By collaborating with AI, he produced a tool that is more versatile and sounds better than his original hand-coded version. He aims to utilize this current era of AI—where it is helpful but doesn't entirely replace the human element.
📎 Side Notes & Recent Feed
- Biorisk: Personal efforts to reduce biorisk.
- AI Skepticism: Analysis of Ed Zitron's prediction track record.
- Conference: The Overhang by Sparrow Infrastructure (Sept 19-20 in D.C.).
- Whimsical Thought: A snippet from Nora Wise's blog about a child wanting a sibling to play with because their current sibling is in preschool.