The layers around a local speech model

Sonari transcribes your voice with a model running on your Mac. Parakeet through FluidAudio, or WhisperKit if you want a lighter download. Both are genuinely good, and I’m grateful to the people who made them run on the Neural Engine so I didn’t have to.

Neither is enough on its own though, and I want to talk about why, because honestly the interesting part of building this app was never the model.

A raw local model hands you back two problems. It mangles the same handful of words every time, usually names and technical terms it has never heard. And it gives you exactly what you said, in the shape you said it, which is almost never the shape you want to send.

So the model is one layer.

There are three more, and each of them does a job the model would do worse.

The dictionary does the part that isn’t a guess

The first layer runs before any AI touches the text. It’s a plain dictionary: a list of find-and-replace pairs.

I say “Claude Code” out loud a dozen times a day, and the model hears “Clawed Code” almost every time.

That isn’t a hard problem. It’s a lookup. I know the wrong output and I know the right one, so I don’t need anything clever, I need a table. The dictionary rewrites “Clawed Code” to “Claude Code” deterministically, whether or not Apple Intelligence is even available on your Mac.

I wrote about this at more length in an earlier post, so the short version: “always turn X into Y” is the wrong job to hand a probabilistic model. It’s a job for a lookup table, and a lookup table always gets it right.

The transforms do the part the model is actually good at

Once the text is transcribed and corrected, you usually want it reshaped. The same content, in the form you’d actually send. Sonari calls these transforms, and they’re named output styles: Clean up, Email, Message, List. They run on Apple’s on-device foundation model, so the text never leaves the machine.

The trick is giving that model a small, bounded job. Reshape the text I already handed you. Don’t know things, don’t fill anything in, just take these words and make them read like an email. Scoped that tight, it’s pretty great.

I was super happy the first time it turned a rambling voice memo into a clean three-line message and got out of the way.

And when the model isn’t there, Apple Intelligence off or the chip too old, Sonari inserts the transcript unchanged. The dictation still works. The AI sits on top of something that already does its job without it.

The app you’re in is a fact, so read it

Here’s the layer I’m most happy with, and the newest one.

A transform is only right some of the time. The email cleanup is perfect for Mail and wrong for Slack, where I want the casual one. For a while I assumed the answer was to have the model guess the tone from context.

Then it occurred to me that I already have the context. I don’t need to infer which app you’re writing into.

I can ask the OS.

So Sonari grabs the frontmost app the moment you start recording, and you map a transform to it. Slack gets Message. Mail gets Email. Your editor gets Clean up, or nothing at all. I’m calling them app-aware transforms: you pick a running app from a list, choose a style, and from then on dictating into that app uses it without you thinking about it.

There’s no model anywhere in that decision. The frontmost app is a bundle identifier I read at record-start and match against the table you built.

It matches, or it falls back to your default. The one thing it leans on is you setting the mappings up once, a little upfront fiddling that I decided beat trying to be clever about guessing. That’s a tradeoff I’m comfortable with, and honestly not one I lose sleep over.

Where I put the intelligence

Step back from the four layers and the design is really one idea. Each job sits at the layer that can actually do it.

A word the model always gets wrong is a lookup, so it’s a table. Which app you’re in is a fact, so it’s a read. Tone is genuinely fuzzy, so that one goes to the model, scoped down to the single thing it does well and made optional in case it isn’t there. The model does the one job I can’t do without it, and everything around it is boring, deterministic code I can reason about at 2am.

Most of what makes a local speech model feel good instead of frustrating lives in those boring layers. The model gets all the attention. The table and the OS read are doing a surprising amount of the work.

Sonari is a local dictation app for the Mac, and I’m sending early access to the list at sonari.audio.