Apple Intelligence as a local rewrite engine

Sonari runs its AI transforms on Apple’s on-device model. You dictate, the raw transcript comes back, and it gets reshaped into one of a few named styles: Clean up, Email, Message, List. All of it happens on the Mac. Nothing gets uploaded, ever.

It’s been working great. I mean that plainly, and it surprised me, because I went in pretty skeptical of a model small enough to run on a laptop while you’re doing everything else.

So this is the honest version of why it works. And the reason it works turns out to be the same thing that tells you where it doesn’t.

What it’s actually good at

The model is small. Roughly 3 billion parameters, running entirely on the machine. That’s the constraint everything else follows from, and once you stop fighting it, the job it’s good at gets obvious. Hand it text you already gave it and ask it to reshape that text. Fix the punctuation. Drop the “ums” and the false starts. Turn a paragraph you rambled into three clean bullets. Make the same dictation read like an email in Mail and like a Slack message in Slack.

Give it a bounded job like that and it’s genuinely good. Fast, on-device, and good enough that I mostly stopped reaching for a cloud model to do the same thing. For dictation cleanup that covers most of what I need, so this handles a lot of ground.

Honestly, that was the surprise. I expected to want more model.

Where it falls short

Here’s the part I had to actually think about, because in daily use it just works. Every place it falls short is a place where I asked it to do more than reshape what I said.

It won’t add what isn’t there. Ask it to clean up a transcript and it’s great. Ask it to “fix the details” or fill in something you didn’t actually say, and it either flattens the text into something generic or it makes something up. That’s correct behavior for a rewrite engine. It’s still a limit, because people expect a thing labeled “AI” to know things, and this one doesn’t. It reshapes. It doesn’t reason and it doesn’t look anything up.

It’s only as good as the transcript it gets. If the speech model hears “Prism” as “prison,” the rewrite confidently keeps “prison” and makes the sentence around it read beautifully. A better rewrite makes a wrong transcript worse, because now the mistake is well-dressed. This is exactly why Sonari runs a deterministic custom dictionary before the model ever sees the text. The words a speech model mangles the same way every time get fixed by a plain find-and-replace. I don’t want a model deciding whether to “help” with those. I want them right every time.

It only runs where Apple Intelligence does. That needs the right hardware and it needs to be switched on. On a Mac without it, there’s no rewrite step at all. I have to treat that as a real product constraint, because it means the feature can’t be something the app leans on to be useful.

What I do about it

None of this made me want to rip the model out. It made me want to give it a smaller job and build guardrails around the edges.

Three things fall out of that:

The dictionary runs first, and it’s deterministic. Anything I need to be exactly right, every time, does not go through a model. A model is the wrong tool for “always turn X into Y.”

The AI transform is a bounded task with a named style. “Make this an email” has a shape the model can hit. An open-ended “do the right thing with this” gives it nothing to aim at.

And there’s always a no-AI path. If Apple Intelligence isn’t available, or the model refuses, or it’s just slow that second, you get your transcript inserted unchanged. Cleaned up by the dictionary, but never held hostage to a model being ready.

That last one is the part I’d tell anyone building on a local model first. The model is going to be unavailable sometimes. Design like it, and every day it does work feels like a bonus.

I’m still poking at where the edges actually are, and honestly I’m pretty curious how far the small model goes before it stops being enough. I’ll write more as I find out. If you’ve pushed a small local model somewhere it broke in an interesting way, I want to hear about it.

Sonari isn’t ready to buy yet, I’m building the list first. If you’d rather your dictation stayed on your Mac, drop your email at sonari.audio and I’ll tell you the moment it’s ready.

TJ