Posted the SpeechAnalyzer vs Whisper benchmark here recently and a few of you asked about Parakeet V2/V3. Ran both through the same harness, plus MOSS-Transcribe-Diarize which someone on HN suggested. Same 5,559 LibriSpeech utterances, same normalizer and scorer as round 1.
Results (WER, clean / other):
- Parakeet TDT v2 int8: 2.01 / 3.40 (wins)
- MOSS: 2.07 / 4.68
- SpeechAnalyzer: 2.12 / 4.56
- Parakeet TDT v3 int8: 2.51 / 4.28
- Whisper Small: 3.74 / 7.95
The stuff that actually matters if you're picking an engine:
Integration was easy. Parakeet ran via FluidAudio (Swift package, CoreML, runs on the ANE). We already link it for diarization so adding it to the benchmark was maybe 50 lines. If you have an AVAudioPCMBuffer pipeline already it's an afternoon.
Budget for the quantization tax though. NVIDIA's published 1.69/3.19 is bf16 on GPU. The int8 CoreML port you'd ship scores 2.01/3.40. Still the most accurate thing you can run on-device, but a decent chunk of its paper lead over Apple's free engine evaporates once it's actually shippable.
v3 gives you 25 languages for a real English accuracy cost (2.51 vs 2.01 clean). Still beats Whisper Small everywhere though.
MOSS does transcription + diarization in one pass and the speaker labels were perfect in a small controlled test. But there's no Swift or CoreML path (Python/MLX only), it's 1.7GB with ~2.7GB RAM, and it can't stream. Not output-streaming, I mean it literally cannot start transcribing until it has the complete recording, so anything with live captions needs a second engine anyway. Also fun: audio where speech starts at exactly t=0 returns an empty transcript, deterministically. Prepend a second of silence and it's fine.
This benchmark got Whisper removed from our app btw. Parakeet v3 replaced it as the downloadable engine (beats Whisper Small everywhere at a similar size, 25 languages), and for English Auto still prefers SpeechAnalyzer: 0.11 points, zero bundle bytes, improves with the OS. The cost-benefit section of the article is probably the useful part if you're making this call yourself.
Article + raw per-utterance transcripts if you want to rescore: https://get-inscribe.com/blog/parakeet-moss-apple-speech-benchmark.html
Disclosure: I build Inscribe (ships the Apple engines + Parakeet, formerly Whisper), all engines ran through identical production code paths.