The Pipecat STT benchmark is an open-source evaluation of leading speech-to-text models for real-time voice agents. It measures what matters most in production voice AI: transcription accuracy and latency. The benchmark is public, reproducible, and the results table is the single source of truth for every number published on this page.
- Accuracy: Soniox
stt-rt-v6 has the lowest semantic WER in the benchmark: 0.74% mean WER, 0.71% pooled WER, and 88.8% perfect transcripts. - Latency:
stt-rt-v6 is 266ms median time to final segment (324ms P95, 343ms P99). - Dataset: 1,000 real-world samples from the
pipecat-ai/smart-turn-data-v3.1-train dataset, with ground truth generated by Gemini and human-reviewed. - Mode: Real-time streaming transcription, the setting that defines voice agent performance.
Results
Sorted by semantic WER (lower is better). Latency is reported as time to final segment in milliseconds. Price is the public pay-as-you-go real-time rate per hour of audio, shown where the benchmarked model maps to a listed price. Among the most accurate services, WER differences under about 0.25 percentage points are within the benchmark's 95% confidence interval.
| Provider | Model | Price / hr | WER mean | Pooled WER | Perfect | TTFS median | TTFS P95 | TTFS P99 |
|---|
| Soniox | stt-rt-v6
| $0.12 | 0.74% | 0.71% | 88.8% | 266ms | 324ms | 343ms |
| Meta | muse-voice-transcribe-1.0
| $0.18 | 0.80% | 0.73% | 89.5% | 392ms | 1292ms | 1922ms |
| AssemblyAI | universal-3-6-pro
| — | 0.89% | 0.80% | 88.7% | 307ms | 401ms | 498ms |
| Azure | — | $1.60 | 0.98% | 1.01% | 83.5% | 1016ms | 1345ms | 1791ms |
| Speechmatics | linden-1
| — | 1.11% | 0.96% | 83.9% | 369ms | 438ms | 690ms |
| AssemblyAI | universal-3-5-pro
| $0.62 | 1.12% | 0.95% | 86.0% | 282ms | 354ms | 393ms |
| xAI | grok-voice-transcribe-2.0
| $0.20 | 1.12% | 0.79% | 89.4% | 1322ms | 1895ms | 2148ms |
| Cartesia | ink-2
| $0.43 | 1.14% | 1.02% | 83.6% | 299ms | 328ms | 1584ms |
| Speechmatics | — | $0.54 | 1.17% | 0.96% | 83.1% | 495ms | 676ms | 736ms |
| AWS | — | — | 1.26% | 1.35% | 79.1% | 1136ms | 1527ms | 1897ms |
| Deepgram | nova-3-general
| $0.75 | 1.32% | 1.37% | 77.7% | 247ms | 298ms | 326ms |
| Smallest AI | pulse
| — | 1.85% | 1.94% | 74.0% | 398ms | 533ms | 1593ms |
| NVIDIA | Nemotron 3.0 ASR (en)
| — | 1.90%† | 1.95%† | 76.1% | 221ms | 238ms | 252ms |
| Google | gemini-3.5-transcribe-live
| $0.54 | 1.93% | 1.97% | 78.3% | 458ms | 532ms | 599ms |
| OpenAI | gpt-realtime-whisper
| $1.02 | 2.41% | 2.30% | 73.3% | 740ms | 878ms | 1080ms |
| Google | latest-long
| — | 2.46% | 2.46% | 69.7% | 878ms | 1155ms | 1570ms |
| AssemblyAI | universal-streaming-english
| — | 2.81% | 2.42% | 66.8% | 256ms | 362ms | 417ms |
| ElevenLabs | scribe_v2_realtime
| $0.39 | 3.05% | 3.13% | 80.3% | 281ms | 348ms | 407ms |
| OpenAI | gpt-4o-transcribe
| — | 3.10% | 2.95% | 75.7% | 637ms | 965ms | 1655ms |
| Cartesia | ink-whisper
| — | 3.32% | 3.78% | 61.4% | 266ms | 364ms | 898ms |
| Gradium | default
| — | 3.81% | 3.79% | 65.6% | 585ms | 614ms | 623ms |
| Mistral | voxtral-mini-transcribe-realtime-2602
| — | 4.49% | 5.11% | 67.7% | 525ms | 973ms | 1913ms |
| NVIDIA | Nemotron 3.5 ASR (multilingual)
| — | 4.54%† | 4.58%† | 62.0% | 236ms | 253ms | 266ms |
† Scored by the previous judge (Claude Sonnet 4.5), which reports WER about 15% higher on average than the current judge (Claude Sonnet 5.5). Pipecat plans to re-score these rows.
Pricing reflects public pay-as-you-go rates and may not match every benchmarked configuration. See Soniox pricing for details.