Today, we are launching Soniox Compare, a set of open-source tools for fair and transparent voice AI evaluation.
It includes:
- Compare Speech-to-Text
- Compare Text-to-Speech
- Compare Speech Translation
The goal is simple: to make voice AI evaluation more open, fair, and useful.
Instead of relying only on curated datasets, cherry-picked demos, or benchmark charts reduced to a single score, Soniox Compare lets you test real provider APIs side by side using your own audio, text, languages, and use case.
You can see the actual outputs, compare publicly available pricing, and inspect every provider integration and configuration.
The complete source code is available under the MIT License, making every part of the evaluation transparent, verifiable, and open to scrutiny.
Benchmarks do not answer the question that matters
Voice AI benchmarks reduce a complex production system to a single number. That number may be useful, but it rarely tells you how a provider will perform inside your application.
A model can perform well on a clean English benchmark and still fail on the accents, background noise, language switching, names, numbers, and streaming conditions your users encounter in production.
The same problem applies to text-to-speech. A voice may sound impressive in a polished demo but mispronounce customer names, scramble an account number, change the words provided, or break when multiple languages appear in the same sentence.
Speech translation is even harder to evaluate. Translation quality matters, but so do streaming behavior, language switching, speaker separation, source transcripts, one-way versus two-way conversations, latency, language coverage, and total cost.
No universal benchmark captures all of this. The evaluation that matters most is the one performed on your own data.
Compare the complete voice AI stack
Soniox Compare gives you three direct ways to test voice AI providers.
Compare Speech-to-Text
Stream the same audio to multiple speech-to-text providers simultaneously and watch their transcripts appear side by side in real time.
Test the capabilities that determine whether speech recognition actually works in production, including accuracy on real-world speech, streaming latency, language switching, speaker diarization, endpoint detection, custom terminology, alphanumeric precision, and multilingual performance.
Compare Soniox with OpenAI, Google, Azure, Deepgram, Speechmatics, AssemblyAI, ElevenLabs, Cartesia, and other providers.
Try it: https://soniox.com/compare-stt
Compare Text-to-Speech
Send the same text to multiple text-to-speech providers and listen to the results side by side.
Compare naturalness, latency, pronunciation, and whether each model reliably preserves names, numbers, addresses, technical terminology, mixed-language text, and the exact words provided.
Every playback makes a live API request. Every playback makes a live API request, so you hear the actual output returned by each provider.
Try it: https://soniox.com/compare-tts
Compare Speech Translation
Speak once and compare how different providers translate the same audio in real time.
Test translation quality, streaming responsiveness, language switching, speaker separation, source transcripts, language coverage, and one-way or two-way conversation modes.
Speech translation systems are architected very differently. Some translate only into a single fixed target language. Some require separate models for transcription and translation. Some do not preserve speakers. Others return only final translations instead of streaming the result while the person is speaking.
Now you can experience those differences directly.
Try it: https://soniox.com/compare-translation
Compare performance and cost
Technical performance determines whether an API works. Economics determine whether it can scale.
That is why every Soniox Compare page also includes a pricing comparison based on publicly available API prices.
Choose your expected monthly volume and compare the estimated all-in cost across providers.
Headline prices can be misleading. Providers may charge separately for real-time processing, premium models or voices, translation, diarization, multilingual support, source transcripts, and other production features.
A service that looks competitive at first can become significantly more expensive once it is configured for your actual use case.
The calculators use publicly available pay-as-you-go rates and are regularly reviewed and updated. Enterprise and committed-use pricing may differ, but the comparison provides a clear starting point for understanding production cost.
The complete project is open source
We have released the complete Soniox Compare source code under the MIT License.
You can inspect every provider integration, see which models and configuration parameters are used, and verify how audio, text, and results are handled.
You can also run the tools locally with your own API keys, change any configuration, add another provider, or build your own internal evaluation system on top of the source code.
Soniox Compare does not rewrite, correct, normalize, or clean up provider outputs. What the provider API returns is what you see.
Every part of the evaluation is transparent, verifiable, and open to scrutiny.
GitHub: https://github.com/soniox/soniox-compare
Test what matters to your application
Choosing a voice AI provider should not depend on a single benchmark, a curated demo, or claims made by the provider. You should be able to test every system directly on the data that matters to your product.
Use your hardest audio, the languages your customers actually speak, and the names, numbers, codes, and terminology your application encounters every day. Test the edge cases that can break the experience, then compare what each system will actually cost to run in production.
We built Soniox Compare to make voice AI evaluation practical, transparent, and grounded in real-world performance.
Soniox Compare is live. Test it on the data that matters to your application.