Back to Soniox

What is audio intelligence?

Information derived from transcripts and acoustic signals

Updated June 29, 2026

A one-hour support call may be represented by a transcript, summary, sentiment labels, action items, redacted sensitive information, and detected escalation events. The latter outputs are derived from the transcript, the acoustic signal, or both. Audio intelligence comprises the methods used to produce these higher-level representations.

Textual and acoustic information

Some of what you want lives in the words. Once you have the transcript, plus timestamps, speaker labels, and confidence, you can summarize the call, pull out the action item, judge sentiment from word choice, and scrub the credit-card number. These are text problems applied to a transcript.

The rest never made it into the words. The caller crying, the long sigh before "fine," the sarcasm the prosody gives away, the smoke alarm in the background, none of that is in the transcript, and a baby crying or a gunshot was never a word. To recover any of it, you have to go back to the signal itself.

Sentiment is the clean example of why you often want both. On the words alone, "fine, this is exactly what I needed" reads as a happy customer. Let the model hear the flat, clipped delivery and it catches the sarcasm the text misses.

Processing after transcription

Everything word-derived inherits the transcript's mistakes silently. A summary built on a transcript that misheard the dosage summarizes the wrong dosage with full confidence; sentiment built on a hallucinated line reads emotion into words nobody said. The accuracy ceiling of these tasks is the accuracy of the transcript beneath them and not one point higher, one more reason the entity-level accuracy of recognition matters more than its average.

Audio intelligence tasks

Each branch gets its own page. Conversation summarization turns a long call or meeting into notes and action items. Speech sentiment analysis reads emotion, the cleanest example of the text-versus-acoustic split. PII redaction finds and removes sensitive information for compliance. Keyword spotting and wake words listen for specific trigger phrases, the always-listening case. Audio event detection recognizes non-speech sounds, the part with no transcript at all. Speaker diarization and language identification, covered elsewhere, are themselves forms of intelligence about audio.

Practical applications

Audio intelligence is where transcription turns into business outcomes, so the demand concentrates in a few places. Call centers use it for quality assurance, compliance, and coaching, scoring and summarizing calls nobody could review by hand. Meetings become notes, decisions, and action items. Media uses it for search, indexing, and moderation across archives too large to watch. Healthcare turns dictation into structured notes. Each wants the same thing: the meaning behind the words and the sound around them, extracted at a scale humans cannot match.

Much text-derived intelligence now runs by feeding transcripts to large language models, which makes summaries, extraction, and classification fast to build. That convenience does not escape the rule above. A language model reading a flawed transcript does not hesitate or hedge; it produces a fluent, confident summary of words that were never spoken, and the wrong summary reads as convincingly as the right one. The layer on top can only be as accurate as the transcript it was handed.

Common questions

What is audio intelligence?

Everything you extract from a call after the words: the three-line summary, the action item, the moment the customer got angry, the credit-card number scrubbed before storage. If transcription gives you the floor, this is the part people actually read.

How is audio intelligence different from speech-to-text?

Speech-to-text is the input; audio intelligence is what you do with it. The transcript answers "what words," then summaries, sentiment, redaction, and event detection turn those words and the sound around them into something you can act on. Without the transcription layer there is nothing to build on.

Why does some audio intelligence use the audio and not just the transcript?

Because the transcript threw the rest away on purpose. It cannot tell you the caller was crying, that two people talked over each other, or that a smoke alarm went off; a gunshot or applause was never a word. Anything about how it was said, or any non-speech sound, has to be read from the signal directly.

Does audio intelligence depend on transcript accuracy?

For the word-derived tasks, entirely. A summary on a transcript that misheard the dosage summarizes the wrong dosage, and a language model states it with full confidence. The names and numbers that carry the meaning are exactly the ones recognition is most likely to miss, so entity-level accuracy matters more than the average.

References

  1. Kong, Q., Cao, Y., Iqbal, T., Wang, Y., Wang, W., & Plumbley, M. D. (2019). PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition. arXiv preprint arXiv:1912.10211.
  2. Zhong, M., Yin, D., Yu, T., Zaidi, A., Mutuma, M., Jha, R., et al. (2021). QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization. arXiv preprint arXiv:2104.05938.