Introducing Saaras Multispeaker: Built for Real-Time Conversations
Saaras-multispk is our flagship multispeaker ASR model which can transcribe overlapped speech seamlessly across multiple speakers. Lowest diarization error and lowest cpWER on Indic DiarBench
Single-speaker speech recognition is largely a solved problem. A room is not.
The audio this model targets is the one where multiple people are speaking at the same time and on top of each other’s speech, with a lot of interjections, back-channels, etc. For example a news panel where nobody waits their turn. For those recordings, a transcript that doesn’t say who spoke when is close to useless.
Saaras-multispk is built for exactly that audio, and it is live in the Sarvam API and playground today.
A true multi-speaker transcript
Most systems answer an audio clip with a single strip of text. When multiple speakers are present, speech gets cut into turns, each turn is transcribed, the pieces are concatenated. That structure has a hard ceiling: when two people speak at once, only one of them can be written down or in some systems, both speakers get wrongly transcribed in the same utterance.
Saaras-multispk returns a lane per speaker. Where two people overlap, both are transcribed.
Below is one ~30-second clip from an Indian news debate — an anchor and a panelist talking over each other almost continuously — run through three systems on the same time axis. Press play and watch the three boxes underneath fill in as the audio moves.
Multi-Speaker Comparison
3 systemsSpeaker 1उस पर भी सरकार की जवाबदेही बनती है, और reform को लेकर जब सुधार की बात हो रही है वो संसद में ही होगा ना sir। वहां भी
Speaker 1उस पर भी सरकार की जवाबदेही बनती है और reform को लेकर क्या सुधार की जब बात हो रही हो संसद में ही होगा ना सर वहां भी होगा हंगामा।
Speaker 1उस पर भी सरकार की जवाबदेही बनती है और reform को लेकर क्या सुधार की जब बात हो रही है वह संसद में ही होगा ना sir वहां भी होगा हंगामा
Click or drag anywhere on the ribbon to scrub. Hatched strips mark seconds where two people are speaking and that system returned only one of them. Segment boundaries and speaker labels are the systems’ own outputs.
The other systems’ words are mostly fine. What they lose is the anchor cutting in — six separate overlap regions each, including the whole exchange from 27 to 30 seconds where both people are talking and only one voice comes back.
What changed, before the numbers
Four things you can observe without running a scoring script.
Overlapping speech survives
The headline. In the clip above, the anchor’s interruptions land on the anchor while the panelist keeps going. Both of the other systems resolve those same seconds to whoever was louder and discard the rest.
Short turns are kept, not swallowed
Backchannels like हाँ, अच्छा and बिल्कुल are the most-dropped unit in multi-speaker transcriptions as the ‘cut audio→transcribe→concatenate’ pipeline most systems use force the ASR models to transcribe on short chunks resulting in missing or garbage output.
Fewer hallucinated continuations
The classic multi-speaker failure is one speaker’s sentence being completed with another speaker’s words — grammatical, fluent, entirely wrong, and invisible unless you have the audio open next to the transcript. Compare Deepgram’s opening of the second turn in the clip, where a mis-joined boundary produces झड़प पर ही भी तो कर हैं लोग प्रदर्शन against the actual सड़क पर तो कर ही रहे हैं आप लोग प्रदर्शन. While ElevenLabs produces संसद पर तो कर ही रहे हैं आप लोग प्रदर्शन which is again wrong — another speaker’s word picked up.
Code-switching holds through turn changes
The debate clip is continuous Hindi–English: reform, sir, प्रदर्शन in one breath. Script and language choice stay correct across speaker boundaries instead of resetting at each turn.
Quantitative, not qualitative
Multi-speaker ASR had no Indic benchmark, so we built one with AI4Bharat. Indic DiarBench is the first open benchmark for joint diarization and speaker-attributed ASR across all 22 scheduled languages, and is being presented at Interspeech 2026.
22 Indian languages · 108 hours of audio · 1,200+ unique speakers
We look at two metrics, because either of them alone can be gamed. DER scores the timestamps with speaker attribution. cpWER — concatenated minimum-permutation word error rate — scores the transcript with attribution, under the best speaker mapping. A system can post a respectable DER and a respectable WER and still post a terrible cpWER and vice-versa, which is what happens when overlaps get dropped: the words exist, they are just filed under the wrong person or not at all.
Both plots put DER on the horizontal axis and cpWER on the vertical. Lower is better on both, so bottom-left wins.
AMI single-distant-microphone is the standard English far-field meeting set, included so the Indic result cannot be waved away as home-ground advantage. The model leads on both axes there too.
Both benchmarks · DER and cpWER · lower is better
| System | DiarBench DER | DiarBench cpWER | AMI DER | AMI cpWER |
|---|---|---|---|---|
| Saaras-multispk | 16.00 | 35.17 | 14.56 | 20.64 |
| AWS Transcribe | 23.50 | 43.70 | 29.23 | 29.02 |
| Deepgram Nova-3 | 32.00 | 63.20 | 48.45 | 58.14 |
| Azure STT | 34.80 | 60.80 | 32.32 | 36.36 |
| ElevenLabs Scribe | 35.00 | 58.30 | 43.54 | 31.85 |
| GPT-4o Transcribe | 36.20 | 83.10 | 29.32 | 23.44 |
Both benchmarks · DER and cpWER · lower is better
Five ways to render the same audio
Saaras-multispk ships with five output modes — transcribe, verbatim, codemix, translit and translate.
Below is the same debate clip from above, run once per mode. Press play and watch all five boxes update on the same time axis.
Five Output Modes
Shared diarizationSpk 1पर भी सरकार की जवाबदेही बनती है और रिफॉर्म को लेकर क्या सुधार की जब बात हो रही है वो संसद में ही होगा ना सर वहां भी
Spk 1पर भी सरकार की जवाबदेही बनती है और रिफॉर्म को लेकर क्या सुधार की जब बात हो रही है वो संसद में ही होगा ना सर वहां भी
Spk 1पर भी सरकार की जवाबदेही बनती है और reform को लेकर क्या सुधार की जब बात हो रही है वो संसद में ही होगा ना sir वहां भी
Spk 1par bhi sarkar ki jawabdehi banti hai aur reform ko lekar kya sudhaar ki jab baat ho rahi hai vo sansad mein hi hoga na sir wahan bhi
Spk 1the government is also accountable for that and when there is talk of reforms what improvements are being made that will happen in the parliament right sir there too there
Same clip and the same speaker-lane cuts as the competitor comparison above. Press play — all five mode boxes update on that shared timeline.
How to hit the API
Speaker labels come from the Batch endpoint. The synchronous REST endpoint caps at 30 seconds and does not diarize at all, so multi-speaker work goes through the job-based flow: create, upload, start, poll, download. Files up to two hours, 20 per job.
Pass num_speakers whenever you know it. Auto-detection is good, a known count is better — and on single-speaker audio it removes the phantom-speaker question entirely.
# pip install sarvamai
from sarvamai import SarvamAI
client = SarvamAI(api_subscription_key="YOUR_API_KEY")
job = client.speech_to_text_job.create_job(
model="saaras-multispk",
with_diarization=True,
num_speakers=2, # omit to auto-detect
language_code="unknown", # or "hi-IN", "ta-IN", ...
with_timestamps=True,
)
job.upload_files(file_paths=["debate.wav"])
job.start()
job.wait_until_complete() # SDK raises after 600 s
job.download_outputs(output_dir="./out")Results land as one JSON per input file. With diarization on you get diarized_transcript alongside the flat transcript — the entries are what drive a speaker-attributed view:
{
"transcript": "...",
"language_code": "hi-IN",
"diarized_transcript": {
"entries": [
{
"speaker_id": "SPEAKER_00",
"transcript": "...",
"start_time_seconds": 0.0,
"end_time_seconds": 6.4
}
]
}
}Don’t poll if you don’t have to. Pass a callback URL at job creation and Sarvam POSTs you on completion. The webhook carries job state, not transcripts — you still call download once it fires, and your endpoint must return 200 within 30 seconds.
Limits worth knowing
Audio works best at 16 kHz. Multichannel input is downmixed to mono, so if you already have one channel per speaker you are better off transcribing them separately than relying on diarization. PCM inputs need input_audio_codec set explicitly.
FAQ section (TO DO)
Curious what else we're building? Explore our APIs and start creating.
Curious what else we're building?
Explore our APIs and start creating.