fix: reconcile speakers automatically by default

The plain transcribe command still used chunk overlap speaker matching,
so users had to discover a model flag or supply a known speaker count to
get the improved voice comparison behavior.

Enable OpenRouter voice comparison with automatic speaker counting for
normal text and SRT invocations. Keep --speakers and --speaker-model as
optional overrides, and let --no-diarization disable both provider labels
and voice comparison. Make the basic command the documentation entry point
and clarify the defaults in CLI help and repository guidance.

Cover bare invocations across one and multiple uploads, default SRT,
optional count constraints, opt-out ordering, and atomic output when the
default comparison fails.

Test Plan:
- python3 -B -m unittest -v: all 85 tests passed.
- bash -n transcribe: passed.
- git diff --cached --check: passed.
- Tests use real audio encoding with fake provider responses. Live voice
  accuracy remains unverified; no live API request was made for this change.
This commit is contained in:
ddidderr committed 2026-09-12 10:28:42 +02:00
1 parent 138c796399
commit d7827d39f5
5 files changed
+107 -48

No files matched your search

+4 -1
View File
@@ -24,7 +24,7 @@
- `chunks.py`: upload boundaries and ordered overlap stitching. - `chunks.py`: upload boundaries and ordered overlap stitching.
- `subtitles.py`: timestamp validation and SRT cue formatting. - `subtitles.py`: timestamp validation and SRT cue formatting.
- `transcript.py`: readable text paragraphs and speaker markers. - `transcript.py`: readable text paragraphs and speaker markers.
- `diarization.py`: optional OpenRouter audio sample comparison and speaker ID reconciliation. - `diarization.py`: automatic OpenRouter audio sample comparison and speaker ID reconciliation.
- `test_*.py` and `tests/fake_provider.py`: local regression tests without real - `test_*.py` and `tests/fake_provider.py`: local regression tests without real
credentials or provider requests. credentials or provider requests.
@@ -36,3 +36,6 @@ must preserve transcription text and timestamps, validate complete sample
assignments, and leave uncertain or conflicting identities separate. Keep chunk assignments, and leave uncertain or conflicting identities separate. Keep chunk
details out of the final transcript. details out of the final transcript.
Progress, processing details, and errors belong on stderr. Progress, processing details, and errors belong on stderr.
The normal command detects the speaker count and reconciles voices automatically;
`--speakers` and `--speaker-model` are optional overrides. `--no-diarization`
disables both provider diarization and the voice comparison request.
+25 -32
View File
@@ -1,23 +1,18 @@
# Transcribe # Transcribe
```sh
./transcribe too_long.webm > transcript.txt
```
Speaker detection and matching across the recording happen automatically.
You do not need to know the number of speakers or choose a model.
Use `--srt` for subtitles or `--no-diarization` for text without speaker labels.
Requires Bash 5.1 or newer, FFmpeg, curl, jq, Python 3, and standard coreutils. Keep Requires Bash 5.1 or newer, FFmpeg, curl, jq, Python 3, and standard coreutils. Keep
`chunks.py`, `subtitles.py`, `transcript.py`, and `diarization.py` beside the script. `chunks.py`, `subtitles.py`, `transcript.py`, and `diarization.py` beside the script.
The API key is read using the 1Password CLI (`op`) from The API key is read using the 1Password CLI (`op`) from
`op://API Keys/openrouter-mai-transcribe-2/credential`. `op://API Keys/openrouter-mai-transcribe-2/credential`.
```sh
./transcribe recording.m4a
./transcribe video.mp4 > transcript.txt
./transcribe --srt video.mp4 > subtitles.srt
./transcribe --no-diarization recording.m4a > transcript.txt
./transcribe --srt --no-diarization video.mp4 > subtitles.srt
./transcribe --chunk-minutes 10 too_long.webm > transcript.txt
./transcribe --jobs 4 too_long.webm > transcript.txt
./transcribe --jobs 1 --srt too_long.webm > subtitles.srt
./transcribe --speakers 2 too_long.webm > transcript.txt
./transcribe --speaker-model google/gemini-3.8-flash recording.m4a > transcript.txt
```
Accepts m4a, mp4, aac, flac, opus, mp3, ogg, wav, and other formats your Accepts m4a, mp4, aac, flac, opus, mp3, ogg, wav, and other formats your
FFmpeg supports. Uses the first audio track, converts it to mono 16 kHz MP3 FFmpeg supports. Uses the first audio track, converts it to mono 16 kHz MP3
at 64 kbps (requires FFmpeg's libmp3lame encoder), at 64 kbps (requires FFmpeg's libmp3lame encoder),
@@ -25,11 +20,14 @@ and sends it to OpenRouter's `microsoft/mai-transcribe-2` model. Temporary
files are removed on exit. The transcript goes to stdout; progress and errors files are removed on exit. The transcript goes to stdout; progress and errors
go to stderr. Output is emitted only after all requests and conversion succeed. go to stderr. Output is emitted only after all requests and conversion succeed.
Speaker diarization is **enabled by default** through Speaker diarization and voice comparison are **enabled by default**.
MAI first labels speech through
`provider.options.azure.diarization.enabled`, following `provider.options.azure.diarization.enabled`, following
[OpenRouter's speaker diarization API](https://openrouter.ai/docs/guides/overview/multimodal/stt#verbose-transcripts-timestamps-and-speakers). [OpenRouter's speaker diarization API](https://openrouter.ai/docs/guides/overview/multimodal/stt#verbose-transcripts-timestamps-and-speakers).
Use `--no-diarization` to explicitly disable it for every request, with either Gemini then compares voice excerpts through OpenRouter to match speakers across
plain text or `--srt` output. Text output prints returned IDs above each paragraph: the recording, automatically estimating their number. `--no-diarization`
disables both steps, with either plain text or `--srt` output. Text output prints
speaker IDs above each paragraph:
```text ```text
[Speaker 0] [Speaker 0]
@@ -83,22 +81,17 @@ are an error. Plain text for a chunked recording is assembled from the words.
SRT timestamps are shifted back to the original recording's timeline before SRT timestamps are shifted back to the original recording's timeline before
cue generation, with continuous cue numbering. Text uses `[Speaker 0]`, cue generation, with continuous cue numbering. Text uses `[Speaker 0]`,
`[Speaker 1]`, etc., without exposing upload boundaries to the reader. `[Speaker 1]`, etc., without exposing upload boundaries to the reader.
For a chunked recording, provider IDs are mapped to transcript-wide IDs: Provider IDs are request-local. The automatic voice comparison maps them to
at least two unambiguous words shared at matching times in the overlap must transcript-wide IDs using audio evidence, including for people absent from a
agree on the speaker correspondence. Conflicting or absent evidence gets a chunk overlap. Numeric provider IDs alone never establish identity across
new ID and a warning on stderr. This is not voice recognition across the whole requests.
recording: the same person can receive another ID when absent from an overlap
or when diarization disagrees. Numeric provider IDs alone are never used to
identify a speaker across requests.
For interviews and other recordings with fragmented speaker IDs, use Voice comparison uses `google/gemini-3.8-flash` by default and estimates the
`--speakers N` (1 to 64) to add a voice comparison pass through OpenRouter. number of speakers. If you already know the count, `--speakers N` (1 to 64)
This selects `google/gemini-3.8-flash` and tells it the known number of people. supplies it as an optional constraint. `--speaker-model MODEL` overrides the
Use `--speaker-model MODEL` to select a different audio chat model supporting default with another audio chat model supporting multiple audio inputs and JSON
multiple audio inputs and JSON schema output; without `--speakers`, the model schema output. Neither flag is needed for the normal command. Both options
estimates the number of voices. Both options require require diarization and cannot be combined with `--no-diarization`.
diarization and cannot be combined with `--no-diarization`. The normal command
without either option retains the overlap-based behavior described above.
The comparison keeps MAI's words and timestamps, initially treating IDs from The comparison keeps MAI's words and timestamps, initially treating IDs from
different requests as separate. It takes up to three nonoverlapping excerpts different requests as separate. It takes up to three nonoverlapping excerpts
@@ -150,7 +143,7 @@ cancels the remaining workers. Interruptions also stop workers and their child
processes before temporary files are removed. Requests already accepted by the processes before temporary files are removed. Requests already accepted by the
provider may still be processed and billed. There is no automatic retry or provider may still be processed and billed. There is no automatic retry or
resume, and rerunning sends earlier chunks again. No transcript is emitted resume, and rerunning sends earlier chunks again. No transcript is emitted
unless every worker, any requested voice comparison, and the final conversion unless every worker, the enabled voice comparison, and the final conversion
succeed. succeed.
`--srt` requests word timestamps using OpenRouter's `verbose_json` response. `--srt` requests word timestamps using OpenRouter's `verbose_json` response.
+64 -4
View File
@@ -53,6 +53,14 @@ class TranscribeTests(unittest.TestCase):
def speaker_request(self): def speaker_request(self):
return json.loads(self.log.with_suffix(".speakers.json").read_text()) return json.loads(self.log.with_suffix(".speakers.json").read_text())
def assert_automatic_speaker_comparison(self):
request = self.speaker_request()["request"]
self.assertEqual(request["model"], "google/gemini-3.8-flash")
schema = request["response_format"]["json_schema"]["schema"]
speaker_schema = schema["properties"]["assignments"]["items"]["properties"]["speaker"]
self.assertNotIn("maximum", speaker_schema)
self.assertNotIn("complete recording has exactly", request["messages"][0]["content"])
def assert_processes_stopped(self): def assert_processes_stopped(self):
for event in self.events(): for event in self.events():
for pid in (event["pid"], event.get("child_pid")): for pid in (event["pid"], event.get("child_pid")):
@@ -236,7 +244,8 @@ class TranscribeTests(unittest.TestCase):
self.assertEqual(self.requests()[0]["response_format"], "verbose_json") self.assertEqual(self.requests()[0]["response_format"], "verbose_json")
self.assertEqual(self.requests()[0]["timestamp_granularities"], ["segment", "word"]) self.assertEqual(self.requests()[0]["timestamp_granularities"], ["segment", "word"])
self.assert_diarization(True) self.assert_diarization(True)
self.assertEqual(result.stderr, "") self.assertIn("no clean speaker samples", result.stderr)
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
self.assert_cleaned() self.assert_cleaned()
def test_chunked_text_has_speaker_markers_without_chunk_details(self): def test_chunked_text_has_speaker_markers_without_chunk_details(self):
@@ -248,7 +257,8 @@ class TranscribeTests(unittest.TestCase):
"[Speaker 1]\ncontinues.\n\n" "[Speaker 1]\ncontinues.\n\n"
"[Speaker 1]\nLast.\n" "[Speaker 1]\nLast.\n"
)) ))
self.assertIn("speaker continuity is uncertain", result.stderr) self.assertIn("no clean speaker samples", result.stderr)
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
self.assertNotIn("Chunk", result.stdout) self.assertNotIn("Chunk", result.stdout)
self.assert_cleaned() self.assert_cleaned()
@@ -276,7 +286,7 @@ class TranscribeTests(unittest.TestCase):
for output in ((), ("--srt",)): for output in ((), ("--srt",)):
for chunked in (False, True): for chunked in (False, True):
with self.subTest(output=output, chunked=chunked): with self.subTest(output=output, chunked=chunked):
self.log.unlink(missing_ok=True) self.reset_logs()
options = ("--chunk-minutes", "1") if chunked else () options = ("--chunk-minutes", "1") if chunked else ()
result = self.run_cli( result = self.run_cli(
*output, *options, "--no-diarization", duration=70 if chunked else 5, *output, *options, "--no-diarization", duration=70 if chunked else 5,
@@ -285,6 +295,7 @@ class TranscribeTests(unittest.TestCase):
self.assertTrue(result.stdout) self.assertTrue(result.stdout)
self.assertEqual(len(self.requests()), 2 if chunked else 1) self.assertEqual(len(self.requests()), 2 if chunked else 1)
self.assert_diarization(False) self.assert_diarization(False)
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
for request in self.requests(): for request in self.requests():
if output or chunked: if output or chunked:
self.assertEqual(request["response_format"], "verbose_json") self.assertEqual(request["response_format"], "verbose_json")
@@ -302,6 +313,22 @@ class TranscribeTests(unittest.TestCase):
self.assert_diarization(False) self.assert_diarization(False)
self.assert_cleaned() self.assert_cleaned()
def test_no_diarization_skips_voice_comparison_with_usable_samples(self):
# Do not compare voices even if a provider returns labels after opt-out.
for output in ((), ("--srt",)):
for options in (("--no-diarization", "--chunk-minutes", "1", "--jobs", "1"),
("--chunk-minutes", "1", "--jobs", "2", "--no-diarization")):
with self.subTest(output=output, options=options):
self.reset_logs()
result = self.run_cli(*output, *options, mode="reconcile")
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn("AlphaLate.", result.stdout)
self.assertEqual(len(self.requests()), 2)
self.assert_diarization(False)
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
self.assertNotIn("Acoustic speaker matching", result.stderr)
self.assert_cleaned()
def test_default_upload_limits_depend_on_diarization(self): def test_default_upload_limits_depend_on_diarization(self):
for options, limit in (((), 600), (("--no-diarization",), 1800)): for options, limit in (((), 600), (("--no-diarization",), 1800)):
with self.subTest(options=options): with self.subTest(options=options):
@@ -356,12 +383,44 @@ class TranscribeTests(unittest.TestCase):
self.assertNotIn("speaker continuity is uncertain", result.stderr) self.assertNotIn("speaker continuity is uncertain", result.stderr)
comparison = self.speaker_request() comparison = self.speaker_request()
self.assertEqual(comparison["request"]["model"], "google/gemini-3.8-flash") self.assertEqual(comparison["request"]["model"], "google/gemini-3.8-flash")
schema = comparison["request"]["response_format"]["json_schema"]["schema"]
self.assertEqual(schema["properties"]["assignments"]["items"]["properties"]["speaker"]["maximum"], 1)
self.assertIn("exactly 2 speakers", comparison["request"]["messages"][0]["content"])
self.assertEqual(len(comparison["durations"]), 5) self.assertEqual(len(comparison["durations"]), 5)
self.assertTrue(all(1.5 <= duration <= 8.1 for duration in comparison["durations"])) self.assertTrue(all(1.5 <= duration <= 8.1 for duration in comparison["durations"]))
self.assertEqual(len(self.requests()), 2) self.assertEqual(len(self.requests()), 2)
self.assert_diarization(True) self.assert_diarization(True)
self.assert_cleaned() self.assert_cleaned()
def test_bare_command_compares_voices_without_a_known_speaker_count(self):
for duration in (30, 611):
with self.subTest(duration=duration):
self.reset_logs()
result = self.run_cli(mode="reconcile", duration=duration)
self.assertEqual(result.returncode, 0, result.stderr)
expected = "[Speaker 0]\nAlpha.\n\n[Speaker 0]\nAgain.\n\n[Speaker 1]\nBeta.\n"
if duration > 600:
expected += "\n[Speaker 0]\nAlphaLate.\n\n[Speaker 1]\nBetaLate.\n"
self.assertEqual(result.stdout, expected)
self.assert_automatic_speaker_comparison()
self.assertEqual(len(self.requests()), 2 if duration > 600 else 1)
self.assertEqual(len(self.speaker_request()["samples"]), 5 if duration > 600 else 3)
self.assert_diarization(True)
self.assert_cleaned()
def test_default_srt_compares_voices_without_speaker_options(self):
result = self.run_cli("--srt", mode="reconcile", duration=30)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout, (
"1\n00:00:01,000 --> 00:00:03,000\nAlpha.\n\n"
"2\n00:00:06,000 --> 00:00:08,000\nAgain.\n\n"
"3\n00:00:12,000 --> 00:00:14,000\nBeta.\n\n"
))
self.assert_automatic_speaker_comparison()
self.assertEqual(len(self.requests()), 1)
self.assert_diarization(True)
self.assert_cleaned()
def test_voice_comparison_model_override_without_count_and_single_request(self): def test_voice_comparison_model_override_without_count_and_single_request(self):
for model, count in (("test/audio-model", ()), ("~audio-model", ("--speakers", "2"))): for model, count in (("test/audio-model", ()), ("~audio-model", ("--speakers", "2"))):
with self.subTest(model=model): with self.subTest(model=model):
@@ -411,7 +470,7 @@ class TranscribeTests(unittest.TestCase):
"reconcile-truncated"): "reconcile-truncated"):
with self.subTest(mode=mode): with self.subTest(mode=mode):
self.reset_logs() self.reset_logs()
result = self.run_cli("--speakers", "2", "--chunk-minutes", "1", mode=mode) result = self.run_cli("--chunk-minutes", "1", mode=mode)
self.assertNotEqual(result.returncode, 0) self.assertNotEqual(result.returncode, 0)
self.assertEqual(result.stdout, "") self.assertEqual(result.stdout, "")
self.assertTrue(self.log.with_suffix(".speakers.json").exists()) self.assertTrue(self.log.with_suffix(".speakers.json").exists())
@@ -438,6 +497,7 @@ class TranscribeTests(unittest.TestCase):
options += [("--speaker-model", value) for value in ("", "-bad", "bad model", "a\nb")] options += [("--speaker-model", value) for value in ("", "-bad", "bad model", "a\nb")]
options += [("--no-diarization", "--speakers", "2"), options += [("--no-diarization", "--speakers", "2"),
("--speakers", "2", "--no-diarization"), ("--speakers", "2", "--no-diarization"),
("--no-diarization", "--speaker-model", "test/audio"),
("--speaker-model", "test/audio", "--no-diarization")] ("--speaker-model", "test/audio", "--no-diarization")]
for args in options: for args in options:
with self.subTest(options=args): with self.subTest(options=args):
+5 -2
View File
@@ -35,6 +35,8 @@ def event(kind, **extra):
event("start") event("start")
if is_comparison: if is_comparison:
samples = json.loads((output.parent / "samples.json").read_text()) samples = json.loads((output.parent / "samples.json").read_text())
manifest = json.loads((staging / "chunks.json").read_text())
late_start = round(manifest[0]["end"]) if len(manifest) > 1 else 60
content = request["messages"][-1]["content"] content = request["messages"][-1]["content"]
audio_parts = [part["input_audio"] for part in content if part["type"] == "input_audio"] audio_parts = [part["input_audio"] for part in content if part["type"] == "input_audio"]
assert len(content) == 2 * len(samples) == 2 * len(audio_parts) assert len(content) == 2 * len(samples) == 2 * len(audio_parts)
@@ -73,7 +75,7 @@ if is_comparison:
if mode == "reconcile-bad-json": if mode == "reconcile-bad-json":
output.write_text("not json") output.write_text("not json")
sys.exit(0) sys.exit(0)
assignments = [{"sample": sample["id"], "speaker": 0 if round(sample["start"]) in (1, 6, 60) else 1} assignments = [{"sample": sample["id"], "speaker": 0 if round(sample["start"]) in (1, 6, late_start) else 1}
for sample in samples] for sample in samples]
if mode == "reconcile-missing": if mode == "reconcile-missing":
assignments.pop() assignments.pop()
@@ -182,12 +184,13 @@ else:
if mode.startswith("reconcile"): if mode.startswith("reconcile"):
# Two labels already describe voice A inside the first request; later # Two labels already describe voice A inside the first request; later
# chunks reuse the same local numbers for different request identities. # chunks reuse the same local numbers for different request identities.
late_start = round(manifest[0]["end"]) if len(manifest) > 1 else 60
words = [ words = [
{"word": text, "start": start - chunk["start"], "end": start + 2 - chunk["start"], {"word": text, "start": start - chunk["start"], "end": start + 2 - chunk["start"],
"speaker": speaker} "speaker": speaker}
for text, start, speaker in [ for text, start, speaker in [
("Alpha.", 1, 7), ("Again.", 6, 8), ("Beta.", 12, 9), ("Alpha.", 1, 7), ("Again.", 6, 8), ("Beta.", 12, 9),
("AlphaLate.", 60, 9), ("BetaLate.", 65, 7), ("AlphaLate.", late_start, 9), ("BetaLate.", late_start + 5, 7),
] if start >= chunk["start"] and start + 2 <= chunk["end"] ] if start >= chunk["start"] and start + 2 <= chunk["end"]
] ]
response = {"text": " ".join(w["word"] for w in words), "words": words} response = {"text": " ".join(w["word"] for w in words), "words": words}
+9 -9
View File
@@ -10,12 +10,12 @@ speaker_model=
speakers= speakers=
usage() { usage() {
echo "Usage: ${0##*/} [--srt] [--no-diarization] [--speakers N] [--speaker-model MODEL] [--chunk-minutes 1..30] [--jobs 1..64] AUDIO_FILE" echo "Usage: ${0##*/} [--srt] [--no-diarization] [--speakers N] [--speaker-model MODEL] [--chunk-minutes 1..30] [--jobs 1..64] AUDIO_FILE"
echo 'Transcribe audio using OPENROUTER_API_KEY; print wrapped text with speaker markers to stdout.' echo 'Transcribe audio with automatic speaker detection; print wrapped text to stdout.'
echo 'Speakers are matched across the recording automatically through OpenRouter.'
echo ' --srt Print SRT subtitles in short cues using word timestamps instead.' echo ' --srt Print SRT subtitles in short cues using word timestamps instead.'
echo ' --no-diarization Disable speaker diarization (enabled by default).' echo ' --no-diarization Disable speaker detection and voice comparison.'
echo ' --speakers N Reconcile voices through OpenRouter with a known count (1..64).' echo ' --speakers N Optional known speaker count (1..64; otherwise detected automatically).'
echo ' --speaker-model MODEL Reconcile voices with an audio chat model (count optional).' echo ' --speaker-model MODEL Override the voice comparison model (default: google/gemini-3.8-flash).'
echo ' --speakers defaults to google/gemini-3.8-flash; adds one audio request.'
echo ' --chunk-minutes N Maximum upload duration in minutes (default: 10; 30 with --no-diarization).' echo ' --chunk-minutes N Maximum upload duration in minutes (default: 10; 30 with --no-diarization).'
echo ' --jobs N Maximum chunks processed in parallel (default: 8; 1 for sequential).' echo ' --jobs N Maximum chunks processed in parallel (default: 8; 1 for sequential).'
} }
@@ -61,13 +61,13 @@ while [[ $# -gt 0 ]]; do
*) break ;; *) break ;;
esac esac
done done
if [[ -n $speakers && -z $speaker_model ]]; then if [[ -n $speaker_model || -n $speakers ]] && ! "$diarization"; then
speaker_model=google/gemini-3.8-flash
fi
if [[ -n $speaker_model ]] && ! "$diarization"; then
echo '--speakers and --speaker-model cannot be combined with --no-diarization' >&2 echo '--speakers and --speaker-model cannot be combined with --no-diarization' >&2
exit 1 exit 1
fi fi
if "$diarization" && [[ -z $speaker_model ]]; then
speaker_model=google/gemini-3.8-flash
fi
if [[ -z $chunk_minutes ]]; then if [[ -z $chunk_minutes ]]; then
# Azure documents diarization failures for recordings around 15 minutes. # Azure documents diarization failures for recordings around 15 minutes.
if "$diarization"; then chunk_minutes=10; else chunk_minutes=30; fi if "$diarization"; then chunk_minutes=10; else chunk_minutes=30; fi