fix: reconcile speakers automatically by default

The plain transcribe command still used chunk overlap speaker matching,
so users had to discover a model flag or supply a known speaker count to
get the improved voice comparison behavior.

Enable OpenRouter voice comparison with automatic speaker counting for
normal text and SRT invocations. Keep --speakers and --speaker-model as
optional overrides, and let --no-diarization disable both provider labels
and voice comparison. Make the basic command the documentation entry point
and clarify the defaults in CLI help and repository guidance.

Cover bare invocations across one and multiple uploads, default SRT,
optional count constraints, opt-out ordering, and atomic output when the
default comparison fails.

Test Plan:
- python3 -B -m unittest -v: all 85 tests passed.
- bash -n transcribe: passed.
- git diff --cached --check: passed.
- Tests use real audio encoding with fake provider responses. Live voice
  accuracy remains unverified; no live API request was made for this change.
This commit is contained in:
ddidderr committed 2026-09-12 10:28:42 +02:00
1 parent 138c796399
commit d7827d39f5
5 files changed
+107 -48

No files matched your search

+4 -1
View File
@@ -24,7 +24,7 @@
- `chunks.py`: upload boundaries and ordered overlap stitching.
- `subtitles.py`: timestamp validation and SRT cue formatting.
- `transcript.py`: readable text paragraphs and speaker markers.
- `diarization.py`: optional OpenRouter audio sample comparison and speaker ID reconciliation.
- `diarization.py`: automatic OpenRouter audio sample comparison and speaker ID reconciliation.
- `test_*.py` and `tests/fake_provider.py`: local regression tests without real
credentials or provider requests.
@@ -36,3 +36,6 @@ must preserve transcription text and timestamps, validate complete sample
assignments, and leave uncertain or conflicting identities separate. Keep chunk
details out of the final transcript.
Progress, processing details, and errors belong on stderr.
The normal command detects the speaker count and reconciles voices automatically;
`--speakers` and `--speaker-model` are optional overrides. `--no-diarization`
disables both provider diarization and the voice comparison request.
+25 -32
View File
@@ -1,23 +1,18 @@
# Transcribe
```sh
./transcribe too_long.webm > transcript.txt
```
Speaker detection and matching across the recording happen automatically.
You do not need to know the number of speakers or choose a model.
Use `--srt` for subtitles or `--no-diarization` for text without speaker labels.
Requires Bash 5.1 or newer, FFmpeg, curl, jq, Python 3, and standard coreutils. Keep
`chunks.py`, `subtitles.py`, `transcript.py`, and `diarization.py` beside the script.
The API key is read using the 1Password CLI (`op`) from
`op://API Keys/openrouter-mai-transcribe-2/credential`.
```sh
./transcribe recording.m4a
./transcribe video.mp4 > transcript.txt
./transcribe --srt video.mp4 > subtitles.srt
./transcribe --no-diarization recording.m4a > transcript.txt
./transcribe --srt --no-diarization video.mp4 > subtitles.srt
./transcribe --chunk-minutes 10 too_long.webm > transcript.txt
./transcribe --jobs 4 too_long.webm > transcript.txt
./transcribe --jobs 1 --srt too_long.webm > subtitles.srt
./transcribe --speakers 2 too_long.webm > transcript.txt
./transcribe --speaker-model google/gemini-3.8-flash recording.m4a > transcript.txt
```
Accepts m4a, mp4, aac, flac, opus, mp3, ogg, wav, and other formats your
FFmpeg supports. Uses the first audio track, converts it to mono 16 kHz MP3
at 64 kbps (requires FFmpeg's libmp3lame encoder),
@@ -25,11 +20,14 @@ and sends it to OpenRouter's `microsoft/mai-transcribe-2` model. Temporary
files are removed on exit. The transcript goes to stdout; progress and errors
go to stderr. Output is emitted only after all requests and conversion succeed.
Speaker diarization is **enabled by default** through
Speaker diarization and voice comparison are **enabled by default**.
MAI first labels speech through
`provider.options.azure.diarization.enabled`, following
[OpenRouter's speaker diarization API](https://openrouter.ai/docs/guides/overview/multimodal/stt#verbose-transcripts-timestamps-and-speakers).
Use `--no-diarization` to explicitly disable it for every request, with either
plain text or `--srt` output. Text output prints returned IDs above each paragraph:
Gemini then compares voice excerpts through OpenRouter to match speakers across
the recording, automatically estimating their number. `--no-diarization`
disables both steps, with either plain text or `--srt` output. Text output prints
speaker IDs above each paragraph:
```text
[Speaker 0]
@@ -83,22 +81,17 @@ are an error. Plain text for a chunked recording is assembled from the words.
SRT timestamps are shifted back to the original recording's timeline before
cue generation, with continuous cue numbering. Text uses `[Speaker 0]`,
`[Speaker 1]`, etc., without exposing upload boundaries to the reader.
For a chunked recording, provider IDs are mapped to transcript-wide IDs:
at least two unambiguous words shared at matching times in the overlap must
agree on the speaker correspondence. Conflicting or absent evidence gets a
new ID and a warning on stderr. This is not voice recognition across the whole
recording: the same person can receive another ID when absent from an overlap
or when diarization disagrees. Numeric provider IDs alone are never used to
identify a speaker across requests.
Provider IDs are request-local. The automatic voice comparison maps them to
transcript-wide IDs using audio evidence, including for people absent from a
chunk overlap. Numeric provider IDs alone never establish identity across
requests.
For interviews and other recordings with fragmented speaker IDs, use
`--speakers N` (1 to 64) to add a voice comparison pass through OpenRouter.
This selects `google/gemini-3.8-flash` and tells it the known number of people.
Use `--speaker-model MODEL` to select a different audio chat model supporting
multiple audio inputs and JSON schema output; without `--speakers`, the model
estimates the number of voices. Both options require
diarization and cannot be combined with `--no-diarization`. The normal command
without either option retains the overlap-based behavior described above.
Voice comparison uses `google/gemini-3.8-flash` by default and estimates the
number of speakers. If you already know the count, `--speakers N` (1 to 64)
supplies it as an optional constraint. `--speaker-model MODEL` overrides the
default with another audio chat model supporting multiple audio inputs and JSON
schema output. Neither flag is needed for the normal command. Both options
require diarization and cannot be combined with `--no-diarization`.
The comparison keeps MAI's words and timestamps, initially treating IDs from
different requests as separate. It takes up to three nonoverlapping excerpts
@@ -150,7 +143,7 @@ cancels the remaining workers. Interruptions also stop workers and their child
processes before temporary files are removed. Requests already accepted by the
provider may still be processed and billed. There is no automatic retry or
resume, and rerunning sends earlier chunks again. No transcript is emitted
unless every worker, any requested voice comparison, and the final conversion
unless every worker, the enabled voice comparison, and the final conversion
succeed.
`--srt` requests word timestamps using OpenRouter's `verbose_json` response.
+64 -4
View File
@@ -53,6 +53,14 @@ class TranscribeTests(unittest.TestCase):
def speaker_request(self):
return json.loads(self.log.with_suffix(".speakers.json").read_text())
def assert_automatic_speaker_comparison(self):
request = self.speaker_request()["request"]
self.assertEqual(request["model"], "google/gemini-3.8-flash")
schema = request["response_format"]["json_schema"]["schema"]
speaker_schema = schema["properties"]["assignments"]["items"]["properties"]["speaker"]
self.assertNotIn("maximum", speaker_schema)
self.assertNotIn("complete recording has exactly", request["messages"][0]["content"])
def assert_processes_stopped(self):
for event in self.events():
for pid in (event["pid"], event.get("child_pid")):
@@ -236,7 +244,8 @@ class TranscribeTests(unittest.TestCase):
self.assertEqual(self.requests()[0]["response_format"], "verbose_json")
self.assertEqual(self.requests()[0]["timestamp_granularities"], ["segment", "word"])
self.assert_diarization(True)
self.assertEqual(result.stderr, "")
self.assertIn("no clean speaker samples", result.stderr)
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
self.assert_cleaned()
def test_chunked_text_has_speaker_markers_without_chunk_details(self):
@@ -248,7 +257,8 @@ class TranscribeTests(unittest.TestCase):
"[Speaker 1]\ncontinues.\n\n"
"[Speaker 1]\nLast.\n"
))
self.assertIn("speaker continuity is uncertain", result.stderr)
self.assertIn("no clean speaker samples", result.stderr)
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
self.assertNotIn("Chunk", result.stdout)
self.assert_cleaned()
@@ -276,7 +286,7 @@ class TranscribeTests(unittest.TestCase):
for output in ((), ("--srt",)):
for chunked in (False, True):
with self.subTest(output=output, chunked=chunked):
self.log.unlink(missing_ok=True)
self.reset_logs()
options = ("--chunk-minutes", "1") if chunked else ()
result = self.run_cli(
*output, *options, "--no-diarization", duration=70 if chunked else 5,
@@ -285,6 +295,7 @@ class TranscribeTests(unittest.TestCase):
self.assertTrue(result.stdout)
self.assertEqual(len(self.requests()), 2 if chunked else 1)
self.assert_diarization(False)
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
for request in self.requests():
if output or chunked:
self.assertEqual(request["response_format"], "verbose_json")
@@ -302,6 +313,22 @@ class TranscribeTests(unittest.TestCase):
self.assert_diarization(False)
self.assert_cleaned()
def test_no_diarization_skips_voice_comparison_with_usable_samples(self):
# Do not compare voices even if a provider returns labels after opt-out.
for output in ((), ("--srt",)):
for options in (("--no-diarization", "--chunk-minutes", "1", "--jobs", "1"),
("--chunk-minutes", "1", "--jobs", "2", "--no-diarization")):
with self.subTest(output=output, options=options):
self.reset_logs()
result = self.run_cli(*output, *options, mode="reconcile")
self.assertEqual(result.returncode, 0, result.stderr)
self.assertIn("AlphaLate.", result.stdout)
self.assertEqual(len(self.requests()), 2)
self.assert_diarization(False)
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
self.assertNotIn("Acoustic speaker matching", result.stderr)
self.assert_cleaned()
def test_default_upload_limits_depend_on_diarization(self):
for options, limit in (((), 600), (("--no-diarization",), 1800)):
with self.subTest(options=options):
@@ -356,12 +383,44 @@ class TranscribeTests(unittest.TestCase):
self.assertNotIn("speaker continuity is uncertain", result.stderr)
comparison = self.speaker_request()
self.assertEqual(comparison["request"]["model"], "google/gemini-3.8-flash")
schema = comparison["request"]["response_format"]["json_schema"]["schema"]
self.assertEqual(schema["properties"]["assignments"]["items"]["properties"]["speaker"]["maximum"], 1)
self.assertIn("exactly 2 speakers", comparison["request"]["messages"][0]["content"])
self.assertEqual(len(comparison["durations"]), 5)
self.assertTrue(all(1.5 <= duration <= 8.1 for duration in comparison["durations"]))
self.assertEqual(len(self.requests()), 2)
self.assert_diarization(True)
self.assert_cleaned()
def test_bare_command_compares_voices_without_a_known_speaker_count(self):
for duration in (30, 611):
with self.subTest(duration=duration):
self.reset_logs()
result = self.run_cli(mode="reconcile", duration=duration)
self.assertEqual(result.returncode, 0, result.stderr)
expected = "[Speaker 0]\nAlpha.\n\n[Speaker 0]\nAgain.\n\n[Speaker 1]\nBeta.\n"
if duration > 600:
expected += "\n[Speaker 0]\nAlphaLate.\n\n[Speaker 1]\nBetaLate.\n"
self.assertEqual(result.stdout, expected)
self.assert_automatic_speaker_comparison()
self.assertEqual(len(self.requests()), 2 if duration > 600 else 1)
self.assertEqual(len(self.speaker_request()["samples"]), 5 if duration > 600 else 3)
self.assert_diarization(True)
self.assert_cleaned()
def test_default_srt_compares_voices_without_speaker_options(self):
result = self.run_cli("--srt", mode="reconcile", duration=30)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout, (
"1\n00:00:01,000 --> 00:00:03,000\nAlpha.\n\n"
"2\n00:00:06,000 --> 00:00:08,000\nAgain.\n\n"
"3\n00:00:12,000 --> 00:00:14,000\nBeta.\n\n"
))
self.assert_automatic_speaker_comparison()
self.assertEqual(len(self.requests()), 1)
self.assert_diarization(True)
self.assert_cleaned()
def test_voice_comparison_model_override_without_count_and_single_request(self):
for model, count in (("test/audio-model", ()), ("~audio-model", ("--speakers", "2"))):
with self.subTest(model=model):
@@ -411,7 +470,7 @@ class TranscribeTests(unittest.TestCase):
"reconcile-truncated"):
with self.subTest(mode=mode):
self.reset_logs()
result = self.run_cli("--speakers", "2", "--chunk-minutes", "1", mode=mode)
result = self.run_cli("--chunk-minutes", "1", mode=mode)
self.assertNotEqual(result.returncode, 0)
self.assertEqual(result.stdout, "")
self.assertTrue(self.log.with_suffix(".speakers.json").exists())
@@ -438,6 +497,7 @@ class TranscribeTests(unittest.TestCase):
options += [("--speaker-model", value) for value in ("", "-bad", "bad model", "a\nb")]
options += [("--no-diarization", "--speakers", "2"),
("--speakers", "2", "--no-diarization"),
("--no-diarization", "--speaker-model", "test/audio"),
("--speaker-model", "test/audio", "--no-diarization")]
for args in options:
with self.subTest(options=args):
+5 -2
View File
@@ -35,6 +35,8 @@ def event(kind, **extra):
event("start")
if is_comparison:
samples = json.loads((output.parent / "samples.json").read_text())
manifest = json.loads((staging / "chunks.json").read_text())
late_start = round(manifest[0]["end"]) if len(manifest) > 1 else 60
content = request["messages"][-1]["content"]
audio_parts = [part["input_audio"] for part in content if part["type"] == "input_audio"]
assert len(content) == 2 * len(samples) == 2 * len(audio_parts)
@@ -73,7 +75,7 @@ if is_comparison:
if mode == "reconcile-bad-json":
output.write_text("not json")
sys.exit(0)
assignments = [{"sample": sample["id"], "speaker": 0 if round(sample["start"]) in (1, 6, 60) else 1}
assignments = [{"sample": sample["id"], "speaker": 0 if round(sample["start"]) in (1, 6, late_start) else 1}
for sample in samples]
if mode == "reconcile-missing":
assignments.pop()
@@ -182,12 +184,13 @@ else:
if mode.startswith("reconcile"):
# Two labels already describe voice A inside the first request; later
# chunks reuse the same local numbers for different request identities.
late_start = round(manifest[0]["end"]) if len(manifest) > 1 else 60
words = [
{"word": text, "start": start - chunk["start"], "end": start + 2 - chunk["start"],
"speaker": speaker}
for text, start, speaker in [
("Alpha.", 1, 7), ("Again.", 6, 8), ("Beta.", 12, 9),
("AlphaLate.", 60, 9), ("BetaLate.", 65, 7),
("AlphaLate.", late_start, 9), ("BetaLate.", late_start + 5, 7),
] if start >= chunk["start"] and start + 2 <= chunk["end"]
]
response = {"text": " ".join(w["word"] for w in words), "words": words}
+9 -9
View File
@@ -10,12 +10,12 @@ speaker_model=
speakers=
usage() {
echo "Usage: ${0##*/} [--srt] [--no-diarization] [--speakers N] [--speaker-model MODEL] [--chunk-minutes 1..30] [--jobs 1..64] AUDIO_FILE"
echo 'Transcribe audio using OPENROUTER_API_KEY; print wrapped text with speaker markers to stdout.'
echo 'Transcribe audio with automatic speaker detection; print wrapped text to stdout.'
echo 'Speakers are matched across the recording automatically through OpenRouter.'
echo ' --srt Print SRT subtitles in short cues using word timestamps instead.'
echo ' --no-diarization Disable speaker diarization (enabled by default).'
echo ' --speakers N Reconcile voices through OpenRouter with a known count (1..64).'
echo ' --speaker-model MODEL Reconcile voices with an audio chat model (count optional).'
echo ' --speakers defaults to google/gemini-3.8-flash; adds one audio request.'
echo ' --no-diarization Disable speaker detection and voice comparison.'
echo ' --speakers N Optional known speaker count (1..64; otherwise detected automatically).'
echo ' --speaker-model MODEL Override the voice comparison model (default: google/gemini-3.8-flash).'
echo ' --chunk-minutes N Maximum upload duration in minutes (default: 10; 30 with --no-diarization).'
echo ' --jobs N Maximum chunks processed in parallel (default: 8; 1 for sequential).'
}
@@ -61,13 +61,13 @@ while [[ $# -gt 0 ]]; do
*) break ;;
esac
done
if [[ -n $speakers && -z $speaker_model ]]; then
speaker_model=google/gemini-3.8-flash
fi
if [[ -n $speaker_model ]] && ! "$diarization"; then
if [[ -n $speaker_model || -n $speakers ]] && ! "$diarization"; then
echo '--speakers and --speaker-model cannot be combined with --no-diarization' >&2
exit 1
fi
if "$diarization" && [[ -z $speaker_model ]]; then
speaker_model=google/gemini-3.8-flash
fi
if [[ -z $chunk_minutes ]]; then
# Azure documents diarization failures for recordings around 15 minutes.
if "$diarization"; then chunk_minutes=10; else chunk_minutes=30; fi