fix: reconcile speakers automatically by default
The plain transcribe command still used chunk overlap speaker matching, so users had to discover a model flag or supply a known speaker count to get the improved voice comparison behavior. Enable OpenRouter voice comparison with automatic speaker counting for normal text and SRT invocations. Keep --speakers and --speaker-model as optional overrides, and let --no-diarization disable both provider labels and voice comparison. Make the basic command the documentation entry point and clarify the defaults in CLI help and repository guidance. Cover bare invocations across one and multiple uploads, default SRT, optional count constraints, opt-out ordering, and atomic output when the default comparison fails. Test Plan: - python3 -B -m unittest -v: all 85 tests passed. - bash -n transcribe: passed. - git diff --cached --check: passed. - Tests use real audio encoding with fake provider responses. Live voice accuracy remains unverified; no live API request was made for this change.
This commit is contained in:
1 parent
138c796399
commit
d7827d39f5
5 files changed
+107
-48
No files matched your search
@@ -24,7 +24,7 @@
|
||||
- `chunks.py`: upload boundaries and ordered overlap stitching.
|
||||
- `subtitles.py`: timestamp validation and SRT cue formatting.
|
||||
- `transcript.py`: readable text paragraphs and speaker markers.
|
||||
- `diarization.py`: optional OpenRouter audio sample comparison and speaker ID reconciliation.
|
||||
- `diarization.py`: automatic OpenRouter audio sample comparison and speaker ID reconciliation.
|
||||
- `test_*.py` and `tests/fake_provider.py`: local regression tests without real
|
||||
credentials or provider requests.
|
||||
|
||||
@@ -36,3 +36,6 @@ must preserve transcription text and timestamps, validate complete sample
|
||||
assignments, and leave uncertain or conflicting identities separate. Keep chunk
|
||||
details out of the final transcript.
|
||||
Progress, processing details, and errors belong on stderr.
|
||||
The normal command detects the speaker count and reconciles voices automatically;
|
||||
`--speakers` and `--speaker-model` are optional overrides. `--no-diarization`
|
||||
disables both provider diarization and the voice comparison request.
|
||||
@@ -1,23 +1,18 @@
|
||||
# Transcribe
|
||||
|
||||
```sh
|
||||
./transcribe too_long.webm > transcript.txt
|
||||
```
|
||||
|
||||
Speaker detection and matching across the recording happen automatically.
|
||||
You do not need to know the number of speakers or choose a model.
|
||||
Use `--srt` for subtitles or `--no-diarization` for text without speaker labels.
|
||||
|
||||
Requires Bash 5.1 or newer, FFmpeg, curl, jq, Python 3, and standard coreutils. Keep
|
||||
`chunks.py`, `subtitles.py`, `transcript.py`, and `diarization.py` beside the script.
|
||||
The API key is read using the 1Password CLI (`op`) from
|
||||
`op://API Keys/openrouter-mai-transcribe-2/credential`.
|
||||
|
||||
```sh
|
||||
./transcribe recording.m4a
|
||||
./transcribe video.mp4 > transcript.txt
|
||||
./transcribe --srt video.mp4 > subtitles.srt
|
||||
./transcribe --no-diarization recording.m4a > transcript.txt
|
||||
./transcribe --srt --no-diarization video.mp4 > subtitles.srt
|
||||
./transcribe --chunk-minutes 10 too_long.webm > transcript.txt
|
||||
./transcribe --jobs 4 too_long.webm > transcript.txt
|
||||
./transcribe --jobs 1 --srt too_long.webm > subtitles.srt
|
||||
./transcribe --speakers 2 too_long.webm > transcript.txt
|
||||
./transcribe --speaker-model google/gemini-3.8-flash recording.m4a > transcript.txt
|
||||
```
|
||||
|
||||
Accepts m4a, mp4, aac, flac, opus, mp3, ogg, wav, and other formats your
|
||||
FFmpeg supports. Uses the first audio track, converts it to mono 16 kHz MP3
|
||||
at 64 kbps (requires FFmpeg's libmp3lame encoder),
|
||||
@@ -25,11 +20,14 @@ and sends it to OpenRouter's `microsoft/mai-transcribe-2` model. Temporary
|
||||
files are removed on exit. The transcript goes to stdout; progress and errors
|
||||
go to stderr. Output is emitted only after all requests and conversion succeed.
|
||||
|
||||
Speaker diarization is **enabled by default** through
|
||||
Speaker diarization and voice comparison are **enabled by default**.
|
||||
MAI first labels speech through
|
||||
`provider.options.azure.diarization.enabled`, following
|
||||
[OpenRouter's speaker diarization API](https://openrouter.ai/docs/guides/overview/multimodal/stt#verbose-transcripts-timestamps-and-speakers).
|
||||
Use `--no-diarization` to explicitly disable it for every request, with either
|
||||
plain text or `--srt` output. Text output prints returned IDs above each paragraph:
|
||||
Gemini then compares voice excerpts through OpenRouter to match speakers across
|
||||
the recording, automatically estimating their number. `--no-diarization`
|
||||
disables both steps, with either plain text or `--srt` output. Text output prints
|
||||
speaker IDs above each paragraph:
|
||||
|
||||
```text
|
||||
[Speaker 0]
|
||||
@@ -83,22 +81,17 @@ are an error. Plain text for a chunked recording is assembled from the words.
|
||||
SRT timestamps are shifted back to the original recording's timeline before
|
||||
cue generation, with continuous cue numbering. Text uses `[Speaker 0]`,
|
||||
`[Speaker 1]`, etc., without exposing upload boundaries to the reader.
|
||||
For a chunked recording, provider IDs are mapped to transcript-wide IDs:
|
||||
at least two unambiguous words shared at matching times in the overlap must
|
||||
agree on the speaker correspondence. Conflicting or absent evidence gets a
|
||||
new ID and a warning on stderr. This is not voice recognition across the whole
|
||||
recording: the same person can receive another ID when absent from an overlap
|
||||
or when diarization disagrees. Numeric provider IDs alone are never used to
|
||||
identify a speaker across requests.
|
||||
Provider IDs are request-local. The automatic voice comparison maps them to
|
||||
transcript-wide IDs using audio evidence, including for people absent from a
|
||||
chunk overlap. Numeric provider IDs alone never establish identity across
|
||||
requests.
|
||||
|
||||
For interviews and other recordings with fragmented speaker IDs, use
|
||||
`--speakers N` (1 to 64) to add a voice comparison pass through OpenRouter.
|
||||
This selects `google/gemini-3.8-flash` and tells it the known number of people.
|
||||
Use `--speaker-model MODEL` to select a different audio chat model supporting
|
||||
multiple audio inputs and JSON schema output; without `--speakers`, the model
|
||||
estimates the number of voices. Both options require
|
||||
diarization and cannot be combined with `--no-diarization`. The normal command
|
||||
without either option retains the overlap-based behavior described above.
|
||||
Voice comparison uses `google/gemini-3.8-flash` by default and estimates the
|
||||
number of speakers. If you already know the count, `--speakers N` (1 to 64)
|
||||
supplies it as an optional constraint. `--speaker-model MODEL` overrides the
|
||||
default with another audio chat model supporting multiple audio inputs and JSON
|
||||
schema output. Neither flag is needed for the normal command. Both options
|
||||
require diarization and cannot be combined with `--no-diarization`.
|
||||
|
||||
The comparison keeps MAI's words and timestamps, initially treating IDs from
|
||||
different requests as separate. It takes up to three nonoverlapping excerpts
|
||||
@@ -150,7 +143,7 @@ cancels the remaining workers. Interruptions also stop workers and their child
|
||||
processes before temporary files are removed. Requests already accepted by the
|
||||
provider may still be processed and billed. There is no automatic retry or
|
||||
resume, and rerunning sends earlier chunks again. No transcript is emitted
|
||||
unless every worker, any requested voice comparison, and the final conversion
|
||||
unless every worker, the enabled voice comparison, and the final conversion
|
||||
succeed.
|
||||
|
||||
`--srt` requests word timestamps using OpenRouter's `verbose_json` response.
|
||||
|
||||
+64
-4
@@ -53,6 +53,14 @@ class TranscribeTests(unittest.TestCase):
|
||||
def speaker_request(self):
|
||||
return json.loads(self.log.with_suffix(".speakers.json").read_text())
|
||||
|
||||
def assert_automatic_speaker_comparison(self):
|
||||
request = self.speaker_request()["request"]
|
||||
self.assertEqual(request["model"], "google/gemini-3.8-flash")
|
||||
schema = request["response_format"]["json_schema"]["schema"]
|
||||
speaker_schema = schema["properties"]["assignments"]["items"]["properties"]["speaker"]
|
||||
self.assertNotIn("maximum", speaker_schema)
|
||||
self.assertNotIn("complete recording has exactly", request["messages"][0]["content"])
|
||||
|
||||
def assert_processes_stopped(self):
|
||||
for event in self.events():
|
||||
for pid in (event["pid"], event.get("child_pid")):
|
||||
@@ -236,7 +244,8 @@ class TranscribeTests(unittest.TestCase):
|
||||
self.assertEqual(self.requests()[0]["response_format"], "verbose_json")
|
||||
self.assertEqual(self.requests()[0]["timestamp_granularities"], ["segment", "word"])
|
||||
self.assert_diarization(True)
|
||||
self.assertEqual(result.stderr, "")
|
||||
self.assertIn("no clean speaker samples", result.stderr)
|
||||
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
|
||||
self.assert_cleaned()
|
||||
|
||||
def test_chunked_text_has_speaker_markers_without_chunk_details(self):
|
||||
@@ -248,7 +257,8 @@ class TranscribeTests(unittest.TestCase):
|
||||
"[Speaker 1]\ncontinues.\n\n"
|
||||
"[Speaker 1]\nLast.\n"
|
||||
))
|
||||
self.assertIn("speaker continuity is uncertain", result.stderr)
|
||||
self.assertIn("no clean speaker samples", result.stderr)
|
||||
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
|
||||
self.assertNotIn("Chunk", result.stdout)
|
||||
self.assert_cleaned()
|
||||
|
||||
@@ -276,7 +286,7 @@ class TranscribeTests(unittest.TestCase):
|
||||
for output in ((), ("--srt",)):
|
||||
for chunked in (False, True):
|
||||
with self.subTest(output=output, chunked=chunked):
|
||||
self.log.unlink(missing_ok=True)
|
||||
self.reset_logs()
|
||||
options = ("--chunk-minutes", "1") if chunked else ()
|
||||
result = self.run_cli(
|
||||
*output, *options, "--no-diarization", duration=70 if chunked else 5,
|
||||
@@ -285,6 +295,7 @@ class TranscribeTests(unittest.TestCase):
|
||||
self.assertTrue(result.stdout)
|
||||
self.assertEqual(len(self.requests()), 2 if chunked else 1)
|
||||
self.assert_diarization(False)
|
||||
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
|
||||
for request in self.requests():
|
||||
if output or chunked:
|
||||
self.assertEqual(request["response_format"], "verbose_json")
|
||||
@@ -302,6 +313,22 @@ class TranscribeTests(unittest.TestCase):
|
||||
self.assert_diarization(False)
|
||||
self.assert_cleaned()
|
||||
|
||||
def test_no_diarization_skips_voice_comparison_with_usable_samples(self):
|
||||
# Do not compare voices even if a provider returns labels after opt-out.
|
||||
for output in ((), ("--srt",)):
|
||||
for options in (("--no-diarization", "--chunk-minutes", "1", "--jobs", "1"),
|
||||
("--chunk-minutes", "1", "--jobs", "2", "--no-diarization")):
|
||||
with self.subTest(output=output, options=options):
|
||||
self.reset_logs()
|
||||
result = self.run_cli(*output, *options, mode="reconcile")
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertIn("AlphaLate.", result.stdout)
|
||||
self.assertEqual(len(self.requests()), 2)
|
||||
self.assert_diarization(False)
|
||||
self.assertFalse(self.log.with_suffix(".speakers.json").exists())
|
||||
self.assertNotIn("Acoustic speaker matching", result.stderr)
|
||||
self.assert_cleaned()
|
||||
|
||||
def test_default_upload_limits_depend_on_diarization(self):
|
||||
for options, limit in (((), 600), (("--no-diarization",), 1800)):
|
||||
with self.subTest(options=options):
|
||||
@@ -356,12 +383,44 @@ class TranscribeTests(unittest.TestCase):
|
||||
self.assertNotIn("speaker continuity is uncertain", result.stderr)
|
||||
comparison = self.speaker_request()
|
||||
self.assertEqual(comparison["request"]["model"], "google/gemini-3.8-flash")
|
||||
schema = comparison["request"]["response_format"]["json_schema"]["schema"]
|
||||
self.assertEqual(schema["properties"]["assignments"]["items"]["properties"]["speaker"]["maximum"], 1)
|
||||
self.assertIn("exactly 2 speakers", comparison["request"]["messages"][0]["content"])
|
||||
self.assertEqual(len(comparison["durations"]), 5)
|
||||
self.assertTrue(all(1.5 <= duration <= 8.1 for duration in comparison["durations"]))
|
||||
self.assertEqual(len(self.requests()), 2)
|
||||
self.assert_diarization(True)
|
||||
self.assert_cleaned()
|
||||
|
||||
def test_bare_command_compares_voices_without_a_known_speaker_count(self):
|
||||
for duration in (30, 611):
|
||||
with self.subTest(duration=duration):
|
||||
self.reset_logs()
|
||||
result = self.run_cli(mode="reconcile", duration=duration)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
expected = "[Speaker 0]\nAlpha.\n\n[Speaker 0]\nAgain.\n\n[Speaker 1]\nBeta.\n"
|
||||
if duration > 600:
|
||||
expected += "\n[Speaker 0]\nAlphaLate.\n\n[Speaker 1]\nBetaLate.\n"
|
||||
self.assertEqual(result.stdout, expected)
|
||||
self.assert_automatic_speaker_comparison()
|
||||
self.assertEqual(len(self.requests()), 2 if duration > 600 else 1)
|
||||
self.assertEqual(len(self.speaker_request()["samples"]), 5 if duration > 600 else 3)
|
||||
self.assert_diarization(True)
|
||||
self.assert_cleaned()
|
||||
|
||||
def test_default_srt_compares_voices_without_speaker_options(self):
|
||||
result = self.run_cli("--srt", mode="reconcile", duration=30)
|
||||
self.assertEqual(result.returncode, 0, result.stderr)
|
||||
self.assertEqual(result.stdout, (
|
||||
"1\n00:00:01,000 --> 00:00:03,000\nAlpha.\n\n"
|
||||
"2\n00:00:06,000 --> 00:00:08,000\nAgain.\n\n"
|
||||
"3\n00:00:12,000 --> 00:00:14,000\nBeta.\n\n"
|
||||
))
|
||||
self.assert_automatic_speaker_comparison()
|
||||
self.assertEqual(len(self.requests()), 1)
|
||||
self.assert_diarization(True)
|
||||
self.assert_cleaned()
|
||||
|
||||
def test_voice_comparison_model_override_without_count_and_single_request(self):
|
||||
for model, count in (("test/audio-model", ()), ("~audio-model", ("--speakers", "2"))):
|
||||
with self.subTest(model=model):
|
||||
@@ -411,7 +470,7 @@ class TranscribeTests(unittest.TestCase):
|
||||
"reconcile-truncated"):
|
||||
with self.subTest(mode=mode):
|
||||
self.reset_logs()
|
||||
result = self.run_cli("--speakers", "2", "--chunk-minutes", "1", mode=mode)
|
||||
result = self.run_cli("--chunk-minutes", "1", mode=mode)
|
||||
self.assertNotEqual(result.returncode, 0)
|
||||
self.assertEqual(result.stdout, "")
|
||||
self.assertTrue(self.log.with_suffix(".speakers.json").exists())
|
||||
@@ -438,6 +497,7 @@ class TranscribeTests(unittest.TestCase):
|
||||
options += [("--speaker-model", value) for value in ("", "-bad", "bad model", "a\nb")]
|
||||
options += [("--no-diarization", "--speakers", "2"),
|
||||
("--speakers", "2", "--no-diarization"),
|
||||
("--no-diarization", "--speaker-model", "test/audio"),
|
||||
("--speaker-model", "test/audio", "--no-diarization")]
|
||||
for args in options:
|
||||
with self.subTest(options=args):
|
||||
|
||||
@@ -35,6 +35,8 @@ def event(kind, **extra):
|
||||
event("start")
|
||||
if is_comparison:
|
||||
samples = json.loads((output.parent / "samples.json").read_text())
|
||||
manifest = json.loads((staging / "chunks.json").read_text())
|
||||
late_start = round(manifest[0]["end"]) if len(manifest) > 1 else 60
|
||||
content = request["messages"][-1]["content"]
|
||||
audio_parts = [part["input_audio"] for part in content if part["type"] == "input_audio"]
|
||||
assert len(content) == 2 * len(samples) == 2 * len(audio_parts)
|
||||
@@ -73,7 +75,7 @@ if is_comparison:
|
||||
if mode == "reconcile-bad-json":
|
||||
output.write_text("not json")
|
||||
sys.exit(0)
|
||||
assignments = [{"sample": sample["id"], "speaker": 0 if round(sample["start"]) in (1, 6, 60) else 1}
|
||||
assignments = [{"sample": sample["id"], "speaker": 0 if round(sample["start"]) in (1, 6, late_start) else 1}
|
||||
for sample in samples]
|
||||
if mode == "reconcile-missing":
|
||||
assignments.pop()
|
||||
@@ -182,12 +184,13 @@ else:
|
||||
if mode.startswith("reconcile"):
|
||||
# Two labels already describe voice A inside the first request; later
|
||||
# chunks reuse the same local numbers for different request identities.
|
||||
late_start = round(manifest[0]["end"]) if len(manifest) > 1 else 60
|
||||
words = [
|
||||
{"word": text, "start": start - chunk["start"], "end": start + 2 - chunk["start"],
|
||||
"speaker": speaker}
|
||||
for text, start, speaker in [
|
||||
("Alpha.", 1, 7), ("Again.", 6, 8), ("Beta.", 12, 9),
|
||||
("AlphaLate.", 60, 9), ("BetaLate.", 65, 7),
|
||||
("AlphaLate.", late_start, 9), ("BetaLate.", late_start + 5, 7),
|
||||
] if start >= chunk["start"] and start + 2 <= chunk["end"]
|
||||
]
|
||||
response = {"text": " ".join(w["word"] for w in words), "words": words}
|
||||
|
||||
+9
-9
@@ -10,12 +10,12 @@ speaker_model=
|
||||
speakers=
|
||||
usage() {
|
||||
echo "Usage: ${0##*/} [--srt] [--no-diarization] [--speakers N] [--speaker-model MODEL] [--chunk-minutes 1..30] [--jobs 1..64] AUDIO_FILE"
|
||||
echo 'Transcribe audio using OPENROUTER_API_KEY; print wrapped text with speaker markers to stdout.'
|
||||
echo 'Transcribe audio with automatic speaker detection; print wrapped text to stdout.'
|
||||
echo 'Speakers are matched across the recording automatically through OpenRouter.'
|
||||
echo ' --srt Print SRT subtitles in short cues using word timestamps instead.'
|
||||
echo ' --no-diarization Disable speaker diarization (enabled by default).'
|
||||
echo ' --speakers N Reconcile voices through OpenRouter with a known count (1..64).'
|
||||
echo ' --speaker-model MODEL Reconcile voices with an audio chat model (count optional).'
|
||||
echo ' --speakers defaults to google/gemini-3.8-flash; adds one audio request.'
|
||||
echo ' --no-diarization Disable speaker detection and voice comparison.'
|
||||
echo ' --speakers N Optional known speaker count (1..64; otherwise detected automatically).'
|
||||
echo ' --speaker-model MODEL Override the voice comparison model (default: google/gemini-3.8-flash).'
|
||||
echo ' --chunk-minutes N Maximum upload duration in minutes (default: 10; 30 with --no-diarization).'
|
||||
echo ' --jobs N Maximum chunks processed in parallel (default: 8; 1 for sequential).'
|
||||
}
|
||||
@@ -61,13 +61,13 @@ while [[ $# -gt 0 ]]; do
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
if [[ -n $speakers && -z $speaker_model ]]; then
|
||||
speaker_model=google/gemini-3.8-flash
|
||||
fi
|
||||
if [[ -n $speaker_model ]] && ! "$diarization"; then
|
||||
if [[ -n $speaker_model || -n $speakers ]] && ! "$diarization"; then
|
||||
echo '--speakers and --speaker-model cannot be combined with --no-diarization' >&2
|
||||
exit 1
|
||||
fi
|
||||
if "$diarization" && [[ -z $speaker_model ]]; then
|
||||
speaker_model=google/gemini-3.8-flash
|
||||
fi
|
||||
if [[ -z $chunk_minutes ]]; then
|
||||
# Azure documents diarization failures for recordings around 15 minutes.
|
||||
if "$diarization"; then chunk_minutes=10; else chunk_minutes=30; fi
|
||||
|
||||
Reference in new issue
Block a user