ddidderr 138c796399 feat: reconcile speaker identities through OpenRouter audio
Short chunk overlaps often contain only one participant, so returning
speakers receive new transcript IDs. MAI can also split one voice into
several IDs within a single request, which overlap stitching cannot fix.

Add an opt-in audio comparison pass: --speakers N selects Gemini 3.8 Flash
with a known count, and --speaker-model allows another compatible model.
Keep request-local identities separate, select up to three clean excerpts
per ID, and compare all excerpts in one OpenRouter request. Gemini uses
Google AI Studio because the Vertex backend limits prompts to one audio
file. No local model or extra Python dependency is required.

Only reconcile identities whose sample assignments agree. Preserve
uncertain identities separately, reject incomplete or malformed replies,
and retain the original words and timestamps. Bound sample count and
request size, clip sampling to the real audio duration, and extend worker
cleanup and buffered output to the comparison step. Document the sampled
method's limitations and additional provider request.

Test Plan:
- python3 -B -m unittest -v: all 82 tests passed, including real MP3
  encoding with fake transcription/comparison responses and cancellation.
- bash -n transcribe: passed.
- git diff --cached --check: passed.
- Live API and voice accuracy unverified: this workspace could not access
  the configured 1Password credential. Mocked responses prove pipeline
  behavior, not model accuracy on the supplied recording.
2026-09-12 10:04:56 +02:00
2026-09-12 08:26:16 +02:00

Transcribe

Requires Bash 5.1 or newer, FFmpeg, curl, jq, Python 3, and standard coreutils. Keep chunks.py, subtitles.py, transcript.py, and diarization.py beside the script. The API key is read using the 1Password CLI (op) from op://API Keys/openrouter-mai-transcribe-2/credential.

./transcribe recording.m4a
./transcribe video.mp4 > transcript.txt
./transcribe --srt video.mp4 > subtitles.srt
./transcribe --no-diarization recording.m4a > transcript.txt
./transcribe --srt --no-diarization video.mp4 > subtitles.srt
./transcribe --chunk-minutes 10 too_long.webm > transcript.txt
./transcribe --jobs 4 too_long.webm > transcript.txt
./transcribe --jobs 1 --srt too_long.webm > subtitles.srt
./transcribe --speakers 2 too_long.webm > transcript.txt
./transcribe --speaker-model google/gemini-3.8-flash recording.m4a > transcript.txt

Accepts m4a, mp4, aac, flac, opus, mp3, ogg, wav, and other formats your FFmpeg supports. Uses the first audio track, converts it to mono 16 kHz MP3 at 64 kbps (requires FFmpeg's libmp3lame encoder), and sends it to OpenRouter's microsoft/mai-transcribe-2 model. Temporary files are removed on exit. The transcript goes to stdout; progress and errors go to stderr. Output is emitted only after all requests and conversion succeed.

Speaker diarization is enabled by default through provider.options.azure.diarization.enabled, following OpenRouter's speaker diarization API. Use --no-diarization to explicitly disable it for every request, with either plain text or --srt output. Text output prints returned IDs above each paragraph:

[Speaker 0]
How did you get started?

[Speaker 1]
It was a long time ago.

Text wraps at 80 columns (except an unusually long unbroken word), with blank lines at speaker changes and pauses of at least two seconds. Long turns split at sentence endings after about 400 characters, or at 800 characters even without punctuation. The formatter preserves the words and their order. Unlabelled passages use [Speaker unknown] when other words have labels; a response without any labels is printed without speaker markers. A warning on stderr reports missing or incomplete labels when diarization was requested. Speaker labels use audio evidence; they are not names inferred from text. SRT uses labels to break cues at speaker changes but does not print the IDs.

Long recordings are automatically split into uploads of at most 10 minutes with diarization, or 30 minutes with --no-diarization, including overlap and MP3 padding. These defaults account for Microsoft's documented diarization failures around 15 minutes and longer. Use --chunk-minutes N (an integer from 1 to 30) to override the limit in either mode. Larger chunks with diarization may fail with a timeout or diarization_unavailable; reduce the chunk size or use --no-diarization. At 64 kbps, a 10-minute upload is about 4.8 MB of MP3 or 6.4 MB of base64; a 30-minute upload is about 14.4 MB of MP3 or 19.2 MB of base64. These are local defaults, not guaranteed provider limits: OpenRouter documents a 60-second processing timeout, so reduce the chunk size if requests still time out.

For each boundary, the script looks for a pause of at least 0.3 seconds in the last 30 seconds before the limit, using a -35 dBFS peak threshold. It joins in the middle of the last suitable pause, or at the duration boundary if there is no pause. Each upload includes three seconds beyond the join, so neighbouring uploads share six seconds of audio even during continuous speech. The duration cap includes this overlap; the next chunk starts before the previous one ends.

Requests with diarization and all chunked requests ask for word timestamps, including for plain text. Short requests with --no-diarization retain the provider's existing text line breaks and add wrapping to long lines. The script matches words and their times in the overlap, switches transcripts at a shared word, and keeps the repeated passage once. It prefers a matching phrase to distinguish intentional repetitions. If no words match, it uses timestamp ownership and warns when speech near the join needs review. Overlap protects words at the audio cuts, but recognition errors or inaccurate model timestamps can still require corrections. Missing or invalid word timestamps are an error. Plain text for a chunked recording is assembled from the words.

SRT timestamps are shifted back to the original recording's timeline before cue generation, with continuous cue numbering. Text uses [Speaker 0], [Speaker 1], etc., without exposing upload boundaries to the reader. For a chunked recording, provider IDs are mapped to transcript-wide IDs: at least two unambiguous words shared at matching times in the overlap must agree on the speaker correspondence. Conflicting or absent evidence gets a new ID and a warning on stderr. This is not voice recognition across the whole recording: the same person can receive another ID when absent from an overlap or when diarization disagrees. Numeric provider IDs alone are never used to identify a speaker across requests.

For interviews and other recordings with fragmented speaker IDs, use --speakers N (1 to 64) to add a voice comparison pass through OpenRouter. This selects google/gemini-3.8-flash and tells it the known number of people. Use --speaker-model MODEL to select a different audio chat model supporting multiple audio inputs and JSON schema output; without --speakers, the model estimates the number of voices. Both options require diarization and cannot be combined with --no-diarization. The normal command without either option retains the overlap-based behavior described above.

The comparison keeps MAI's words and timestamps, initially treating IDs from different requests as separate. It takes up to three nonoverlapping excerpts per provisional speaker (1.5 to 8 seconds each) from uninterrupted labelled speech and sends them together in one additional audio request. Comparing voices in a shared context can reunite a person absent from a chunk overlap and merge IDs that MAI split within a single request. The model receives the voice excerpts and sample numbers, without the written transcript or names. This uses OpenRouter audio chat and Gemini's documented audio diarization capability, with no local model, new Python package, or separate API key. Gemini requests use OpenRouter's Google AI Studio provider, whose API supports multiple audio inputs, with provider fallback disabled. The Vertex backend documents a one-audio-file limit.

Every returned sample assignment must be present exactly once. Excerpts for the same provisional speaker must agree on one non-null model ID before it can be merged with others. Speakers without suitable excerpts, uncertain assignments, or conflicting voices retain separate IDs and produce warnings; a known count does not force uncertain speakers into that many output labels. If no suitable excerpts exist, comparison is skipped with a warning. Malformed, incomplete, truncated, or failed API responses fail the command without partial stdout.

This is sampled voice comparison, not a fresh segmentation of every word. It cannot repair a MAI label that incorrectly combines several people into one turn, and consistent model mistakes can still misidentify a voice. A single usable excerpt has weaker evidence than several. Review brief interjections, crosstalk, and edited intros in particular. Word timing and spoken text are never generated by the comparison model. Requests are bounded to 256 excerpts and 20 MB of encoded JSON; exceeding a bound fails rather than silently omitting voices. The extra request adds processing time and provider charges. Its actual voice-matching accuracy must be checked on real audio; fake-provider tests only verify the pipeline and its validation rules.

Audio is first decoded to temporary mono 16 kHz PCM (about 115 MB per hour), so all cuts share the same sample clock even when container duration metadata is missing. Up to eight chunks are encoded and transcribed concurrently by default. Use --jobs N (an integer from 1 to 64) to change the limit; --jobs 1 runs sequentially. Each worker uses separate temporary files and releases its MP3 and request payload when finished, so staged upload data is bounded by the number of workers. Lower --jobs to reduce simultaneous CPU, memory, disk, or provider demand.

Responses may finish in any order; merging and speaker numbering always follow the recording's original order. A failed worker stops further scheduling and cancels the remaining workers. Interruptions also stop workers and their child processes before temporary files are removed. Requests already accepted by the provider may still be processed and billed. There is no automatic retry or resume, and rerunning sends earlier chunks again. No transcript is emitted unless every worker, any requested voice comparison, and the final conversion succeed.

--srt requests word timestamps using OpenRouter's verbose_json response. Words are grouped into cues of at most two lines (42 characters per line, except an unusually long unbroken word) and six seconds. Cues also break at sentence endings, pauses of at least 0.7 seconds, and speaker changes when speaker labels are available. Timing comes from the words, rounded to milliseconds. If word start times move backwards, they are clamped to the preceding word's start without reordering the transcript; end times are extended to the adjusted start only when needed. Subtitle output goes to stdout; missing or invalid word timestamps produce an error. Without --srt, output remains plain text.

One input file per invocation. To run the local regression tests:

python3 -B -m unittest -v

OpenRouter model and API reference

S
Description
No description provided
Readme
272 KiB
0 Stars 1 Watchers 0 Forks
Languages
Python 91.7%
Shell 8.3%