transcribe/README.md

# Transcribe

```sh
./transcribe too_long.webm > transcript.txt
```

Speaker detection and matching across the recording happen automatically.
You do not need to know the number of speakers or choose a model.
Use `--srt` for subtitles or `--no-diarization` for text without speaker labels.

Requires Bash 5.1 or newer, FFmpeg, curl, jq, Python 3, and standard coreutils. Keep
`chunks.py`, `subtitles.py`, `transcript.py`, and `diarization.py` beside the script.
The API key is read using the 1Password CLI (`op`) from
`op://API Keys/openrouter-mai-transcribe-2/credential`.

Accepts m4a, mp4, aac, flac, opus, mp3, ogg, wav, and other formats your
FFmpeg supports. Uses the first audio track, converts it to mono 16 kHz MP3
at 64 kbps (requires FFmpeg's libmp3lame encoder),
and sends it to OpenRouter's `microsoft/mai-transcribe-2` model. Temporary
files are removed on exit. The transcript goes to stdout; progress and errors
go to stderr. Output is emitted only after all requests and conversion succeed.

Speaker diarization and voice comparison are **enabled by default**.
MAI first labels speech through
`provider.options.azure.diarization.enabled`, following
[OpenRouter's speaker diarization API](https://openrouter.ai/docs/guides/overview/multimodal/stt#verbose-transcripts-timestamps-and-speakers).
Gemini then compares voice excerpts through OpenRouter to match speakers across
the recording, automatically estimating their number. `--no-diarization`
disables both steps, with either plain text or `--srt` output. Text output prints
speaker IDs above each paragraph:

```text
[Speaker 0]
How did you get started?

[Speaker 1]
It was a long time ago.
```

Text wraps at 80 columns (except an unusually long unbroken word), with blank
lines at speaker changes and pauses of at least two seconds. Long turns split
at sentence endings after about 400 characters, or at 800 characters even
without punctuation. The formatter preserves the words and their order.
Unlabelled passages use `[Speaker unknown]` when other words have labels; a
response without any labels is printed without speaker markers. A warning on
stderr reports missing or incomplete labels when diarization was requested.
Speaker labels use audio evidence; they are not names inferred from text.
SRT uses labels to break cues at speaker changes but does not print the IDs.

Long recordings are automatically split into uploads of at most **10 minutes**
with diarization, or **30 minutes** with `--no-diarization`, including overlap
and MP3 padding. These defaults account for
[Microsoft's documented diarization failures around 15 minutes and longer](https://learn.microsoft.com/en-us/azure/ai-services/speech-service/mai-transcribe).
Use `--chunk-minutes N` (an integer from 1 to 30) to override the limit in either
mode. Larger chunks with diarization may fail with a timeout or
`diarization_unavailable`; reduce the chunk size or use `--no-diarization`.
At 64 kbps, a 10-minute upload is about 4.8 MB of MP3 or 6.4 MB of base64;
a 30-minute upload is about 14.4 MB of MP3 or 19.2 MB of base64.
These are local defaults, not guaranteed provider limits:
[OpenRouter documents a 60-second processing timeout](https://openrouter.ai/docs/guides/overview/multimodal/stt),
so reduce the chunk size if requests still time out.

For each boundary, the script looks for a pause of at least 0.3 seconds in the
last 30 seconds before the limit, using a -35 dBFS peak threshold. It joins in
the middle of the last suitable pause, or at the duration boundary if there is
no pause. Each upload includes three seconds beyond the join, so neighbouring
uploads share six seconds of audio even during continuous speech. The duration
cap includes this overlap; the next chunk starts before the previous one ends.

Requests with diarization and all chunked requests ask for word timestamps,
including for plain text. Short requests with `--no-diarization` retain the
provider's existing text line breaks and add wrapping to long lines.
The script matches words and their times in the overlap, switches transcripts
at a shared word, and keeps the repeated passage once. It prefers a matching
phrase to distinguish intentional repetitions. If no words match, it uses
timestamp ownership and warns when speech near the join needs review. Overlap
protects words at the audio cuts, but recognition errors or inaccurate model
timestamps can still require corrections. Missing or invalid word timestamps
are an error. Plain text for a chunked recording is assembled from the words.

SRT timestamps are shifted back to the original recording's timeline before
cue generation, with continuous cue numbering. Text uses `[Speaker 0]`,
`[Speaker 1]`, etc., without exposing upload boundaries to the reader.
Provider IDs are request-local. The automatic voice comparison maps them to
transcript-wide IDs using audio evidence, including for people absent from a
chunk overlap. Numeric provider IDs alone never establish identity across
requests.

Voice comparison uses `google/gemini-3.8-flash` by default and estimates the
number of speakers. If you already know the count, `--speakers N` (1 to 64)
supplies it as an optional constraint. `--speaker-model MODEL` overrides the
default with another audio chat model supporting multiple audio inputs and JSON
schema output. Neither flag is needed for the normal command. Both options
require diarization and cannot be combined with `--no-diarization`.

The comparison keeps MAI's words and timestamps, initially treating IDs from
different requests as separate. It takes up to three nonoverlapping excerpts
per provisional speaker (1.5 to 8 seconds each) from uninterrupted labelled
speech and sends them together in **one additional audio request**. Comparing
voices in a shared context can reunite a person absent from a chunk overlap
and merge IDs that MAI split within a single request. The model receives the
voice excerpts and sample numbers, without the written transcript or names.
This uses [OpenRouter audio chat](https://openrouter.ai/docs/guides/overview/multimodal/audio)
and [Gemini's documented audio diarization capability](https://ai.google.dev/gemini-api/docs/audio),
with no local model, new Python package, or separate API key.
Gemini requests use OpenRouter's Google AI Studio provider, whose API supports
[multiple audio inputs](https://ai.google.dev/gemini-api/docs/generate-content/audio#technical-details-about-audio),
with provider fallback disabled. The Vertex backend
[documents a one-audio-file limit](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/capabilities/audio-understanding).

Every returned sample assignment must be present exactly once. Excerpts for
the same provisional speaker must agree on one non-null model ID before it
can be merged with others. Speakers without suitable excerpts, uncertain
assignments, or conflicting voices retain separate IDs and produce warnings;
a known count does **not** force uncertain speakers into that many output labels.
If no suitable excerpts
exist, comparison is skipped with a warning. Malformed, incomplete, truncated,
or failed API responses fail the command without partial stdout.

This is sampled voice comparison, not a fresh segmentation of every word.
It cannot repair a MAI label that incorrectly combines several people into one
turn, and consistent model mistakes can still misidentify a voice. A single
usable excerpt has weaker evidence than several. Review brief interjections,
crosstalk, and edited intros in particular. Word timing and spoken text are
never generated by the comparison model. Requests are bounded to 256 excerpts
and 20 MB of encoded JSON; exceeding a bound fails rather than silently omitting
voices. The extra request adds processing time and provider charges. Its actual
voice-matching accuracy must be checked on real audio; fake-provider tests only
verify the pipeline and its validation rules.

Audio is first decoded to temporary mono 16 kHz PCM (about 115 MB per hour),
so all cuts share the same sample clock even when container duration metadata
is missing. Up to **eight chunks** are encoded and transcribed concurrently
by default. Use `--jobs N` (an integer from 1 to 64) to change the limit;
`--jobs 1` runs sequentially. Each worker uses separate temporary files and
releases its MP3 and request payload when finished, so staged upload data is
bounded by the number of workers. Lower `--jobs` to reduce simultaneous CPU,
memory, disk, or provider demand.

Responses may finish in any order; merging and speaker numbering always follow
the recording's original order. A failed worker stops further scheduling and
cancels the remaining workers. Interruptions also stop workers and their child
processes before temporary files are removed. Requests already accepted by the
provider may still be processed and billed. There is no automatic retry or
resume, and rerunning sends earlier chunks again. No transcript is emitted
unless every worker, the enabled voice comparison, and the final conversion
succeed.

`--srt` requests word timestamps using OpenRouter's `verbose_json` response.
Words are grouped into cues of at most two lines (42 characters per line,
except an unusually long unbroken word) and six seconds. Cues also break at
sentence endings, pauses of at least 0.7 seconds, and speaker changes when
speaker labels are available. Timing comes from the words, rounded to
milliseconds. If word start times move backwards, they are clamped to the
preceding word's start without reordering the transcript; end times are extended
to the adjusted start only when needed. Subtitle output goes to stdout; missing
or invalid word timestamps produce an error. Without `--srt`, output remains
plain text.

One input file per invocation. To run the local regression tests:

```sh
python3 -B -m unittest -v
```

[OpenRouter model and API reference](https://openrouter.ai/microsoft/mai-transcribe-2)