feat: reconcile speaker identities through OpenRouter audio

Short chunk overlaps often contain only one participant, so returning
speakers receive new transcript IDs. MAI can also split one voice into
several IDs within a single request, which overlap stitching cannot fix.

Add an opt-in audio comparison pass: --speakers N selects Gemini 3.8 Flash
with a known count, and --speaker-model allows another compatible model.
Keep request-local identities separate, select up to three clean excerpts
per ID, and compare all excerpts in one OpenRouter request. Gemini uses
Google AI Studio because the Vertex backend limits prompts to one audio
file. No local model or extra Python dependency is required.

Only reconcile identities whose sample assignments agree. Preserve
uncertain identities separately, reject incomplete or malformed replies,
and retain the original words and timestamps. Bound sample count and
request size, clip sampling to the real audio duration, and extend worker
cleanup and buffered output to the comparison step. Document the sampled
method's limitations and additional provider request.

Test Plan:
- python3 -B -m unittest -v: all 82 tests passed, including real MP3
  encoding with fake transcription/comparison responses and cancellation.
- bash -n transcribe: passed.
- git diff --cached --check: passed.
- Live API and voice accuracy unverified: this workspace could not access
  the configured 1Password credential. Mocked responses prove pipeline
  behavior, not model accuracy on the supplied recording.
This commit is contained in:
ddidderr committed 2026-09-12 10:04:56 +02:00
1 parent 1d03dcd587
commit 138c796399
9 files changed
+988 -19

No files matched your search

+14 -5
View File
@@ -149,7 +149,8 @@ def speaker_mapping(left, right, seam):
if count >= 2 and len(candidates[new]) == len(reverse[old]) == 1}
def merge(chunks, responses):
def merge(chunks, responses, scope_speakers=False):
"""Stitch words, optionally leaving speaker identity to an audio pass."""
if len(chunks) != len(responses) or not chunks:
raise ValueError("Expected one transcription response per chunk")
merged = []
@@ -163,7 +164,7 @@ def merge(chunks, responses):
raise ValueError("Word timestamp exceeds the chunk duration")
word["start"] += chunk["start"]
word["end"] += chunk["start"]
mapping = speaker_mapping(previous, words, chunk["seam"])
mapping = {} if scope_speakers else speaker_mapping(previous, words, chunk["seam"])
unmatched = False
for word in words:
speaker = word.get("speaker")
@@ -173,7 +174,7 @@ def merge(chunks, responses):
next_speaker += 1
unmatched = True
word["speaker"] = mapping[speaker]
if index and unmatched:
if index and unmatched and not scope_speakers:
print(
f"Warning: speaker continuity is uncertain near {chunk['seam']:.2f}s; "
"unmatched speakers received new transcript IDs.", file=sys.stderr,
@@ -201,6 +202,9 @@ def main():
merger = commands.add_parser("merge")
merger.add_argument("manifest", type=Path)
merger.add_argument("--srt", action="store_true")
merger.add_argument("--json", action="store_true", help="Emit stitched words for further processing")
merger.add_argument("--scope-speakers", action="store_true",
help="Keep request-local speakers separate for subsequent audio comparison")
args = parser.parse_args()
try:
if args.command == "plan":
@@ -210,8 +214,13 @@ def main():
responses = [json.loads(
(args.manifest.parent / f"response-{i}.json").read_text(encoding="utf-8")
) for i in range(len(chunks))]
response = merge(chunks, responses)
sys.stdout.write(convert(response) if args.srt else render_text(response))
if args.json and args.srt or args.scope_speakers and not args.json:
raise ValueError("Use --scope-speakers with --json, and --json without --srt")
response = merge(chunks, responses, scope_speakers=args.scope_speakers)
if args.json:
print(json.dumps(response))
else:
sys.stdout.write(convert(response) if args.srt else render_text(response))
except (ValueError, OSError, wave.Error) as error:
parser.exit(1, f"Audio chunking failed: {error}\n")