feat: transcribe chunks with configurable parallel workers
Process up to eight chunks concurrently by default and add --jobs 1..64, with --jobs 1 retaining sequential operation. Give each worker separate encoding and request files, release payloads after completion, and merge responses by original chunk index regardless of completion order. Use a bounded worker scheduler that handles jobs finishing before wait is called. Stop scheduling on an observed worker failure and terminate worker process groups on failure or interruption before removing temporary files. Include jobs interrupted between launch and scheduler registration in cleanup. Preserve all-or-nothing stdout for both text and SRT output. Document the Bash 5.1 requirement, concurrency option, resource usage, and limits of cancelling requests already accepted by a provider. Test Plan: - python3 -B -m unittest -v: all 52 tests passed. - Mocked provider tests verified eight concurrent requests by default, configurable limits, rolling worker replacement, isolated payloads, ordered text, and identical sequential/parallel SRT output. - Verified immediate encoder failure, concurrent API failure, SIGINT and SIGTERM, child-process cancellation, cleanup, and no partial stdout. - bash -n transcribe, ./transcribe --help, and git diff --cached --check: passed. - No live provider request; credentials are unavailable in this environment.
This commit is contained in:
1 parent
5dd1b5811b
commit
1d03dcd587
4 files changed
+298
-18
No files matched your search
@@ -1,6 +1,6 @@
|
||||
# Transcribe
|
||||
|
||||
Requires Bash, FFmpeg, curl, jq, Python 3, and standard coreutils. Keep
|
||||
Requires Bash 5.1 or newer, FFmpeg, curl, jq, Python 3, and standard coreutils. Keep
|
||||
`chunks.py`, `subtitles.py`, and `transcript.py` beside the script. The API key is read using
|
||||
the 1Password CLI (`op`) from `op://API Keys/openrouter-mai-transcribe-2/credential`.
|
||||
|
||||
@@ -11,6 +11,8 @@ the 1Password CLI (`op`) from `op://API Keys/openrouter-mai-transcribe-2/credent
|
||||
./transcribe --no-diarization recording.m4a > transcript.txt
|
||||
./transcribe --srt --no-diarization video.mp4 > subtitles.srt
|
||||
./transcribe --chunk-minutes 10 too_long.webm > transcript.txt
|
||||
./transcribe --jobs 4 too_long.webm > transcript.txt
|
||||
./transcribe --jobs 1 --srt too_long.webm > subtitles.srt
|
||||
```
|
||||
|
||||
Accepts m4a, mp4, aac, flac, opus, mp3, ogg, wav, and other formats your
|
||||
@@ -88,9 +90,20 @@ identify a speaker across requests.
|
||||
|
||||
Audio is first decoded to temporary mono 16 kHz PCM (about 115 MB per hour),
|
||||
so all cuts share the same sample clock even when container duration metadata
|
||||
is missing. Only one MP3 chunk and its request payload are staged at a time.
|
||||
Requests run sequentially. A failed request stops the run; there is no automatic
|
||||
retry or resume, and rerunning sends the earlier chunks again.
|
||||
is missing. Up to **eight chunks** are encoded and transcribed concurrently
|
||||
by default. Use `--jobs N` (an integer from 1 to 64) to change the limit;
|
||||
`--jobs 1` runs sequentially. Each worker uses separate temporary files and
|
||||
releases its MP3 and request payload when finished, so staged upload data is
|
||||
bounded by the number of workers. Lower `--jobs` to reduce simultaneous CPU,
|
||||
memory, disk, or provider demand.
|
||||
|
||||
Responses may finish in any order; merging and speaker numbering always follow
|
||||
the recording's original order. A failed worker stops further scheduling and
|
||||
cancels the remaining workers. Interruptions also stop workers and their child
|
||||
processes before temporary files are removed. Requests already accepted by the
|
||||
provider may still be processed and billed. There is no automatic retry or
|
||||
resume, and rerunning sends earlier chunks again. No transcript is emitted
|
||||
unless every worker and the final conversion succeed.
|
||||
|
||||
`--srt` requests word timestamps using OpenRouter's `verbose_json` response.
|
||||
Words are grouped into cues of at most two lines (42 characters per line,
|
||||
|
||||
Reference in new issue
Block a user