Dubbings

Generate a dubbed video from an existing transcript and target-language segments. Create the work from the Jobs API with serviceType: dubbing, then read generated artifacts and speaker metadata from the Dubbing endpoints.

Dubbing reuses existing Perchat Media transcription, translation, voice, storage, and worker patterns. It does not ask clients to upload transcript segments in the job request. For translated dubbing, create or verify the target-language Translation record first, then queue the dubbing job with the same transcriptionId and targetLanguage.

Endpoints

Dubbing flow

The worker performs the pipeline in reusable stages:

  1. Fetch the transcription by transcriptionId.
  2. Resolve the target-language Translation record by transcription and language.
  3. Prepare source media and artifact keys.
  4. Extract raw audio and a video-only file.
  5. Separate vocals and background/no-vocals audio.
  6. Run diarization and assign speakers to dubbing segments.
  7. Extract speaker samples and assign/create voice profiles.
  8. Generate TTS for translated segments.
  9. Assemble translated speech on the original timeline.
  10. Mix translated speech with background audio.
  11. Mux the final audio with the video-only file.

Common scenarios

  • Existing translation — recommended path. Create translation first, then queue dubbing. The worker uses the stored Translation segments.
  • Manual voice assignment — pass options.voiceAssignments to map SPEAKER_00, SPEAKER_01, etc. to existing Perchat voice IDs. This is the most controlled first test path.
  • Auto voice creation — when no manual voice assignment is provided, the worker extracts speaker samples and calls the existing Perchat Voice service to create or assign voices when possible. This internal dubbing voice creation does not consume the public voice_creation allowance used by /v1/voices.
  • No fallback voice - if no manual, source-transcript, or auto-created voice is available, the worker marks the speaker for manual assignment and skips TTS for affected segments instead of using a default voice.
  • Missing translation — the Dubbing record reports translationStatus: missing. For translated output, create the Translation record first or enable the product flow that creates translation before dubbing.

The Dubbing object

dubbingIdstring
Stable dubbing identifier (dub_…).
jobIdstring
Dubbing job that generated this record.
workspaceIdstring
Owning workspace.
projectIdstring
Owning project.
transcriptionIdstring
Source transcription used for dubbing.
transcriptionJobIdstring | null, optional
Original transcription job id when available.
translationIdstring | null, optional
Translation record used for target-language dubbing when available.
sourceLanguagestring
Source language code used for the dubbing flow.
targetLanguagestring
Requested target language code, for example ta, es, ja, or en-US.
languageKeystring
Normalized target language key used for lookup and storage.
sourceLayerstring
Transcript layer used for source text: current, glossary, or raw.
statusstring
Dubbing record status: prepared, processing, completed, or failed.
translationStatusstring
Whether translated segments were found, missing, or generated.
textPreviewstring
Short preview of the text used for dubbing.
segmentCountinteger
Number of dubbing segments.
durationSecnumber | null, optional
Approximate media or segment timeline duration.
sourceMediaobject | null, optional
Resolved source media details from the original transcription/job input.
sourceMedia.sourceTypestring | null
Source type, such as s3_direct.
sourceMedia.connectionIdstring | null
Storage connection used for source media.
sourceMedia.bucketstring | null
Source media bucket.
sourceMedia.sourceKeystring | null
Source media object key.
sourceMedia.downloadableboolean
Whether the worker resolved the source as downloadable.
artifactPrefixstring | null, optional
Base output prefix for generated dubbing artifacts.
artifactsobject | null, optional
Generated media artifacts for the dubbing pipeline.
artifacts.rawVideoobject | null
Original source video reference.
artifacts.rawAudioobject | null
Extracted raw audio artifact.
artifacts.videoWithoutAudioobject | null
Video-only artifact used for final muxing.
artifacts.vocalsAudioobject | null
Separated vocals/speech artifact.
artifacts.backgroundAudioobject | null
Separated background/no-vocals artifact.
artifacts.translatedSpeechAudioobject | null
Timeline-aligned translated speech-only audio.
artifacts.mixedFinalAudioobject | null
Final audio mix of translated speech and background.
artifacts.dubbedVideoobject | null
Final dubbed video artifact.
artifacts.*.bucketstring | null
Artifact bucket.
artifacts.*.keystring | null
Artifact object key.
artifacts.*.urlstring | null
Artifact URI or URL.
artifacts.*.statusstring | null
Artifact status: planned, available, or failed.
artifacts.*.contentTypestring | null
Artifact MIME type.
artifacts.*.sizeBytesnumber | null
Artifact size when known.
artifacts.*.durationSecnumber | null
Artifact duration when known.
speakersarray
Detected speakers and voice assignment details.
speakers[].speakerstring
Diarization speaker label, for example SPEAKER_00.
speakers[].sampleAudioobject | null
Extracted sample audio for voice cloning/assignment.
speakers[].voiceProviderstring | null
Voice provider used for this speaker, usually elevenlabs.
speakers[].voiceIdstring | null
Perchat voice ID assigned to this speaker.
speakers[].voiceNamestring | null
Display name for the assigned voice.
speakers[].voiceSourcestring | null
How the voice was assigned: manual, auto_created, auto_failed, or transcription_segment.
speakers[].voiceStatusstring | null
Voice assignment status: pending, assigned, manual_assignment_required, or failed.
speakers[].sampleQualitystring | null
Basic sample quality status: good, insufficient, or failed.
speakers[].sampleDurationSecnumber | null
Extracted speaker sample duration.
speakers[].cleanTurnCountinteger | null
Number of clean diarization turns used for the sample.
diarizationobject | null, optional
Speaker diarization metadata and speaker turns.
speakerSamplesobject | null, optional
Speaker sample extraction summary.
voiceAssignmentsobject | null, optional
Voice assignment summary and counts.
ttsobject | null, optional
TTS generation summary and provider metadata.
speechAssemblyobject | null, optional
Translated speech assembly summary.
finalAudioMixobject | null, optional
Final audio mix summary.
finalVideoMuxobject | null, optional
Final video mux summary.
segmentsarray
Dubbing segments used for TTS and timeline assembly.
segments[].idstring
Segment identifier, usually copied from the source transcript/translation segment.
segments[].startnumber
Segment start time in seconds.
segments[].endnumber
Segment end time in seconds.
segments[].textstring
Text synthesized for this segment, usually the translated text.
segments[].sourceTextstring | null
Original source text when available.
segments[].translatedTextstring | null
Translated text returned by the Translation service when available.
segments[].speakerstring | null
Speaker label assigned by diarization.
segments[].voiceIdstring | null
Perchat voice ID used for TTS on this segment.
segments[].ttsStatusstring | null
TTS status for this segment, for example available, skipped, or failed.
segments[].ttsReasonstring | null, optional
Reason for a skipped or failed TTS segment, for example speaker_voice_not_assigned.
segments[].audioobject | null
Generated per-segment TTS audio artifact.
segments[].wordsarray, optional
Optional word-level timing copied from source data when available.
stepsobject | null, optional
High-level dubbing pipeline step status map.
vendorUsageobject | null, optional
Provider usage metadata if available.
createdAttimestamp
Record creation time.
updatedAttimestamp
Last update time.

List dubbings for a transcript

GET/v1/transcriptions/{transcriptionId}/dubbings
Returns Dubbing records generated from one transcription.

Requires permission

transcript:read

Parameters

transcriptionIdstring
Transcript id.

Returns

Array of Dubbing records.

See the the dubbing object for attribute details.

Retrieve a Dubbing record

GET/v1/dubbings/{dubbingId}
Returns the full stored Dubbing record, including artifacts, speakers, and generated output references.

Requires permission

transcript:read

Parameters

dubbingIdstring
Dubbing id.

Returns

Full Dubbing record.

See the the dubbing object for attribute details.