Transcription Automation Comprehensive skill for automating audio/video transcription and content processing. Core Workflows 1. Transcription Pipeline TRANSCRIPTION FLOW: ┌─────────────────┐ │ Audio/Video │ │ Input │ └────────┬────────┘ ▼ ┌─────────────────┐ │ Pre-Processing │ │ - Convert │ │ - Enhance │ │ - Split │ └────────┬────────┘ ▼ ┌─────────────────┐ │ Transcription │ │ - STT Engine │ │ - Diarization │ └────────┬────────┘ ▼ ┌─────────────────┐ │ Post-Processing │ │ - Format │ │ - Timestamps │ │ - Speakers │ └────────┬────────┘ ▼ ┌─────────────────┐ │ Output │ │ - Text/SRT/VTT │ │ - Summary │ └─────────────────┘ 2. Transcription Configuration transcription_config : engine : whisper

whisper, assembly_ai, deepgram

audio_settings : sample_rate : 16000 channels : mono format : wav transcription : language : auto

or specific: en, zh, es

model : large

tiny, base, small, medium, large

task : transcribe

transcribe or translate

features : speaker_diarization : true word_timestamps : true punctuation : true profanity_filter : false output : formats : - txt - srt - vtt - json include_confidence : true include_timestamps : true Meeting Transcription Meeting Notes Template meeting_transcript : metadata : title : "{{meeting_title}}" date : "{{date}}" duration : "{{duration}}" attendees : "{{speakers}}" output_template : |

{{title}}

Date: { { date } } Duration: { { duration } } Attendees: { { attendees } }

Summary

{ { ai_summary } }

Key Points

{ {

each key_points}}

- { { this } } { { /each } }

Action Items

{ {

each action_items}}

- [ ] { { task } } - @ { { assignee } } - Due : { { due_date } } { { /each } }

Full Transcript

{ {

each segments}}

** [ { { timestamp } } ] { { speaker } } : ** { { text } } { { /each } } Speaker Diarization diarization_config : min_speakers : 2 max_speakers : 10 speaker_labels : - name : "Speaker 1" voice_sample : "sample_1.wav"

Optional

- name : "Speaker 2" voice_sample : "sample_2.wav" output_format : speaker_prefix : true speaker_timestamps : true example_output : | [00:00:05] SPEAKER_1: Welcome everyone to today's meeting. [00:00:12] SPEAKER_2: Thanks for having us. [00:00:18] SPEAKER_1: Let's start with the agenda. Subtitle Generation SRT Format subtitle_config : format : srt timing : max_duration : 7

seconds per subtitle

min_gap : 0.1

seconds between subtitles

chars_per_line : 42 max_lines : 2 style : case : sentence

sentence, upper, lower

numbers : words

words, digits

example_output : | 1 00:00:05,000 --> 00:00:08,500 Welcome to today's presentation about transcription automation. 2 00 : 00:09 , 000 - -

00 : 00:12 , 000 Let me start by explaining the basic concepts. VTT Format vtt_config : format : vtt features : cue_settings : true styling : true example_output : | WEBVTT 00 : 00 : 05.000 - -

00 : 00 : 08.500 align : center Welcome to today's presentation about transcription automation. 00 : 00 : 09.000 - -

00 : 00 : 12.000 align : center <v Speaker 1

Let me start by explaining the basic concepts. Integration Workflows Zoom Integration zoom_transcription : trigger : event : recording_completed workflow : - step : download_recording source : zoom_cloud - step : transcribe engine : whisper language : auto - step : diarize identify_speakers : true - step : generate_notes template : meeting_notes include_summary : true extract_action_items : true - step : distribute destinations : - notion_page - slack_channel - email_attendees YouTube Integration youtube_subtitles : trigger : event : video_uploaded workflow : - step : download_audio source : youtube_video - step : transcribe engine : whisper task : transcribe - step : generate_subtitles formats : [ srt , vtt ] - step : translate target_languages : [ es , zh , ja , de , fr ] - step : upload_subtitles destination : youtube as_cc : true Podcast Processing podcast_workflow : input : source : rss_feed format : audio/mp3 processing : - transcribe : engine : whisper model : large - generate_chapters : detect_topics : true min_duration : 60

seconds

- create_show_notes : summarize : true extract_links : true highlight_quotes : true - create_searchable_index : full_text : true timestamps : true output : - transcript_txt - chapters_json - show_notes_md - search_index Language Support Multi-Language Transcription multilingual : auto_detect : true supported_languages : - code : en name : English model : large - code : zh name : Chinese model : large - code : es name : Spanish model : large - code : ja name : Japanese model : medium translation : enabled : true target : en preserve_original : true Code-Switching code_switching : enabled : true primary_language : en secondary_languages : [ zh , es ] output : | [00:01:23] The next topic is about 人工智能, which has been muy importante in recent years. handling : detect_language_per_segment : true tag_language_switches : true Quality Enhancement Post-Processing post_processing : text_cleanup : - remove_filler_words : [ "um" , "uh" , "like" ] - fix_common_errors : true - normalize_numbers : true formatting : - add_punctuation : true - capitalize_sentences : true - paragraph_breaks : true speaker_attribution : - merge_short_segments : true - min_segment_duration : 1.0 output_enhancement : - add_timestamps : true - highlight_keywords : true - generate_summary : true Accuracy Metrics TRANSCRIPTION QUALITY REPORT ═══════════════════════════════════════ File: meeting_2024_01_15.mp3 Duration: 45:32 Engine: Whisper Large METRICS: Word Error Rate (WER): 4.2% Character Error Rate: 2.8% Confidence Score: 0.94 SPEAKER DIARIZATION: Speakers Detected: 4 Diarization Accuracy: 91% PROCESSING TIME: Total: 8m 23s Real-time Factor: 0.18x DETECTED ISSUES: • Low confidence at 12:34 (background noise) • Overlapping speech at 23:45 • Unknown speaker at 34:12 API Examples OpenAI Whisper import openai

Transcribe audio

with open ( "meeting.mp3" , "rb" ) as audio_file : transcript = openai . Audio . transcribe ( model = "whisper-1" , file = audio_file , response_format = "verbose_json" , timestamp_granularities = [ "word" , "segment" ] )

Access results

for

segment

in

transcript

.

segments

:

print

(

f"[

{

segment

.

start

:

.2f

}

]

{

segment

.

text

}

"

)

AssemblyAI

import

assemblyai

as

aai

transcriber

=

aai

.

Transcriber

(

)

config

=

aai

.

TranscriptionConfig

(

speaker_labels

=

True

,

auto_chapters

=

True

,

entity_detection

=

True

)

transcript

=

transcriber

.

transcribe

(

"https://example.com/meeting.mp3"

,

config

=

config

)

for

utterance

in

transcript

.

utterances

:

print

(

f"Speaker

{

utterance

.

speaker

}

:

{

utterance

.

text

}

"

)

Best Practices

Quality Audio

Clean input = better output

Choose Right Model

Balance speed vs accuracy

Use Diarization

Identify speakers clearly

Post-Process

Clean up automated output

Verify Critical Content

Human review important

Consider Privacy

Handle sensitive content

Store Efficiently

Compress and index

Provide Context

Vocabulary hints help