Video Processing & Editing

Expert in FFmpeg-based video editing, processing automation, and export optimization for modern content creation workflows.

When to Use

✅ Use for:

Automated video editing pipelines (script-to-video) Cutting, trimming, concatenating clips Adding transitions, effects, overlays Audio mixing and normalization Subtitle/caption handling Export optimization for platforms Batch video processing Color grading and correction

❌ NOT for:

Real-time video editing UI (use DaVinci Resolve/Premiere) 3D compositing (use After Effects/Blender) Motion graphics animation (use After Effects) Basic screen recording (use OBS) Technology Selection Video Editing Tools Tool Speed Features Use Case FFmpeg Very Fast CLI automation Production pipelines MoviePy Medium Python API Programmatic editing PyAV Fast Low-level control Custom processing DaVinci Resolve Slow Full NLE Manual editing

Decision tree:

Need automation? → FFmpeg Need Python API? → MoviePy Need frame-level control? → PyAV Need manual editing? → DaVinci Resolve

Common Anti-Patterns Anti-Pattern 1: Not Using Keyframe-Aligned Cuts

Novice thinking: "Just cut the video at any timestamp"

Problem: Causes artifacts, black frames, and playback issues.

Wrong approach:

❌ Cut at arbitrary timestamp (not keyframe-aligned)

ffmpeg -i input.mp4 -ss 00:01:23.456 -to 00:02:45.678 -c copy output.mp4

Result: Black frames, artifacts, sync issues

Why wrong:

Video codecs use keyframes (I-frames) every 2-10 seconds Non-keyframe cuts require re-encoding Using -c copy (stream copy) without keyframe alignment breaks playback GOP (Group of Pictures) structure depends on keyframes

Correct approach 1: Re-encode for precise cuts

✅ Re-encode for frame-accurate cutting

ffmpeg -i input.mp4 -ss 00:01:23.456 -to 00:02:45.678 \ -c:v libx264 -crf 18 -preset medium \ -c:a aac -b:a 192k \ output.mp4

Frame-accurate, but slower (re-encoding)

Correct approach 2: Keyframe-aligned stream copy

✅ Fast cutting with keyframe alignment

Step 1: Find keyframes near cut points

ffprobe -select_streams v -show_frames -show_entries frame=pkt_pts_time,key_frame \ -of csv input.mp4 | grep ",1$" | awk -F',' '{print $2}'

Step 2: Cut at nearest keyframes (fast, no re-encoding)

ffmpeg -i input.mp4 -ss 00:01:22.000 -to 00:02:46.000 -c copy output.mp4

Blazing fast, no quality loss, but not frame-accurate

Correct approach 3: Two-pass for best of both worlds

✅ Fast seek + precise cut

ffmpeg -ss 00:01:20.000 -i input.mp4 \ -ss 00:00:03.456 -to 00:01:25.678 \ -c:v libx264 -crf 18 -preset medium \ -c:a aac -b:a 192k \ output.mp4

-ss BEFORE -i: Fast seek to keyframe (no decode)

-ss AFTER -i: Precise trim (only decode needed portion)

Performance comparison:

Method Time (1-hour video) Accuracy Quality Stream copy (arbitrary) 2s ❌ Broken ❌ Artifacts Stream copy (keyframe) 2s ±2s ✅ Perfect Re-encode (simple) 15min ✅ Frame ⚠️ Quality loss Two-pass (optimal) 3min ✅ Frame ✅ Perfect

Timeline context:

2010: FFmpeg required full re-encoding for cuts 2015: -c copy added for stream copying 2020: Two-pass cutting became best practice 2024: Hardware acceleration (NVENC) makes re-encoding viable Anti-Pattern 2: Re-encoding Unnecessarily

Novice thinking: "Apply all edits in one FFmpeg command"

Problem: Multiple re-encodings cause cumulative quality loss.

Wrong approach:

❌ Re-encode for each operation (quality degradation)

Operation 1: Trim

ffmpeg -i input.mp4 -ss 00:01:00 -to 00:05:00 \ -c:v libx264 -crf 23 temp1.mp4

Operation 2: Add audio

ffmpeg -i temp1.mp4 -i audio.mp3 -c:v libx264 -crf 23 \ -map 0:v -map 1:a temp2.mp4

Operation 3: Add subtitles

ffmpeg -i temp2.mp4 -vf subtitles=subs.srt \ -c:v libx264 -crf 23 output.mp4

Result: 3x re-encoding = significant quality loss

Why wrong:

Each re-encode is lossy (even with high CRF) Cumulative quality loss (generation loss) 3x encoding time Wasted disk I/O

Correct approach 1: Chain operations in single command

✅ Single-pass encoding with all operations

ffmpeg -ss 00:01:00 -i input.mp4 -i audio.mp3 \ -to 00:04:00 \ -vf "subtitles=subs.srt" \ -map 0:v -map 1:a \ -c:v libx264 -crf 18 -preset medium \ -c:a aac -b:a 192k \ output.mp4

Single re-encode, all operations applied at once

Correct approach 2: Use stream copy when possible

✅ Lossless operations with stream copy

Trim (stream copy)

ffmpeg -i input.mp4 -ss 00:01:00 -to 00:05:00 -c copy temp.mp4

Add audio (stream copy video, encode audio)

ffmpeg -i temp.mp4 -i audio.mp3 \ -map 0:v -map 1:a \ -c:v copy -c:a aac -b:a 192k \ temp2.mp4

Burn subtitles (must re-encode video)

ffmpeg -i temp2.mp4 -vf subtitles=subs.srt \ -c:v libx264 -crf 18 -preset medium \ -c:a copy \ output.mp4

Only 1 video re-encode (for subtitles)

Quality comparison:

Method Encoding Passes Quality (VMAF) Time 3x re-encode (CRF 23) 3 82/100 45min Single pass (CRF 23) 1 91/100 15min Stream copy + 1 encode 1 95/100 18min All stream copy 0 100/100 30s Anti-Pattern 3: Ignoring Color Space Conversions

Novice thinking: "Just concatenate videos together"

Problem: Color shifts, mismatched brightness, broken playback.

Wrong approach:

❌ Concatenate videos with different color spaces

clip1.mp4: BT.709 (HD), yuv420p

clip2.mp4: BT.601 (SD), yuvj420p (full range)

clip3.mp4: BT.2020 (HDR), yuv420p10le

Create concat list

echo "file 'clip1.mp4'" > list.txt echo "file 'clip2.mp4'" >> list.txt echo "file 'clip3.mp4'" >> list.txt

Concatenate without color normalization

ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

Result: Color shifts between clips, broken HDR metadata

Why wrong:

Different color spaces (BT.601 vs BT.709 vs BT.2020) Different pixel formats (yuv420p vs yuvj420p) Different color ranges (limited vs full) Metadata conflicts

Correct approach:

✅ Normalize color space before concatenation

Step 1: Analyze color space of each clip

ffprobe -v error -select_streams v:0 \ -show_entries stream=color_space,color_transfer,color_primaries,pix_fmt \ -of default=noprint_wrappers=1 clip1.mp4

Step 2: Normalize all clips to common color space

Target: BT.709 (HD), yuv420p, limited range

Normalize clip1 (already BT.709)

ffmpeg -i clip1.mp4 -c copy clip1_normalized.mp4

Normalize clip2 (BT.601 SD → BT.709 HD)

ffmpeg -i clip2.mp4 \ -vf "scale=in_range=full:out_range=limited,colorspace=bt709:iall=bt601:fast=1" \ -color_primaries bt709 \ -color_trc bt709 \ -colorspace bt709 \ -c:v libx264 -crf 18 -preset medium \ -c:a copy \ clip2_normalized.mp4

Normalize clip3 (BT.2020 HDR → BT.709 SDR)

ffmpeg -i clip3.mp4 \ -vf "zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=limited,format=yuv420p" \ -color_primaries bt709 \ -color_trc bt709 \ -colorspace bt709 \ -c:v libx264 -crf 18 -preset medium \ -c:a copy \ clip3_normalized.mp4

Step 3: Concatenate normalized clips

echo "file 'clip1_normalized.mp4'" > list.txt echo "file 'clip2_normalized.mp4'" >> list.txt echo "file 'clip3_normalized.mp4'" >> list.txt

ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

Color space guide:

Standard Color Space Transfer Primaries Use Case BT.601 SD bt470bg bt470bg Old SD content BT.709 HD bt709 bt709 Modern HD/FHD BT.2020 UHD/HDR smpte2084 bt2020 4K HDR sRGB Web iec61966-2-1 bt709 Web delivery Anti-Pattern 4: Poor Audio Sync

Novice thinking: "Video and audio are separate, just overlay them"

Problem: Lip sync issues, audio drift, broken playback.

Wrong approach:

❌ Replace audio without sync consideration

ffmpeg -i video.mp4 -i audio.mp3 \ -map 0:v -map 1:a \ -c:v copy -c:a copy \ output.mp4

Problems:

- Audio duration ≠ video duration

- No audio stretching/compression

- Drift over time

Why wrong:

Audio and video have different durations No timebase synchronization No drift correction Ignores original audio sync

Correct approach 1: Stretch/compress audio to match video

✅ Adjust audio speed to match video duration

Get durations

VIDEO_DUR=$(ffprobe -v error -show_entries format=duration \ -of default=noprint_wrappers=1:nokey=1 video.mp4) AUDIO_DUR=$(ffprobe -v error -show_entries format=duration \ -of default=noprint_wrappers=1:nokey=1 audio.mp3)

Calculate speed ratio

RATIO=$(echo "$VIDEO_DUR / $AUDIO_DUR" | bc -l)

Stretch audio to match video (with pitch correction)

ffmpeg -i video.mp4 -i audio.mp3 \ -filter_complex "[1:a]atempo=${RATIO}[a]" \ -map 0:v -map "[a]" \ -c:v copy -c:a aac -b:a 192k \ output.mp4

Correct approach 2: Precise offset and trim

✅ Sync audio with offset and trim

Audio starts 0.5s late, trim to match video

ffmpeg -i video.mp4 -itsoffset 0.5 -i audio.mp3 \ -map 0:v -map 1:a \ -shortest \ -c:v copy -c:a aac -b:a 192k \ output.mp4

-itsoffset: Delay audio by 0.5s

-shortest: Trim to shortest stream

Correct approach 3: Mix multiple audio tracks with sync

✅ Mix dialogue, music, effects with precise timing

ffmpeg -i video.mp4 -i dialogue.wav -i music.mp3 -i sfx.wav \ -filter_complex " [1:a]adelay=0|0[dlg]; [2:a]volume=0.3,adelay=500|500[mus]; [3:a]adelay=1200|1200[sfx]; [dlg][mus][sfx]amix=inputs=3:duration=first[a] " \ -map 0:v -map "[a]" \ -c:v copy -c:a aac -b:a 256k \ output.mp4

adelay: Precise millisecond timing

amix: Mix multiple audio streams

volume: Normalize levels

Audio sync checklist:

□ Verify video and audio durations match □ Use -shortest to prevent excess audio □ Apply adelay for precise timing offsets □ Use atempo for speed adjustment (maintains pitch) □ Set audio bitrate appropriately (128k-256k) □ Test lip sync at beginning, middle, end

Anti-Pattern 5: Wrong Codec/Bitrate for Platform

Novice thinking: "One export settings for everything"

Problem: Wasted bandwidth, poor quality, rejected uploads, compatibility issues.

Wrong approach:

❌ Export everything at 4K 50 Mbps

ffmpeg -i input.mp4 \ -c:v libx264 -b:v 50M -s 3840x2160 \ -c:a aac -b:a 320k \ output.mp4

For Instagram story: 2 GB file, rejected (max 100 MB)

For YouTube: Could use 10 Mbps and look identical

For Twitter: Exceeds bitrate limits

Why wrong:

Platform-specific size/bitrate limits Over-encoding wastes bandwidth Wrong resolution for platform Incompatible codecs

Correct approach: Platform-optimized exports

YouTube (recommended settings):

✅ YouTube 1080p upload

ffmpeg -i input.mp4 \ -c:v libx264 -preset slow -crf 18 \ -s 1920x1080 -r 30 \ -pix_fmt yuv420p \ -color_primaries bt709 -color_trc bt709 -colorspace bt709 \ -movflags +faststart \ -c:a aac -b:a 192k -ar 48000 \ youtube_1080p.mp4

YouTube 4K upload

ffmpeg -i input.mp4 \ -c:v libx264 -preset slow -crf 18 \ -s 3840x2160 -r 60 \ -pix_fmt yuv420p \ -movflags +faststart \ -c:a aac -b:a 256k -ar 48000 \ youtube_4k.mp4

Instagram (Stories, Reels, Feed):

✅ Instagram Story (9:16, max 100 MB, 15s)

ffmpeg -i input.mp4 \ -c:v libx264 -preset medium -crf 23 \ -s 1080x1920 -r 30 -t 15 \ -pix_fmt yuv420p \ -movflags +faststart \ -c:a aac -b:a 128k \ instagram_story.mp4

✅ Instagram Reel (9:16, max 90s)

ffmpeg -i input.mp4 \ -c:v libx264 -preset medium -crf 23 \ -s 1080x1920 -r 30 -t 90 \ -pix_fmt yuv420p \ -movflags +faststart \ -c:a aac -b:a 128k \ instagram_reel.mp4

✅ Instagram Feed (1:1 or 4:5)

ffmpeg -i input.mp4 \ -c:v libx264 -preset medium -crf 23 \ -s 1080x1080 -r 30 \ -pix_fmt yuv420p \ -movflags +faststart \ -c:a aac -b:a 128k \ instagram_feed.mp4

Twitter/X:

✅ Twitter video (max 512 MB, 2:20)

ffmpeg -i input.mp4 \ -c:v libx264 -preset medium -crf 23 \ -s 1280x720 -r 30 -t 140 \ -maxrate 5000k -bufsize 10000k \ -pix_fmt yuv420p \ -movflags +faststart \ -c:a aac -b:a 128k \ twitter.mp4

TikTok:

✅ TikTok (9:16, max 287 MB, 10 min)

ffmpeg -i input.mp4 \ -c:v libx264 -preset medium -crf 23 \ -s 1080x1920 -r 30 -t 600 \ -pix_fmt yuv420p \ -movflags +faststart \ -c:a aac -b:a 128k \ tiktok.mp4

Web (HTML5 video):

✅ Web optimized (fast load, broad compatibility)

ffmpeg -i input.mp4 \ -c:v libx264 -preset medium -crf 23 \ -s 1920x1080 -r 30 \ -pix_fmt yuv420p \ -profile:v baseline -level 3.0 \ -movflags +faststart \ -c:a aac -b:a 128k -ar 48000 \ web.mp4

Platform specs table:

Platform Max Size Max Duration Resolution FPS Bitrate Codec YouTube Unlimited Unlimited 8K 60 Auto H.264/VP9 Instagram Story 100 MB 15s 1080x1920 30 ~5 Mbps H.264 Instagram Reel 1 GB 90s 1080x1920 30 ~8 Mbps H.264 Twitter 512 MB 2:20 1920x1080 60 5 Mbps H.264 TikTok 287 MB 10min 1080x1920 30 ~4 Mbps H.264 LinkedIn 5 GB 10min 1920x1080 30 5 Mbps H.264 Web Varies Varies 1920x1080 30 2-5 Mbps H.264

Export optimization checklist:

□ Use -movflags +faststart for web (progressive download) □ Use -pix_fmt yuv420p for broad compatibility □ Set -r 30 for most platforms (avoid variable framerate) □ Use -preset slow for final exports (better quality) □ Use -preset ultrafast for drafts □ Apply -maxrate and -bufsize for streaming □ Test playback on target platform before bulk export

Production Checklist □ Align cuts to keyframes (or two-pass seek) □ Chain operations in single FFmpeg command □ Normalize color spaces before concatenating □ Verify audio/video sync (test at multiple points) □ Use platform-specific export presets □ Apply -movflags +faststart for web delivery □ Set proper color metadata (bt709 for HD) □ Test output file on target platform □ Keep lossless intermediate files (ProRes, FFV1) □ Use hardware acceleration for batch jobs (NVENC, VideoToolbox)

When to Use vs Avoid Scenario Appropriate? Automated video pipeline (script → video) ✅ Yes - FFmpeg automation Batch process 100 videos ✅ Yes - parallel FFmpeg jobs Trim/cut clips programmatically ✅ Yes - precise cutting Add subtitles to videos ✅ Yes - burn or soft subs Color grade footage ⚠️ Limited - basic only Multi-cam editing ❌ No - use DaVinci Resolve Motion graphics ❌ No - use After Effects Real-time preview editing ❌ No - use Premiere/Resolve References /references/ffmpeg-guide.md - Complete FFmpeg command reference /references/timeline-editing.md - Timeline concepts, multi-track editing /references/export-optimization.md - Platform-specific export settings Scripts scripts/video_editor.py - Cut, trim, concatenate, transitions, effects scripts/batch_processor.py - Parallel batch video processing

安装

❌ Cut at arbitrary timestamp (not keyframe-aligned)

Result: Black frames, artifacts, sync issues

✅ Re-encode for frame-accurate cutting

Frame-accurate, but slower (re-encoding)

✅ Fast cutting with keyframe alignment

Step 1: Find keyframes near cut points

Step 2: Cut at nearest keyframes (fast, no re-encoding)

Blazing fast, no quality loss, but not frame-accurate

✅ Fast seek + precise cut

-ss BEFORE -i: Fast seek to keyframe (no decode)

-ss AFTER -i: Precise trim (only decode needed portion)

❌ Re-encode for each operation (quality degradation)

Operation 1: Trim

Operation 2: Add audio

Operation 3: Add subtitles

Result: 3x re-encoding = significant quality loss

✅ Single-pass encoding with all operations

Single re-encode, all operations applied at once

✅ Lossless operations with stream copy

Trim (stream copy)

Add audio (stream copy video, encode audio)

Burn subtitles (must re-encode video)

Only 1 video re-encode (for subtitles)

❌ Concatenate videos with different color spaces

clip1.mp4: BT.709 (HD), yuv420p

clip2.mp4: BT.601 (SD), yuvj420p (full range)

clip3.mp4: BT.2020 (HDR), yuv420p10le

Create concat list

Concatenate without color normalization

Result: Color shifts between clips, broken HDR metadata

✅ Normalize color space before concatenation

Step 1: Analyze color space of each clip

Step 2: Normalize all clips to common color space

Target: BT.709 (HD), yuv420p, limited range

Normalize clip1 (already BT.709)

Normalize clip2 (BT.601 SD → BT.709 HD)

Normalize clip3 (BT.2020 HDR → BT.709 SDR)

Step 3: Concatenate normalized clips

❌ Replace audio without sync consideration

Problems:

- Audio duration ≠ video duration

- No audio stretching/compression

- Drift over time

✅ Adjust audio speed to match video duration

Get durations

Calculate speed ratio

Stretch audio to match video (with pitch correction)

✅ Sync audio with offset and trim

Audio starts 0.5s late, trim to match video

-itsoffset: Delay audio by 0.5s

-shortest: Trim to shortest stream

✅ Mix dialogue, music, effects with precise timing

adelay: Precise millisecond timing

amix: Mix multiple audio streams

volume: Normalize levels

❌ Export everything at 4K 50 Mbps

For Instagram story: 2 GB file, rejected (max 100 MB)

For YouTube: Could use 10 Mbps and look identical

For Twitter: Exceeds bitrate limits

✅ YouTube 1080p upload

YouTube 4K upload

✅ Instagram Story (9:16, max 100 MB, 15s)

✅ Instagram Reel (9:16, max 90s)

✅ Instagram Feed (1:1 or 4:5)

✅ Twitter video (max 512 MB, 2:20)

✅ TikTok (9:16, max 287 MB, 10 min)

✅ Web optimized (fast load, broad compatibility)