VideoDB Skill

Perception + memory + actions for video, live streams, and desktop sessions.

When to use

Desktop Perception

Start/stop a

desktop session

capturing

screen, mic, and system audio

Stream

live context

and store

episodic session memory

Run

real-time alerts/triggers

on what's spoken and what's happening on screen

Produce

session summaries

, a searchable timeline, and

playable evidence links

Video ingest + stream

Ingest a

file or URL

and return a

playable web stream link

Transcode/normalize:

codec, bitrate, fps, resolution, aspect ratio

Index + search (timestamps + evidence)

Build

visual

,

spoken

, and

keyword

indexes

Search and return exact moments with

timestamps

and

playable evidence

Auto-create

clips

from search results

Timeline editing + generation

Subtitles:

generate

,

translate

,

burn-in

Overlays:

text/image/branding

, motion captions

Audio:

background music

,

voiceover

,

dubbing

Programmatic composition and exports via

timeline operations

Live streams (RTSP) + monitoring

Connect

RTSP/live feeds

Run

real-time visual and spoken understanding

and emit

events/alerts

for monitoring workflows

How it works

Common inputs

Local

file path

, public

URL

, or

RTSP URL

Desktop capture request:

start / stop / summarize session

Desired operations: get context for understanding, transcode spec, index spec, search query, clip ranges, timeline edits, alert rules

Common outputs

Stream URL

Search results with

timestamps

and

evidence links

Generated assets: subtitles, audio, images, clips

Event/alert payloads

for live streams

Desktop

session summaries

and memory entries

Running Python code

Before running any VideoDB code, change to the project directory and load environment variables:

from

dotenv

import

load_dotenv

load_dotenv

(

".env"

)

import

videodb

conn

=

videodb

.

connect

(

)

This reads

VIDEO_DB_API_KEY

from:

Environment (if already exported)

Project's

.env

file in current directory

If the key is missing,

videodb.connect()

raises

AuthenticationError

automatically.

Do NOT write a script file when a short inline command works.

When writing inline Python (

python -c "..."

), always use properly formatted code — use semicolons to separate statements and keep it readable. For anything longer than ~3 statements, use a heredoc instead:

python

<<

'EOF'

from dotenv import load_dotenv

load_dotenv(".env")

import videodb

conn = videodb.connect()

coll = conn.get_collection()

print(f"Videos: {len(coll.get_videos())}")

EOF

Setup

When the user asks to "setup videodb" or similar:

1. Install SDK

pip

install

"videodb[capture]"

python-dotenv

If

videodb[capture]

fails on Linux, install without the capture extra:

pip

install

videodb python-dotenv

2. Configure API key

The user must set

VIDEO_DB_API_KEY

using

either

method:

Export in terminal

(before starting Claude):

export VIDEO_DB_API_KEY=your-key

Project

.env

file

Save VIDEO_DB_API_KEY=your-key in the project's .env file Get a free API key at console.videodb.io (50 free uploads, no credit card). Do NOT read, write, or handle the API key yourself. Always let the user set it. Quick Reference Upload media

URL

video

coll . upload ( url = "https://example.com/video.mp4" )

YouTube

video

coll . upload ( url = "https://www.youtube.com/watch?v=VIDEO_ID" )

Local file

video

coll . upload ( file_path = "/path/to/video.mp4" ) Transcript + subtitle

force=True skips the error if the video is already indexed

video . index_spoken_words ( force = True ) text = video . get_transcript_text ( ) stream_url = video . add_subtitle ( ) Search inside videos from videodb . exceptions import InvalidRequestError video . index_spoken_words ( force = True )

search() raises InvalidRequestError when no results are found.

Always wrap in try/except and treat "No results found" as empty.

try : results = video . search ( "product demo" ) shots = results . get_shots ( ) stream_url = results . compile ( ) except InvalidRequestError as e : if "No results found" in str ( e ) : shots = [ ] else : raise Scene search import re from videodb import SearchType , IndexType , SceneExtractionType from videodb . exceptions import InvalidRequestError

index_scenes() has no force parameter — it raises an error if a scene

index already exists. Extract the existing index ID from the error.

try : scene_index_id = video . index_scenes ( extraction_type = SceneExtractionType . shot_based , prompt = "Describe the visual content in this scene." , ) except Exception as e : match = re . search ( r"id\s+([a-f0-9]+)" , str ( e ) ) if match : scene_index_id = match . group ( 1 ) else : raise

Use score_threshold to filter low-relevance noise (recommended: 0.3+)

try : results = video . search ( query = "person writing on a whiteboard" , search_type = SearchType . semantic , index_type = IndexType . scene , scene_index_id = scene_index_id , score_threshold = 0.3 , ) shots = results . get_shots ( ) stream_url = results . compile ( ) except InvalidRequestError as e : if "No results found" in str ( e ) : shots = [ ] else : raise Timeline editing Important: Always validate timestamps before building a timeline: start must be >= 0 (negative values are silently accepted but produce broken output) start must be < end end must be <= video.length from videodb . timeline import Timeline from videodb . asset import VideoAsset , TextAsset , TextStyle timeline = Timeline ( conn ) timeline . add_inline ( VideoAsset ( asset_id = video . id , start = 10 , end = 30 ) ) timeline . add_overlay ( 0 , TextAsset ( text = "The End" , duration = 3 , style = TextStyle ( fontsize = 36 ) ) ) stream_url = timeline . generate_stream ( ) Transcode video (resolution / quality change) from videodb import TranscodeMode , VideoConfig , AudioConfig

Change resolution, quality, or aspect ratio server-side

job_id

conn . transcode ( source = "https://example.com/video.mp4" , callback_url = "https://example.com/webhook" , mode = TranscodeMode . economy , video_config = VideoConfig ( resolution = 720 , quality = 23 , aspect_ratio = "16:9" ) , audio_config = AudioConfig ( mute = False ) , ) Reframe aspect ratio (for social platforms) Warning: reframe() is a slow server-side operation. For long videos it can take several minutes and may time out. Best practices: Always limit to a short segment using start / end when possible For full-length videos, use callback_url for async processing Trim the video on a Timeline first, then reframe the shorter result from videodb import ReframeMode

Always prefer reframing a short segment:

reframed

video . reframe ( start = 0 , end = 60 , target = "vertical" , mode = ReframeMode . smart )

Async reframe for full-length videos (returns None, result via webhook):

video . reframe ( target = "vertical" , callback_url = "https://example.com/webhook" )

Presets: "vertical" (9:16), "square" (1:1), "landscape" (16:9)

reframed

video . reframe ( start = 0 , end = 60 , target = "square" )

Custom dimensions

reframed

video . reframe ( start = 0 , end = 60 , target = { "width" : 1280 , "height" : 720 } ) Generative media image = coll . generate_image ( prompt = "a sunset over mountains" , aspect_ratio = "16:9" , ) Error handling from videodb . exceptions import AuthenticationError , InvalidRequestError try : conn = videodb . connect ( ) except AuthenticationError : print ( "Check your VIDEO_DB_API_KEY" ) try : video = coll . upload ( url = "https://example.com/video.mp4" ) except InvalidRequestError as e : print ( f"Upload failed: { e } " ) Common pitfalls Scenario Error message Solution Indexing an already-indexed video Spoken word index for video already exists Use video.index_spoken_words(force=True) to skip if already indexed Scene index already exists Scene index with id XXXX already exists Extract the existing scene_index_id from the error with re.search(r"id\s+([a-f0-9]+)", str(e)) Search finds no matches InvalidRequestError: No results found Catch the exception and treat as empty results ( shots = [] ) Reframe times out Blocks indefinitely on long videos Use start / end to limit segment, or pass callback_url for async Negative timestamps on Timeline Silently produces broken stream Always validate start >= 0 before creating VideoAsset generate_video() / create_collection() fails Operation not allowed or maximum limit Plan-gated features — inform the user about plan limits Examples Canonical prompts "Start desktop capture and alert when a password field appears." "Record my session and produce an actionable summary when it ends." "Ingest this file and return a playable stream link." "Index this folder and find every scene with people, return timestamps." "Generate subtitles, burn them in, and add light background music." "Connect this RTSP URL and alert when a person enters the zone." Screen Recording (Desktop Capture) Use ws_listener.py to capture WebSocket events during recording sessions. Desktop capture supports macOS only. Quick Start Choose state dir : STATE_DIR="${VIDEODB_EVENTS_DIR:-$HOME/.local/state/videodb}" Start listener : VIDEODB_EVENTS_DIR="$STATE_DIR" python scripts/ws_listener.py --clear "$STATE_DIR" & Get WebSocket ID : cat "$STATE_DIR/videodb_ws_id" Run capture code (see reference/capture.md for the full workflow) Events written to : $STATE_DIR/videodb_events.jsonl Use --clear whenever you start a fresh capture run so stale transcript and visual events do not leak into the new session. Query Events import json import os import time from pathlib import Path events_dir = Path ( os . environ . get ( "VIDEODB_EVENTS_DIR" , Path . home ( ) / ".local" / "state" / "videodb" ) ) events_file = events_dir / "videodb_events.jsonl" events = [ ] if events_file . exists ( ) : with events_file . open ( encoding = "utf-8" ) as handle : for line in handle : try : events . append ( json . loads ( line ) ) except json . JSONDecodeError : continue transcripts = [ e [ "data" ] [ "text" ] for e in events if e . get ( "channel" ) == "transcript" ] cutoff = time . time ( ) - 300 recent_visual = [ e for e in events if e . get ( "channel" ) == "visual_index" and e [ "unix_ts" ]

cutoff ] Additional docs Reference documentation is in the reference/ directory adjacent to this SKILL.md file. Use the Glob tool to locate it if needed. reference/api-reference.md - Complete VideoDB Python SDK API reference reference/search.md - In-depth guide to video search (spoken word and scene-based) reference/editor.md - Timeline editing, assets, and composition reference/streaming.md - HLS streaming and instant playback reference/generative.md - AI-powered media generation (images, video, audio) reference/rtstream.md - Live stream ingestion workflow (RTSP/RTMP) reference/rtstream-reference.md - RTStream SDK methods and AI pipelines reference/capture.md - Desktop capture workflow reference/capture-reference.md - Capture SDK and WebSocket events reference/use-cases.md - Common video processing patterns and examples Do not use ffmpeg, moviepy, or local encoding tools when VideoDB supports the operation. The following are all handled server-side by VideoDB — trimming, combining clips, overlaying audio or music, adding subtitles, text/image overlays, transcoding, resolution changes, aspect-ratio conversion, resizing for platform requirements, transcription, and media generation. Only fall back to local tools for operations listed under Limitations in reference/editor.md (transitions, speed changes, crop/zoom, colour grading, volume mixing). When to use what Problem VideoDB solution Platform rejects video aspect ratio or resolution video.reframe() or conn.transcode() with VideoConfig Need to resize video for Twitter/Instagram/TikTok video.reframe(target="vertical") or target="square" Need to change resolution (e.g. 1080p → 720p) conn.transcode() with VideoConfig(resolution=720) Need to overlay audio/music on video AudioAsset on a Timeline Need to add subtitles video.add_subtitle() or CaptionAsset Need to combine/trim clips VideoAsset on a Timeline Need to generate voiceover, music, or SFX coll.generate_voice() , generate_music() , generate_sound_effect() Provenance Reference material for this skill is vendored locally under skills/videodb/reference/ . Use the local copies above instead of following external repository links at runtime. Maintained By: VideoDB