This skill generates high-quality videos using structured prompts and a Python script. The workflow includes creating JSON-formatted prompts and executing video generation with optional reference image.
Core Capabilities
Create structured JSON prompts for AIGC video generation
Support reference image as guidance or the first/last frame of the video
Generate videos through automated Python script execution
Aspect ratio of the generated image (optional, default: 16:9)
[!NOTE]
Do NOT read the python file, instead just call it with the parameters.
Video Generation Example
User request: "Generate a short video clip depicting the opening scene from "The Chronicles of Narnia: The Lion, the Witch and the Wardrobe"
Step 1: Search for the opening scene of "The Chronicles of Narnia: The Lion, the Witch and the Wardrobe" online
Step 2: Create a JSON prompt file with the following content:
{
"title"
:
"The Chronicles of Narnia - Train Station Farewell"
,
"background"
:
{
"description"
:
"World War II evacuation scene at a crowded London train station. Steam and smoke fill the air as children are being sent to the countryside to escape the Blitz."
,
"era"
:
"1940s wartime Britain"
,
"location"
:
"London railway station platform"
}
,
"characters"
:
[
"Mrs. Pevensie"
,
"Lucy Pevensie"
]
,
"camera"
:
{
"type"
:
"Close-up two-shot"
,
"movement"
:
"Static with subtle handheld movement"
,
"angle"
:
"Profile view, intimate framing"
,
"focus"
:
"Both faces in focus, background soft bokeh"
}
,
"dialogue"
:
[
{
"character"
:
"Mrs. Pevensie"
,
"text"
:
"You must be brave for me, darling. I'll come for you... I promise."
}
,
{
"character"
:
"Lucy Pevensie"
,
"text"
:
"I will be, mother. I promise."
}
]
,
"audio"
:
[
{
"type"
:
"Train whistle blows (signaling departure)"
,
"volume"
:
1
}
,
{
"type"
:
"Strings swell emotionally, then fade"
,
"volume"
:
0.5
}
,
{
"type"
:
"Ambient sound of the train station"
,
"volume"
:
0.5
}
]
}
Step 3: Use the image-generation skill to generate the reference image
Load the image-generation skill and generate a single reference image
narnia-farewell-scene-01.jpg
according to the skill.
Step 4: Use the generate.py script to generate the video
python /mnt/skills/public/video-generation/scripts/generate.py
\
--prompt-file /mnt/user-data/workspace/narnia-farewell-scene.json
\
--reference-images /mnt/user-data/outputs/narnia-farewell-scene-01.jpg
\
--output-file /mnt/user-data/outputs/narnia-farewell-scene-01.mp4
\
--aspect-ratio
16
:9
Do NOT read the python file, just call it with the parameters.
Output Handling
After generation:
Videos are typically saved in
/mnt/user-data/outputs/
Share generated videos (come first) with user as well as generated image if applicable, using
present_files
tool
Provide brief description of the generation result
Offer to iterate if adjustments needed
Notes
Always use English for prompts regardless of user's language
JSON format ensures structured, parsable prompts
Reference image enhance generation quality significantly
Iterative refinement is normal for optimal results