music-video

Turn one finished song into a 24fps music video. The song is the master audio. Scene intervals cover it exactly once.

Triggers

• music video / MV / lyric video / visualizer / storyboard
• 音乐视频 / MV合成 / 歌词动画 / 歌曲动画

Capabilities

• code: initialize a pinned p5.js and p5.brush project, then animate the song in JavaScript.
• hybrid: inspect local character images, scene stills, and video clips, then assemble them. No image or video generation API is called.
• Local beat, onset, low-frequency, and loudness analysis for cut points and animation cues.
• Frame-exact assembly with the original song as the only soundtrack.

Inputs

Required:

• song: local audio (.mp3, .wav, .m4a, .flac, .ogg, .aac)
• brief: the visual idea, audience, and any required references
• mode: code or hybrid

Optional:

• lyrics: text or .srt. Required before storyboarding when the vocal timing matters.
• assets: directory of stills and clips for hybrid
• aspect: 16:9 (default), 9:16, 1:1, or 4:3

If the user wants a song created first, use text-to-music, then return here with the chosen file.

Outputs

• <project>/output/final.mp4
• docs/CREATIVE_BRIEF.md, docs/STORYBOARD.md, docs/TASKS.md, and docs/PROGRESS.md
• audio/analysis.json, and docs/inspect.json when assets exist

Workflow

Run python3 skills/music-video/scripts/mv.py from the repository root. If this skill is installed elsewhere, use the scripts/mv.py beside this file. Create the project outside the skill directory.

1. Choose code when the video is drawn in JavaScript. Choose hybrid when the user supplies stills or clips. Ask once when neither is clear.
2. Initialize an isolated project:

python3 skills/music-video/scripts/mv.py init \
  --project ./mv-project --song ./song.mp3 --lyrics ./lyrics.txt --mode code

python3 skills/music-video/scripts/mv.py init \
  --project ./mv-project --song ./song.mp3 --lyrics ./lyrics.srt \
  --assets ./clips --mode hybrid --aspect 16:9

3. Analyze the song. Treat tempo changes as approximate and confirm section boundaries from the audio:

python3 skills/music-video/scripts/mv.py analyze --project ./mv-project

4. Get lyric timing. If the user did not supply it, use speech-to-text only when NOIZ_API_KEY or ~/.config/noiz/api_key already exists. Never print the key. If neither exists, ask for lyrics or timestamps and do not upload the song.
5. Read references/opus-5.5-prompt.md and follow it. Write the creative brief and storyboard before scene code or assembly. Review the opening and main peak before building the other chapters.
6. In code mode, read <project>/animation/ANIMATION_GUIDE.md before drawing. Set PROJECT.bpm and PROJECT.offset from audio/analysis.json. The pinned template canvas is 1920×1080; change studio.html and the renderer window before using another aspect. Check motion with the template renderer:

node render.mjs --sheet=1,2,3 --out=../output/sheets/pilot.jpg
node render.mjs --strip=START:END --out=../output/sheets/pilot-motion.jpg
node render.mjs --clip --range=START:END --out=../output/pilots/opening.mp4

Run those inside <project>/animation. Pass --chrome or set CHROME_PATH when Chrome is not in a standard location. A contact sheet only screens layout. A --strip, or playback of the opening, peak, and ending, is the motion review. 7. In hybrid mode, inspect local media and stop if a required clip is missing. Do not generate a replacement.

python3 skills/music-video/scripts/mv.py inspect --project ./mv-project

8. Write timeline.json using references/timeline.schema.json. Intervals are inclusive at start_frame and exclusive at end_frame.
9. Assemble and probe the result:

python3 skills/music-video/scripts/mv.py assemble --project ./mv-project

Timeline

{
  "fps": 24,
  "width": 1920,
  "height": 1080,
  "audio": "audio/source.mp3",
  "scenes": [
    {
      "id": "opening",
      "start_frame": 0,
      "end_frame": 240,
      "source": "output/pilots/opening.mp4",
      "source_start": 0,
      "fit": "cover",
      "transition": "fade",
      "fade_frames": 6
    }
  ],
  "overlays": []
}

fade stays inside its own scene. Do not overlap scenes to create a crossfade.

Completion

Finish only when every item is true:

• [0, ceil(duration × 24)) is covered exactly once, with no gap or overlap.
• final.mp4 has a video stream and the complete original song.
• The opening, main peak, and ending were reviewed in motion.
• The last story read has time to land.
• docs/PROGRESS.md has no open blocker.

Security

init --mode code clones JohnHeibel/ClaudeAnimationBase at commit 0ac8bf2b31942376cb6b8c4074715595d512acd2 into the project. That template is MIT licensed. Analysis, inspection, rendering, and assembly stay local. Hybrid mode does not upload media and does not call an image, video, or music generation service.

Requirements

• ffmpeg and ffprobe
• Python 3.9+
• analyze: uv pip install 'numpy>=2,<3'
• code mode: git, Node.js, and Google Chrome
• hybrid mode: local assets only

Limitations

• This skill does not compose music, generate sound effects, or create images and video through an API.
• The beat grid assumes a mostly steady tempo.
• Watercolour fills in code mode can take seconds per frame without a discrete GPU. Ask the renderer to avoid those fills on integrated graphics.
• Assembly rejects a clip that is shorter than its scene instead of freezing or retiming it.