How to make a lyric video with Claude Code

A lyric video is the purest test of timing in code-made video. Every line has to appear when it is sung, the big moments have to hit the drops, and the visuals have to give a viewer something to watch for three minutes without fighting the words. It is also a format where Claude does remarkably well, because lyrics are text, timing is data, and the visual ideas can be as abstract as you like.

This tutorial is built mainly on the method Jeff Guo shared for his widely watched lyric video, the long music video briefs other creators published, and the general timing techniques from kinetic typography. It covers the whole pipeline from the folder you start with to the mix you finish with.

What creators published

Jeff Guo (@jeffgwoah) posted a lyric video he asked Claude Fable 5 to make, then shared the method when people asked. He started with a folder: the regular track, the vocals only, a few assets such as screenshots and the album cover he had in mind, and a lyrics file. All graphics were made in Remotion; lyrics were synced with stable-whisper, which works well because the lyrics were supplied; he told Claude the vibe ("semi realistic UI, creatively show text through the UI"), gave it a few specific scene ideas and asked for more in the same spirit, and told it where the important drops were so it could confirm tempo and drop energy with librosa. He notes that Opus struggled with details and continuity between scenes while Fable handled almost every scene in one pass, that some scenes were changed because the idea could have been better, and that he made no assets or animations by hand.

At the other end of the scale, Donald's "Claude Pop" music video, relayed by Movez and seeco, came from a dictated brief of roughly 9,500 characters and a run of about twelve hours; he reports 2.1 million views. His brief asked for lyrics that are huge and present at the hook and more like subtitles elsewhere, with compositions that leave room for the words. Raphael Aubry remade the opening of a music video as a paper-zine piece where every lyric is a cut-paper ransom note stamped on the beat. And Tibo of Revid recreated Jeff Guo's video on his company's product from a one-line prompt, which shows how quickly a good idea becomes a template.

Step 1: build the folder

Start the way Jeff Guo did, with a folder Claude can read. It makes every later step easier and gives a long session something to anchor to.

Project folder

lyric-video/
  track.wav          the full mix (licensed or your own)
  vocals.wav         the vocal stem only
  lyrics.md          one line per sung line, sections marked
  refs/              album cover, screenshots, mood images
  notes.md           the vibe, 3-5 scene ideas, where the drops are
  CLAUDE.md          render contract and style rules

If you do not have a vocal stem, a source separation tool can produce one from the mix. The stem is what makes alignment accurate, because the aligner is not confused by drums and synths.

Make sure you have the right to use the song. A lyric video for your own track, or one you have licensed, is fine; a lyric video of someone else's song is a copyright question that platforms take seriously.

Step 2: align the lyrics to the vocal

Because you already have the words, align them rather than transcribe them. Forced alignment with a whisper-based tool given the lyrics finds where each known word starts and ends in the vocal stem. Save the result as JSON with line and word times. Then check it by ear at a few difficult spots: fast passages, held notes, ad libs. Fix those by hand in the JSON; it is quicker than fighting the aligner.

Alignment with stable-whisper (sketch)

import stable_whisper, json
model = stable_whisper.load_model("medium")
text = open("lyrics.md").read()
result = model.align("vocals.wav", text, language="en")
lines = [{"text": seg.text.strip(), "start": seg.start, "end": seg.end,
          "words": [{"w": w.word.strip(), "t": w.start, "e": w.end}
                    for w in seg.words]} for seg in result.segments]
json.dump(lines, open("lyrics.json", "w"), indent=1)

Step 3: map the song's structure

Next, the music. Analyse the full mix for tempo and beats with librosa, and mark the sections: intro, verses, pre-chorus, chorus, bridge, the drops. Tell Claude where you hear the drops; you know the song better than any beat tracker, and confirming a drop you named is far more reliable than detecting one blind. Our post on beat sync and sound design covers finding the drop by bass energy.

The structure becomes the plan for visual intensity: calmer scenes in verses, the biggest ideas in choruses, something new at the bridge, and the strongest visual on the last chorus.

Step 4: an idea that shows text through a world

A lyric video needs one visual idea that explains why the words are on screen. Jeff Guo's was to show the lyrics through a semi-realistic computer interface: a message thread, several terminals, a social post, each scene a different place where text naturally lives. Raphael Aubry's was cut-paper ransom notes. Other workable ideas: lyrics as street signs in a drive through a city, as handwriting in a notebook, as subtitles on old film, as code being typed.

Give Claude three to five specific scene ideas of your own, then ask it for more in the same style. Your ideas set the bar; its ideas fill the song. Expect to replace some of its scenes, as Jeff Guo did, because a technically good scene can still be the wrong idea.

notes.md (example)

VIBE    semi-realistic desktop UI at night, warm screen glow
        lyrics live INSIDE the UI, never floating on top
SCENES  verse 1: a message thread, lines arrive as texts
        pre-chorus: three terminals, lines typed in sync
        chorus 1: a notification storm, each line a toast
        bridge: the screen powers down, lyrics in the reflection
        (Claude: propose 6 more in this world)
DROPS   0:47, 1:32, 2:41 (big visual change on each)
RULE    hook lines HUGE; verse lines subtitle-sized, low third

Step 5: decide which lines are huge

Not every line should fill the screen. Donald's brief makes the distinction explicit: lyrics very present and big at the hook, where they grab attention, and closer to subtitles elsewhere, with backgrounds kept quieter where the words are big. Mark each line in lyrics.json with a size class and let the scene layouts reserve space accordingly. The contrast is what makes the big lines feel big.

For the big lines, use the techniques from kinetic typography: one accented word, entrances with mass, words arriving on their syllable, readable holds. Our post on kinetic typography cut to the beat goes through them.

Step 6: build scene by scene

A three minute lyric video is long for a code-made film, so build it in scenes with clear boundaries. Each scene owns a time range from the song, receives its lines from lyrics.json, and must start where the previous one ended. Keep fonts, palette and recurring elements in shared settings, and render each scene's boundaries as stills before stitching. Continuity between scenes is exactly where Jeff Guo saw weaker models struggle, so ask for an incoming and outgoing state for every scene and check both sides of every cut.

Remotion is a natural fit here because sequences map to scenes, but a single seek(t) page works just as well for shorter pieces. What matters is that every line's appearance is driven by its timestamp, not by an animation duration someone guessed.

Driving a line from its timestamps

const line = lyrics[i];                    // from lyrics.json
const inT = line.start - 0.08;             // appear a hair early
const outT = line.end + 0.25;              // linger slightly
const shown = t >= inT && t < outT;
const words = line.words.map((w) => ({
  text: w.w,
  on: t >= w.t - 0.03,                      // each word on its syllable
  pop: spring(t - w.t),                     // tiny landing
}));

Step 7: for long runs, write a director's brief with gates

If you want Claude to work through a whole song with little supervision, give it a brief and gates rather than freedom. The long published briefs share a skeleton: the film in one line, references, tools and keys with a budget, a character bible if there are characters, a beat sheet with a visual payoff every few seconds, a plan for on-screen text, workflow gates (plan, stills, animatic, full pass, polish, audio, render) and a critique loop. Donald's included an instruction to watch the whole video multiple times and take screenshots at individual parts, and a reminder to spend economically.

Two warnings from the people who did it. A run that long can spend a lot; set limits on any paid keys. And an unclear reference should produce a question or a test frame, not hours of polished work in the wrong direction.

Step 8: render, mix and check

Render the full mix under the picture, not the stem. Check every chorus line at phone width, scan for single-frame pops, and watch both sides of every scene boundary. Then do one full watch at normal speed with headphones, following only the words: any line you cannot finish reading before it leaves needs a longer hold or fewer words on screen.

Cutting Shorts from the lyric video

A finished lyric video is also a source of Shorts, and they are often where most of its audience comes from. Pick the strongest 15 to 30 seconds (usually the hook), and recompose them for a vertical frame rather than cropping: re-render the same scenes from the same timeline with a 9:16 layout, larger lyrics and fewer elements. Because every line is driven by its timestamp, a vertical version of a scene is a layout change, not a re-edit.

Start the Short slightly before the hook line so the first second already has motion and words, and end on a strong frame rather than mid-line. If the platform loops the Short, check that the last frame flows back into the first. And caption the post with the song and artist, since a lyric Short is often found by people searching for the song.

Pitfalls

• Transcribing instead of aligning: misheard words in your own lyric video.
• No vocal stem: worse alignment in dense mixes.
• Every line huge: nothing feels big.
• Scenes that ignore the song's structure: a chorus that looks like a verse.
• Weak continuity between scenes.
• Accepting every scene the model proposes. Replace the weak ideas.
• Using a song you do not have rights to.

Cost and time

The tools are free (stable-whisper or faster-whisper, librosa, Chrome, ffmpeg), and Remotion is free for individuals and small teams; read its licence if you work for a larger company. The cost is Claude usage, which grows with song length and the number of scene revisions. A short piece or a single chorus is a good first project; a full song with many scenes is a long session or a supervised overnight run.

Doing it in about 3 minutes with WhatToUpload Video Studio

Video Studio does not make lyric videos from your own track today. What it does is the voice-led relative: type one sentence, and Claude writes a script, ElevenLabs voices it and our renderer draws every word as code so it lands on the voice, with captions and music timed, in about three minutes. The styles live today are Kinetic type, Ink stickman and Diagram explainer, with more being added; Kinetic type is the closest to a lyric video.

A 30 second video is 26 to 30 credits, and plans are $19, $49 and $99 a month.