Guides · AI music video
·A shot-list-first, stills-before-animation workflow drawn from the major AI video tools·9 min read
The short answer
The way to make an AI music video that doesn't look like slop is to treat images and video as two separate, sequential steps, not one. Generate and iterate on still images first: block out every shot, lock a consistent character and visual style, and only move on once a still actually looks right. Then animate, and only animate the stills that already work, because video generation costs far more per attempt than image generation and mistakes are expensive to discover after the fact.
The second half of the answer is consistency. Most AI video looks like slop because the character's face, hair, or outfit drifts between shots, the lighting shifts, and the motion has no connection to the story or the music. Fix that by building a small reference set for your character or persona, meaning several clean angles and one locked description, and reusing it across every shot. Cut the finished clips to the beat of your track instead of leaving them full-length, and you get something that reads as a directed video instead of a slideshow of random AI clips.
Start with a shot list, not a prompt. Break your track into sections (intro, verse, chorus, bridge, outro) and assign a small number of shots to each one. For each shot, write down what is in frame, the camera angle, the lighting, and what happens, the same way you would sketch a shot on a storyboard. OpenAI's Sora prompting guide recommends exactly this: script a beat sheet and storyboard, and define a "style spine," a consistent look and feel, before you generate a single frame, because a clear prompt describes the shot as if you were sketching it on a storyboard: framing, depth of field, the action in beats, and the lighting and palette (OpenAI Cookbook: Sora 2 Prompting Guide).
Naming your lighting and color choices matters more than it sounds. The same Sora guide notes that specifying a consistent mix of light sources, and naming three to five colors, keeps the palette stable across shots when you cut them together later (OpenAI Cookbook: Sora 2 Prompting Guide). Decide this once, in your shot list, and every later shot should refer back to it.
Because a still image is cheap and fast to redo, and a video generation is not. Across the major AI generation tools, a few seconds of finished video costs on the order of 13 times as much to generate as a single still image, and a music video needs dozens of clips. If the character's face is wrong, the framing is off, or the outfit doesn't match, you want to catch that on a cheap still you can redo in seconds, not on an expensive video clip you already paid to render.
This is also how the professional tools are built to work. Runway's own guidance is direct about the split: image-to-video means the image handles the appearance while the prompt only has to direct motion, camera behavior, timing, and continuity, whereas text-to-video is not constrained to a specific starting scene, which is powerful for B-roll but means you are not locking anything down first (Runway Help Center: Text to Video Prompting Guide). ElevenLabs' own image and video workflow is built the same way: generate stills first with an image model, use them as storyboards, thumbnails, or source material for video projects, and only then generate video from them (ElevenLabs Blog: Introducing ElevenLabs Image & Video). Lock the shot as a still. Approve it. Then spend on the video.
This is the single biggest thing separating a real music video from a pile of random AI clips, and every major tool has its own version of the same fix: build a small set of reference images and reuse them everywhere.
Luma's own guidance calls this a "Master Reference Asset Pack": a handful of clean images of your character (front, three-quarter, side, and full-body, one angle per image, not combined into a single sheet) that you keep re-anchoring to every new generation, because giving a model only one angle forces it to invent the angles it cannot see, which is when the character starts to drift (Luma Learning Center: Character and Object Consistency). Kling's own blog recommends the same idea under the name of a "character bible": a master description covering face, hair, and clothing that you repeat with the same wording in every prompt, plus four to six reference images with consistent lighting and pose (Kling AI Blog: The Ultimate Guide to Keeping Your Character Consistent in AI). Midjourney's Character Reference feature works the same way from the image side: point it at an existing image and it pulls the face, hair, and clothing into every new generation, with a character weight setting that lets you keep the face while changing the outfit (Midjourney: Introducing Character References).
The pattern across all three tools is identical: pick your reference set once, write down what must never change, and re-anchor every new shot to that set instead of to whatever you generated last. Drift compounds if you keep building on the newest version instead of the original reference.
Only the ones that survived the still stage. Once a still is locked, meaning the right character, framing, and lighting, use it as the first frame for image-to-video rather than describing the scene from scratch in text. Runway is explicit that this trade-off exists for a reason: text-to-video is better when exact character or scene consistency is not the priority, like B-roll, while image-to-video is the choice when it is (Runway Help Center: Text to Video Prompting Guide).
Keep the prompt for the animation step almost entirely about motion, not appearance: camera movement, what the subject does, pacing. The image already answers what things look like. OpenAI's guidance adds one more practical point: models follow instructions more reliably in shorter clips, so it is often better to generate two clean shorter clips and cut them together than to force one longer, less controlled generation (OpenAI Cookbook: Sora 2 Prompting Guide).
Map your shots to the song's structure before you generate anything, and cut to that map afterward. Do not cut on every single beat, that reads as frantic. Reserve hard cuts for downbeats, section changes (verse to chorus), and lyrical hits, and let some shots breathe across a full phrase. If a shot has a strong action or gesture in it, timing the peak of that action to land on the beat, rather than the cut itself landing on the beat, usually reads as more intentional. Treat the song's arrangement as your storyboard's spine: it tells you where a shot needs to change, and gives you an objective reason to keep a clip short instead of stretching a mediocre generation to fill time.
The tells are specific and repeatable, which means they are avoidable.
Different face, different outfit, different hair, because each shot was generated independently with no shared reference. Fix it with the reference-set approach above, applied to every single shot, not just some of them.
These are the hardest things for a video model to hold steady, since it is re-rendering fine detail every frame. When hands are not essential to the shot, compose them out of frame. When a hand needs to do something, favor slower, simpler motion. It gives the model more frames to resolve the geometry, and gives you a still to lock down first.
Camera drifts, or a subject moves for no story reason. This traces straight back to skipping the plan: if your shot list does not say what happens and why, the generation will not either.
Clips placed end to end at their native length instead of being cut to the track's structure. This is the editing-to-the-beat step, and skipping it is the fastest way to make a video feel like a demo reel instead of a music video.
Flat lighting, no specific palette, nothing that looks like a deliberate choice. A locked style spine, and naming your actual lighting and colors in every prompt, is what prevents it.
Every one of these traces back to the same root cause: skipping the still-image, locked-reference stage and generating video directly from a bare prompt.
No. The workflow is a checklist, not a craft skill: plan shots against your song, lock a still before you animate it, and reuse the same character reference everywhere. None of that requires camera or editing training.
Yes, for shots where an exact consistent character or scene does not matter, like abstract B-roll, texture shots, or environmental cutaways. For any shot with your artist or a recurring character in it, start from a locked still instead.
Most guidance from AI tool makers lands in the same range: a handful of clean images, generally three to six, covering different angles, with consistent lighting and one clear angle per image rather than a combined sheet.
Video generation adds motion and time, which gives the model far more opportunity to guess wrong about anything that was not already fixed in the image, especially faces, hands, and fine detail. Locking those in the still first is what carries over cleanly into the video.
Usually you do not need to restart the whole project. Identify which specific shots have the drifting character or the warped detail, regenerate just those from a locked still with a proper reference set, and re-cut the edit to the beat. The planning and consistency steps matter most for the shots that are actually broken.
Wavmark Studio's Director agent plans the video with you in chat, quotes the cost of every image and video generation before you spend, and keeps your character and look consistent across every shot. Plan your shot list, lock your character, and start with stills at studio.wavmark.io.
Create your free accountWe use analytics cookies to understand how Wavmark and Wavmark Studio are used, so we can make them better. Essential cookies are always on. Your choice applies to both.