How to Add Animated Captions to a Video With AI

Two kinds of captions, and why you usually want both

Animated captions are the words you say, drawn onto the video as you say them: one word at a time, a short phrase with the current word highlighted, or a block of type that changes with each sentence. The name people search for varies (auto captions, word-by-word captions, TikTok-style captions), but the build is the same.

There are two ways a caption reaches a viewer:

• Open captions are part of the picture. W3C’s captions page describes them as always displayed and impossible to turn off. Animated captions are open captions, because the motion and styling live in the rendered pixels.
• Closed captions live in a separate file, such as SubRip (.srt) or WebVTT (.vtt), and the player shows them when the viewer switches them on.

The two do different jobs. Open captions carry the look and play wherever the file goes. Closed captions can be corrected after publishing, can be turned off, and are the kind a platform’s player can read. Both come from one source: a transcript with a time for every word. Make that transcript once, check it once, and you get the animated render and the caption file from the same data.

This guide stays on captions. For words used as the design itself (scaled, stacked or moving around the frame), read how to make AI kinetic typography. For a list of one-job prompts, see AI video editing prompts for Claude Code.

What you need before you start

• Claude Code, open in a folder that holds your clip. If you have never used it for video, start with the beginner tutorial for editing videos with Claude Code.
• Python 3.9 or newer, which faster-whisper lists as its requirement. Its README says FFmpeg does not need to be installed for transcription, because audio is decoded with PyAV, which bundles the FFmpeg libraries.
• ffmpeg on your computer for the render itself.
• A short test clip, 30 to 60 seconds, with clear speech. Captions are quick to judge on a short clip and slow to fix on a long one.
• A font file you are allowed to use in video, and one or two colours.

Step 1: Transcribe with word timestamps

Every animated caption depends on knowing when each word starts and ends. faster-whisper, an open-source reimplementation of Whisper, returns that when word_timestamps=True is passed to transcribe(): each segment then carries a list of words, each with a start time, an end time and the text.

Ask Claude Code for a file you can read and edit:

Install faster-whisper and transcribe clip.mp4 with word_timestamps=True.
Save words.json: one entry per word with start, end and text.
Also save transcript.txt as plain sentences so I can proofread it.

Two details matter here. The library can filter out non-speech audio with the Silero VAD model, and its README says the default behaviour is conservative, removing only silence longer than 2 seconds. The model also downloads on first use, so the first run takes longer than the ones after it.

Step 2: Check the transcript before anything is drawn

This is the step that decides whether the captions are right. W3C states plainly that automatically generated captions do not meet user needs or accessibility requirements unless they are confirmed to be fully accurate. Its example is the word “not”: drop it and the caption says the opposite of the speaker. Wikipedia’s article on closed captioning adds that automatic captioning is often inaccurate on videos with background music or exaggerated emotion in speaking.

A fast way to proofread:

1. Read transcript.txt while the clip plays. Mark every wrong word.
2. Give Claude Code a spelling list for names, brands and technical terms, and ask it to apply them to words.json without changing any timings.
3. Ask for a list of words shorter than 0.1 s or longer than 2 s. Odd durations often point to a mis-split or merged word.
4. Decide on non-speech sounds. WCAG’s guidance on prerecorded captions asks for meaningful sound effects and, where it is not obvious, who is speaking. A bracketed cue such as [laughs] belongs in the caption file even if it never appears in the animated version.

Fix the text now. Every later step copies it.

Step 3: Choose a caption style

Word-level timing allows several looks. Pick one before you render, because each needs different grouping rules:

Some editing styles on this site build captions into the edit itself. Personal Reels keeps one big word on screen at a time while cuts land on spoken words. Fitted Captions turns each phrase into a justified block, with one huge serif italic accent word and the other words sized so every line spans the same width. B-roll and Stacked Text shows the key phrase of each sentence as a two-line stack, a bold sans line over a serif italic accent line.

Step 4: Render the words onto the video

Now give Claude Code the style as numbers. A brief like this leaves little to guess:

Add animated captions to clip.mp4 using the corrected words.json.
- Group 2 to 4 words per caption. Break at punctuation and at any gap over 0.3 s.
- Each caption appears with its first word. The word being spoken is
  [ACCENT HEX]; the others are white. [FONT FILE], [SIZE] px, bold.
- Centre the captions horizontally, with their bottom edge [Y] px above the
  bottom of the frame. Add a dark outline 4 px wide.
- Pop-in: scale from 90% to 100% over 0.12 s. No exit animation.
- Render three stills (one at 2 s, one mid-sentence, one at a long word)
  and a 10-second 540x960 preview. Wait for my approval before the full render.

Claude can draw the captions in more than one way. A common route is an Advanced SubStation Alpha (.ass) file burned in with ffmpeg’s subtitles filter. That filter draws subtitles with the libass library and is only available in ffmpeg builds configured with --enable-libass. Its force_style option overrides style values such as the font name and colour, and its fontsdir option points at a folder of fonts to use. If your ffmpeg lacks libass, ask Claude to draw each frame’s captions in Python instead and overlay them; the result looks the same, it just takes a different path.

Approve the stills before the preview and the preview before the full render. A wrong size or position costs seconds to fix at the still stage and minutes once the whole clip is rendered.

Step 5: Export, and add an .srt for platform captions

After the animated MP4, ask Claude to write a caption file from the same words.json. SubRip is the simplest format. Wikipedia’s SubRip article describes each entry as a counter starting at 1, a start and end time in hours:minutes:seconds,milliseconds form separated by -->, one or more lines of text, and a blank line to end the entry.

Write captions.srt from words.json for platform captions.
- One to two lines per cue, grouped by sentence or clause, not word by word.
- Times as 00:00:00,000. Plain UTF-8, no styling tags.
- Include sound cues such as [music] where I marked them.
Also write captions.vtt with the same cues.

Word-by-word cues read badly in a player’s caption box, which is why the brief groups the .srt by clause. Plain text matters too. YouTube’s list of supported files accepts SRT only in its basic form, does not recognise style markup in it, and requires plain UTF-8. The same page lists WebVTT among the advanced formats and notes that it supports positioning, with styling limited to bold, italic and underline.

To upload, YouTube’s help page says to open YouTube Studio, choose Subtitles from the left menu, select the video, add a language, then upload a file and choose “With timing”. It also offers auto-sync, where you paste the transcript text and YouTube times it, but it does not recommend that for videos over an hour or with poor audio.

Readability rules for captions on a phone

Captions earn their place only if a viewer can read them in the time they are on screen. Check these on the stills before the full render:

• Size. Judge size on a phone screen, not on your monitor. Send a still to your phone and read it at a normal viewing distance, and prefer a bold weight over a light one.
• Contrast. WCAG’s contrast minimum asks for 4.5:1 for normal text and 3:1 for large text, which it defines as 18 point, or 14 point bold. It also notes that a narrow border around each letter counts towards the letter’s contrast. Over moving footage the background changes every frame, so an outline or a solid box behind the words keeps contrast steady.
• Safe zones. The app you post to draws its own buttons and text on top of your video. Keep captions clear of the edges, and upload a private test or check a screenshot of the posting screen before you publish.
• Line length. Short lines read faster than long ones. Keep each caption to one or two lines and break at natural pauses, never in the middle of a name.
• Position. Keep captions off the face and away from any on-screen text or graphics. Wikipedia notes that roll-up captions on broadcast TV can be placed on any of 14 screen rows to avoid covering graphics or action; on your own render you have full control, so use it.
• Timing. Captions should appear with the word, not before it. If a caption flickers on and off between close words, ask Claude to hold it through gaps shorter than a set length.

Which caption approach fits your video

For most short-form creators the practical answer is the third and second rows together: animated captions burned in for the feed, and a corrected .srt or .vtt uploaded wherever the platform reads one.