Home
cd ../playbooks
Media ProductionIntermediate

AI YouTube Shorts Generator: Turn Long Videos Into Vertical Clips

Feed in a long video and get ranked, scored 9:16 clips with titles, hooks and reasons, using an open-source pipeline that can run fully local except for one LLM call

20 minutes
By Anil-matchaSource
#youtube-shorts#video-clipping#whisper#reels#tiktok#repurposing#ffmpeg#python

One hour-long podcast hides a dozen clips worth posting, and finding them means watching the whole thing again. This pipeline transcribes the video, has an LLM rank the moments most likely to hold attention, and crops each one to vertical, so you review five clips instead of scrubbing sixty minutes.

Who it's for: podcasters, YouTubers repurposing long videos, agencies producing clips for clients, course creators, founders with recorded talks and webinars

Example

"Make 5 shorts from my latest podcast episode, locally if you can" → local mode transcribes with faster-whisper, an LLM ranks candidates 0-100 on hooks and emotional peaks, and five 9:16 clips land in output/ with a title, hook line and one-line reason each, plus a result.json

CLAUDE.md Template

New here? 3-minute setup guide → | Already set up? Copy the template below.

Or install from your terminal:
npx ccplaybooks add ai-youtube-shorts-generator
# AI YouTube Shorts Generator: Long Video In, Vertical Clips Out

Use this when someone wants to turn a long video (a podcast, interview, tutorial or livestream) into ranked, short vertical clips for Shorts, Reels or TikTok. Trigger phrases: "make shorts from this video", "clip the best moments", "turn this podcast into reels", "find the highlights".

Source: https://github.com/Anil-matcha/AI-Youtube-Shorts-Generator (MIT). It ranks highlights with an LLM, transcribes speech with Whisper, and crops each clip to vertical. It runs in two modes, and **the difference matters for privacy and cost**.

## Which mode, and where your data goes

| Step | API mode (`--mode api`, the default) | Local mode (`--mode local`) |
| --- | --- | --- |
| Download | MuAPI `/youtube-download` | `yt-dlp` for remote URLs, or a local file path |
| Transcription | MuAPI `/openai-whisper` | `faster-whisper` on the user's machine (CPU or CUDA) |

Get new playbooks like this one

One email a week with new Claude Code workflows. Free, like everything here.

No spam. Unsubscribe anytime.

README.md

What This Does

An open-source alternative to paid clipping tools. Give it a long-form video and it returns the best moments as vertical clips:

  1. Downloads the video (or reads a local file)
  2. Transcribes it with Whisper
  3. Classifies the content (podcast, interview, tutorial and so on)
  4. Has an LLM rank highlights on a virality framework: hook moments, emotional peaks, opinion bombs, revelations, conflict, quotable lines, story peaks and practical value
  5. Removes near-duplicate candidates
  6. Crops the top clips to 9:16 (or any aspect ratio you choose)

Each clip comes with a score from 0 to 100, a hook sentence, and a one-line reason it should work. Long videos (over 30 minutes) are processed in overlapping chunks so nothing falls in a gap. Claude helps you pick the mode, set up the environment, run the job and review the results.


Quick Start

Step 1: Create a Project Folder

mkdir -p ~/Documents/ShortsGenerator

Step 2: Download the Template

Click Download above, then:

mv ~/Downloads/CLAUDE.md ~/Documents/ShortsGenerator/

Step 3: Start Working

cd ~/Documents/ShortsGenerator
claude

Say: "Set up the Shorts generator, I want to run it in local mode." Claude clones the repo, creates a virtual environment, installs the dependencies, and has you add your own keys to a .env file.


Pick a Mode First

API mode (default) Local mode
Download, transcribe, rank, crop All through MuAPI, a third-party service On your machine, except the LLM ranking
What leaves your machine The video URL, audio, transcript and clips Only the transcript text, to OpenAI or Gemini
Keys you need MUAPI_API_KEY OPENAI_API_KEY or GEMINI_API_KEY, plus ffmpeg
Setup effort Minimal More (ffmpeg, faster-whisper, OpenCV)

If your footage is private or unreleased, or contains other people's information, use local mode. Both modes use your own accounts and credits.


Examples of What to Ask

  • "Make 5 clips from this YouTube video in local mode, 9:16, 1080p source" → python main.py URL --mode local --num-clips 5 --format 1080
  • "Clip my recording at ~/Videos/webinar.mp4 into square videos" → local mode with --aspect-ratio 1:1
  • "Give me the candidates in a file so I can pick myself" → --output-json result.json
  • "Force English transcription" → --language en
  • "Why was this clip ranked 92?" → Claude reads the hook and reason from the JSON and explains it

Tips & Best Practices

  • Review every clip with the sound on. Check the speaker stays in frame, the clip doesn't start or end mid-sentence, and it doesn't change what the speaker meant.
  • Scores are an LLM's opinion of a transcript. Treat them as a way to shortlist, not as a forecast of views.
  • Tune the framework. The ranking criteria and prompt live in shorts_generator/highlights.py, so you can steer it toward your kind of content.
  • Use a smaller Whisper model to go faster (tiny or base) or a GPU (LOCAL_WHISPER_DEVICE=cuda).
  • Re-runs are cheaper locally. Local mode caches the transcript and the downloaded source, and reuses them.
  • Cost-check batches. A batch run multiplies API usage by the number of videos.

Limitations

  • Only for videos you can reuse. Clipping other people's videos can breach YouTube's terms and copyright.
  • Quality depends on the transcript and model. Videos with no clear speech, or in a language Whisper struggles with, can fail or give weak results. The repo's own README suggests a hosted, improved algorithm for better clips, which is the vendor's paid product.
  • API mode is a dependency on one vendor. If MuAPI changes or goes away, API mode stops working. Local mode doesn't have that dependency (except the LLM call).
  • Vertical cropping is automatic. Face tracking can miss on complex scenes, so check the framing.

Safety Notes

An automated security scan flagged this project for handling API keys in a .env file, running ffmpeg as a subprocess, and sending data to a third-party API. That's what the tool is designed to do, but you should know what it means. A static scan isn't proof of safety, and this one only partially covered the repository.

  • API mode sends your video to MuAPI. The default mode downloads, transcribes, ranks and crops through a third-party commercial service, using your key and credits. Use local mode, or ask Claude to confirm which mode a command will run in, before you process anything private.
  • Local mode still sends the transcript to an LLM provider. The transcript text goes to OpenAI or Gemini (your choice) for ranking. Don't process a transcript you wouldn't send to that provider.
  • Your keys sit in a .env file. Keep it out of git, don't paste keys into the chat, and use keys with spending limits where the provider offers them.
  • It runs system tools. It calls ffmpeg and, in local mode, yt-dlp. Install them from official sources and read any command before approving it.
  • Respect copyright. Only clip videos you own or have permission to reuse, and credit sources where required.

$Related Playbooks

Media Production

AutoSubs: Local AI Subtitles and Transcripts With Speaker Labels

Transcribe audio and video on your own machine with AutoSubs, and let Claude run its command line to produce SRT, VTT, readable transcripts and word-timed JSON, with optional speaker labels and translation

15 minutes
Beginner
Media Production

ASR Transcribe to Text

Transcribe audio and video to text with Qwen3-ASR — local MLX inference on Apple Silicon (no API key, 15–27x realtime) or a remote API endpoint, auto-detecting the best path.

10 minutes
Intermediate
Media Production

Claude Video: Let Claude Watch Any Video

Turns a video URL or local file into frames plus a timestamped transcript, so Claude answers from what it saw and heard instead of the title

10 minutes
Intermediate
Media Production

Editly: Make Videos From Code With Node.js and ffmpeg

Describe a video in a JSON5 spec (clips, images, titles, transitions, music) and let Claude write it and render it with Editly, for slideshows, promos, GIFs and batch resizing

10 minutes
Intermediate
Media Production

Creator Style Reverse-Engineer: One Video In, Style Guide Out

Feed Claude a video from a creator you admire and get a breakdown of their style, a transcript, and a step-by-step replication plan, then blend several creators into your own style

10 minutes
Intermediate
Media Production

Herk Edit Kit: AI Video Editing for Talking-Head Footage

Clone a video-editing kit and let Claude Code cut your silences, review your mistakes, plan the story and build motion graphics, with you approving every cut

30 minutes
Advanced
Media Production

Panel Moderator Guide

Generate a complete moderator guide - sharp questions, minute-by-minute agenda, follow-ups, and audience prompts for any panel.

5 minutes
Intermediate
Media Production

Podcast Production Team

A coordinated five-role agent team — researcher, scriptwriter, show-note editor, distribution manager, and reviewer — that produces a full podcast episode's content in one pass, with the team size scaling to match the actual request.

10 minutes
Intermediate
Media Production

Podcast Production Assistant: Show Notes, Mastering and Distribution

A Claude Code workflow for podcast post-production: master audio to -16 LUFS, generate chapters and timestamped show notes from a transcript, draft promo posts, and run a distribution checklist for every episode.

10 minutes
Advanced
Media Production

Podcast Creator System

Full podcast production workflow from episode ideation and guest research to show notes, SEO optimization, and promotional content.

10 minutes
Intermediate
Media Production

Spotify Integration & Analytics

Automate Spotify music playback, playlist management, and audio analysis workflows

10 minutes
Advanced
Media Production

Social Media Clip Creator

Repurpose long-form video and audio into short-form clips with optimized captions, hooks, and hashtags for TikTok, Reels, and Shorts.

10 minutes
Beginner

Browse all Media Production playbooks →