Claude Video Vision

Claude Code Video Vision

Give Claude the ability to watch and understand videos.

A Claude Code plugin that extracts frames via ffmpeg and processes audio via multiple backends (Gemini API, local Whisper, or OpenAI API). Claude receives frames as images and audio transcription with timestamps — the plugin is a perception layer, not an interpretation layer.

Features

- Multimodal perception — Claude sees video frames directly and reads audio transcriptions with timestamps
- YouTube URL support — Pass a YouTube URL directly; the MCP server downloads it with yt-dlp, preserving source metadata and captions for context
- Flexible backends — Choose between cloud APIs or fully local processing
- Adaptive extraction — Claude adjusts fps, time range, and resolution based on your question
- Auto-installation — Whisper models download automatically on first use
- Interactive setup wizard — /setup-video-vision walks you through configuration

Quick Start

1. Install the plugin

Inside Claude Code, run these commands one at a time:

Then:

The MCP server will auto-install via npx from npm on first use — no build step required.

Alternative: local development

2. Configure

Inside Claude Code, run the interactive wizard:

It will walk you through backend selection, whisper configuration (if local), frame options, and dependency verification.

Usage

Slash command

Conversational

Just mention a video file or YouTube URL — Claude will detect it:
"analyze this video for me: ~/Downloads/demo.mp4"
"take a look at the first second of ~/videos/bug-report.mov"
"summarize this YouTube Short: https://www.youtube.com/shorts/..."

Claude adapts parameters automatically:
- "the first second" → extracts at original fps from 00:00:00 to 00:00:01
- "summarize this 1h lecture" → low fps, full…