ClippyMe is a self-hosted web app for creators and editors. Give it a YouTube link or a video file; it finds the moments worth sharing, cuts them into 9:16 clips with captions, and lets you polish and publish them to TikTok, Instagram and YouTube. It can also watch Kick, Twitch and YouTube channels and do all of that automatically.
- Finds the best moments. The video is transcribed, and Google Gemini picks and ranks self-contained clips, with a title, hook and captions for each. Clip edges land on sentence boundaries and silences, never mid-word.
- Frames them for vertical. Active-speaker tracking keeps the person talking in frame, with a camera that stays still within each shot. You can switch a clip to subject-centred framing or to a letterboxed full frame at any time.
- Lets you edit before you post. Colour grade, six animated caption styles, silence and filler-word removal, manual transcript trimming (or "cut the intro" in plain language), hook text, logo and attribution banner — applied when you download or publish, previewed first.
- Publishes and schedules through Zernio, picking prime-time slots and spacing posts to respect platform limits.
- Runs unattended. The live monitor follows streams and uploads, clips each segment, and publishes the best clips on its own.
- Stays under your control. Everything runs on your hardware; clips, settings and keys stay in local folders. Long jobs survive restarts and resume where they stopped.
flowchart LR
A[YouTube link<br/>or upload] --> B[Transcribe]
B --> C[AI picks moments]
C --> D[Cut + reframe 9:16]
D --> E[Edit: captions, grade,<br/>smart cut, hook, logo]
E --> F[Download or publish]
Transcription uses Deepgram (the default) or ElevenLabs when you add its key and select it in Settings, and local Whisper otherwise. Without a Gemini key, ClippyMe still splits the video by topic, but clips are not ranked. The architecture overview explains the design.
You need Docker with Compose v2. Then:
git clone https://github.com/fralapo/clippyme.git
cd clippyme
docker compose up --buildOpen http://localhost:5175, go to Settings, and add the API keys you have:
| Key | Needed for |
|---|---|
| Gemini | Choosing and ranking moments, titles (recommended) |
| Deepgram or ElevenLabs | Fast cloud transcription (optional; local Whisper otherwise) |
| Zernio | Publishing and scheduling (optional) |
| Twitch app credentials | Monitoring Twitch channels (optional) |
Then paste a video link in Create. The getting started guide walks through the first job, updating, and where files are stored.
An NVIDIA GPU is optional
(docker compose -f docker-compose.yml -f docker-compose.gpu.yml up --build);
see deployment for this and the production frontend.
By default ClippyMe listens only on 127.0.0.1 and trusts every client on the
local network for its settings. Before opening it to other devices on a
trusted network, set an API token; exposing it to the internet is not
supported. Details:
deployment → network exposure.
To report a vulnerability, see SECURITY.md.
Contributions are welcome. CONTRIBUTING.md covers setup, the checks CI runs, and the project's rules. The stack is Python 3.11 with FastAPI for the backend, a subprocess pipeline built on ffmpeg, OpenCV, MediaPipe and YOLOv8, and React 18 with Vite and Tailwind for the dashboard.
ClippyMe started as a fork of OpenShorts. It builds on yt-dlp, Faster-Whisper, PySceneDetect, Ultralytics YOLO, MediaPipe, auto-editor, FFmpeg, FastAPI and React, and on the services Google Gemini, Deepgram, ElevenLabs and Zernio. Ideas ported from other open-source projects — ClipsAI (topic segmentation), FrameShift (subject framing), VideoLingo (subtitle line splitting) and several reframing projects — are credited in docs/research.