Setup Local system AI Clipping Tool

via Freelancer ·

Budget / Salary₹600–1,500
TypeFreelance project
LocationRemote
Posted1 hour ago
Python + Gemini API + FFmpeg Developer for 1-Time Local Multimodal AI Auto-Clipping Setup (Google Meet)

Job Description:
I am looking for an experienced Python, Google Gemini API (`google-genai` SDK), and FFmpeg developer for a one-time, complete setup of a local, multimodal AI video auto-clipping pipeline on my computer. The setup and live verification will be completed over a Google Meet session (screen share).

Project Goal:
Build and configure a repeatable local CLI tool where I pass a long-form video file or gaming/podcast VOD URL (e.g., GTA 6 streams or talking-head videos), optionally enter a custom topic prompt, and automatically generate copyright-safe, vertical 9:16 YouTube Shorts/Reels—using Gemini's native multimodal capabilities with minimal API cost and zero paid SaaS subscriptions.

Required Architecture & Features:
1. Video Ingestion & Multimodal Analysis (Gemini File API):
- Support local video files and `yt-dlp` time-slice downloads (YouTube/Twitch/Kick).
- Upload long-form video/audio directly via the Gemini File API to analyze both spoken dialogue and visual cues (scene changes, reactions, emotions, on-screen action/slides) without requiring a separate transcription tool.

2. Token & Cost Optimization:
- Implement Gemini Context Caching so the same uploaded video can be re-queried multiple times (e.g., new topic searches or extra clips) without re-uploading or repaying full input token costs.
- Configure frame-rate sampling / media resolution settings to keep processing fast and API costs low.

3. AI Highlight, Hook & Copyright-Safe Detection (Gemini 2.5 Pro / Flash):
- Prompt Gemini to identify the top 5–10 standalone, high-retention clips (30–60 seconds) with exact `HH:MM:SS` start and end timestamps.
- Include safety filtering (e.g., skipping story cutscenes/spoilers for gaming clips).

4. Structured JSON Outputs (Response Schema):
- Use Gemini Structured Outputs (Pydantic / Response Schema) so every call returns guaranteed valid JSON containing: start/end timestamps, virality score (1–100), on-screen hook text, reasoning, speaker/subject position tags, metadata, and timestamped captions.

5. Topic-Based Custom Clipping (Prompt Search):
- Support an optional CLI flag (e.g., `--topic "police chases"` or `--topic "moments discussing pricing"`) so Gemini extracts topic-specific clips on demand from the cached video.

6. Smart Framing & Speaker Location Hints (Dynamic 9:16 FFmpeg Cropping):
- Use Gemini visual analysis to tag subject/speaker framing (`left`, `center`, `right`, or `split-screen`) per timestamp segment and translate those hints into automated 9:16 (`1080x1920`) FFmpeg crop/layout commands.

7. Timestamped Subtitles, Translation & Auto-Generated Metadata:
- Extract time-aligned captions from Gemini, burn styled bold subtitles (`.ass`/`.srt`) cleanly into the Shorts safe zone via FFmpeg, and support an optional `--lang` flag for multi-language subtitle translation.
- Auto-generate scroll-stopping on-screen text hooks, YouTube Shorts/Reels titles, descriptions, and hashtags saved alongside each clip.

8. Copyright Audio Shield (Optional Toggle):
- Integrate local `Demucs` (toggleable flag like `--strip-music`) to remove copyrighted background/radio music while keeping clean vocals/dialogue for gaming clips.

9. Clean 1-Command CLI & Handoff:
- Wrap everything into a clean virtual environment with `.env` config so I can run commands like:
`python clip.py "VIDEO_URL_OR_PATH" --clips 5 --topic "funny fails" --lang en --strip-music`
- Include a short `README.txt` cheat sheet with all flags and usage examples.

My System: [Enter OS: macOS / Windows 11, RAM, CPU/GPU]
Session Format: One-time live Google Meet session (estimated 2–3 hours; you can pre-write the script beforehand and spend the call installing, tuning, and testing on my machine).
Budget: [Suggested: ₹4,500–₹8,000 / $60–$120 Fixed Price]

Definition of Done (Before Payment Release):
- Clean local setup (`venv`, `.env`, FFmpeg, `yt-dlp`, `google-genai`, `demucs`) with zero missing dependencies.
- Live end-to-end test in a brand-new terminal window processing 1 long-form video via Gemini File API + Context Caching.
- Live test of a second topic-based query against the cached video to verify Context Caching works.
- Successfully outputs ready-to-upload 1080x1920 (`yuv420p`) Shorts with smart framing, synced burned-in captions (plus 1 translated test), on-screen hook text, and metadata files.

To Apply, Please Answer These 3 Questions:
1. Have you built production scripts using the new `google-genai` Python SDK (specifically File API, Context Caching, and Structured Outputs / Response Schema)?
2. How would you handle FFmpeg dynamic cropping (`left`/`center`/`right`/`split-screen`) and ensure accurate caption timing across segment cuts?
3. What is your fixed price and availability to pre-build the script and complete the live Google Meet setup?
javascript python software architecture video editing video processing ai model development ai content creation ai development ai automation ai workflow automation
Apply on Freelancer →

Project sourced from Freelancer.com. Applications happen directly on the original platform — we never collect your data.