Video Summary - Build Full Stack AI Long-to-Short Video Generator & Post Scheduler

Overview

This video is a full-stack tutorial/project walkthrough for an AI “long-to-short” viral video generator + post scheduler SaaS. It demonstrates building the product end-to-end:

Upload a long video
Run AI analysis (scene detection + captions)
Generate multiple short clips (with scores)
Preview and edit caption styles
Render downloadable clips via Remotion Lambda
Schedule posts across social platforms via Zernio
Protect the system with ArjEJet

Core Product Capabilities (What the App Does)

Long video upload → viral shorts output

Supports long videos from ~5 minutes up to 1–2 hours
Produces viral shorts in ~10–20 seconds

AI auto pipeline

The app automatically:

Detects high-engagement moments/scenes
Generates captions
Produces multiple candidate short clips (~4–5)
Assigns SEO/engagement scores plus explanations like:
- why the clip was selected
- high engagement potential (for Reels/TikTok context)

Smart cropping + one-click export

Includes smart cropping and one-click export (described as built-in features)

Caption editing and style customization

Per-short caption editing
Ability to customize caption styles for each short clip

Preview and export

In-browser short preview using a Remotion player
Render & download final clips via Remotion Lambda:
- cloud rendering
- progress + polling

Social posting automation

Connect 15+ social networks through a single API endpoint (Zernio)
Schedule posts with a calendar UI
Use AI to auto-generate:
- post title
- post description
Background scheduling runs via an ingest job every ~15 minutes

Tech Stack & Integrations Used

Next.js (React/TS): frontend + app structure
shadcn/ui: UI components
AWS S3: store uploaded videos and rendered assets
- upload via signed/presigned URLs
- bucket policy tweaks for successful access
Ingest: background workflows / step functions for async processing (upload → transcription → highlight detection → clip generation)
Neon Postgres + Drizzle ORM: store users, projects, transcriptions/captions, and short-clip metadata
Deepgram (free credits mentioned): video transcription + caption generation (SRT-like captions)
Google Gemini: best-segment selection
- returns start/end timestamps
- rationale for the clip
- captions tied to that segment
- ranking out of 100
Remotion: programmatic playback + client preview
Remotion Lambda: server-side rendering + download
ArjEJet: security protections
- endpoint abuse prevention
- rate limiting
- prompt injection detection
- broader abuse/spam protections
Clerk: authentication (sign-in/sign-up, sessions, optional billing mention)
Zernio: social connections + scheduling
- one unified backend API for connecting and publishing

Workflow Implemented (End-to-End)

Landing page
- Generated with an AI agent (hero/features/pricing/FAQ)
- Theme rules handled via a theme.md file
Authentication (Clerk)
- Protect dashboard routes
- Sign in/out UI
- Create user records
Database setup (Neon + Drizzle)
- Define user schema
- Later extend with:
  - projects
  - transcription/caption tables
  - short-video metadata tables
Dashboard layout
- Sidebar navigation:
  - Home
  - My Videos
  - Schedule Post
  - Pricing/Settings
Video upload UI
- Hero-style upload + preview + progress bar
Upload to AWS S3
- Ingest step for upload
- Troubleshooting covered:
  - body size limits
  - CORS bucket policy
  - presigned URL issues
Transcription (Deepgram)
- Ingest runs on “Start AI analysis”
- Generates transcript + captions
- Stores transcript and caption data in the DB
Best moment selection (Gemini)
- Ingest sends transcription to Gemini
- Gemini returns multiple candidate shorts with:
  - start/end timestamps
  - rationale
  - SEO/engagement score
  - captions tied to the selected segment
Short clip rendering/preview (Remotion)
- Initial preview uses Remotion Player (to avoid cloud bandwidth costs)
- Shows:
  - SEO score
  - why it’s engaging
  - captions
Per-clip caption styling editor
- “Edit style” opens a dialog
- Style changes update Remotion preview live
- Saved styles re-applied on refresh
Cloud render + download (Remotion Lambda)
- Download flow:
  - if already rendered → download immediately
  - else → render in cloud with progress polling → enable download
Protect endpoints (ArjEJet)
- Adds rate limits (example: 2 uploads per 24 hours per user)
- Prompt injection detection
- Abuse prevention for HTTP routes/agent tools
Scheduling posts (Zernio)
- “Schedule Post” page:
  - calendar UI + “Add post” dialog
- Select short clip (using exported URLs)
- Select connected social accounts (multi-select)
- AI generates title/description/hashtags
- User chooses schedule time
- Background scheduler job runs every ~15 minutes and posts when due
My Videos page
- Shows user uploads/projects
- Clicking a project shows generated shorts
Deployment (Vercel + Ingest integration)
- Deploy via GitHub → Vercel
- Set environment variables in Vercel
- Sync Ingest with Vercel domain
- Add a deployment protection key so background jobs run in production

Notable Implementation Details / “Gotchas”

Vercel free plan time limits Long workflows require Ingest background jobs to avoid API execution limits.
Ingest payload/body size constraints Discussed tradeoffs for where to do S3 uploading.
S3 upload constraints
- “Next.js body size limit exceeded” error
- updated server action body size up to 2GB
- bucket CORS + bucket policy required origin header changes (localhost vs production)
ArjEJet security
- endpoint protection + rate limiting + prompt injection detection via configuration/prompt
Remotion Lambda setup
- requires AWS IAM policies/roles and Remotion Lambda deployment
- may need concurrency increases (first-run quotes referenced)
Zernio scheduling
- UI stores scheduled posts; background worker checks every 15 minutes and publishes
- verification done via ingest server function + Zernio dashboard + confirmation on actual platforms

Guides / Tutorial Sections Explicitly Covered

Set up a Next.js project from scratch (npx create next app), including Tailwind v4 defaults
Install and wire shadcn/ui
Generate a landing page with an AI agent using a theme.md
Implement Clerk auth:
- sign-in/sign-up
- protecting dashboard routes
Set up Neon + Drizzle ORM and push migrations
Integrate AWS S3 upload (presigned URL approach + CORS/bucket policy edits)
Build a multi-step Ingest workflow:
- upload → transcription → Gemini highlight selection → save short metadata
Use Deepgram for transcription + captions
Use Gemini for best short segment selection:
- timestamps + scores + captions
Create short previews with Remotion Player
Produce final downloadable renders with Remotion Lambda
Implement a caption style editor:
- live Remotion preview + apply/save behavior
Add ArjEJet security:
- endpoint abuse, prompt injection, rate limiting
Integrate Zernio for:
- social account connections
- scheduling through one API
- background scheduling every 15 minutes
Vercel deployment + Ingest production sync using a deployment protection key

Main Speakers / Sources

Speaker/host: the creator of the “Tupji/Tubeguri” channel (first-person presenter who walks through the demo and code)
Tools referenced (not speakers):
- Next.js, React, Tailwind, shadcn/ui
- AWS S3
- Ingest, Neon Postgres, Drizzle ORM
- Deepgram
- Google Gemini
- Remotion (and Remotion Lambda)
- Clerk, ArjEJet, Zernio
- Vercel
- ffmpeg (mentioned as a traditional alternative for cutting)
- Mongo? (not used; main DB is Neon)

Build Full Stack AI Long-to-Short Video Generator & Post Scheduler — Next.js , React, Zernio, Arcjet

Key takeaways