Gemini Adds Agentic Video Understanding
Gemini CLIGoogle introduced agentic video understanding for Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite, letting the models dynamically navigate a video's timeline instead of processing every frame at a fixed rate. The approach pulls transcripts, frames, or audio tracks only when needed, cutting token consumption by up to 88% on long-form video while improving accuracy by up to 7% and lowering analysis cost by up to 66%. The capability is live now through the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform, with rollout to the consumer Gemini app and YouTube's Ask YouTube feature planned for later this year.
Key Takeaways
- Up to 88% fewer tokens are consumed on long-form video thanks to Gemini's new dynamic, on-demand scanning approach instead of fixed-frame-rate sampling.
- The feature ships across three models at once, Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite, rather than being limited to a single flagship model.
- Accuracy actually improves by up to 7% alongside the efficiency gains, since the model can concentrate on relevant segments instead of diluting context with irrelevant frames.
- Analysis costs drop by as much as 66%, with no separate fee required: the feature runs on standard Gemini API token pricing.
- Enabling the capability requires no new API endpoint, just setting the processing mode to "agentic" in Google AI Studio or the Gemini Enterprise Agent Platform.
- Google plans to extend the technology beyond developer tools, bringing it to the consumer Gemini app and powering YouTube's "Ask YouTube" feature in the coming months.
Sources & Mentions
2 external resources covering this update
A New Way to Watch Video
Google rolled out agentic video understanding on September 1, 2026, a capability that changes how Gemini's Flash-tier models process video. Previously, Gemini analyzed video the same way most multimodal models do: by sampling frames at a fixed rate regardless of what is actually happening on screen. That approach works, but it burns tokens on static or irrelevant footage and struggles to scale efficiently to long recordings.
With agentic video understanding, Gemini instead takes an active role in deciding what to watch. The model can request specific frames, audio segments, or transcript excerpts on demand, effectively scanning a video the way a person would skim a long recording for the parts that matter, rather than watching it frame by frame from start to finish.
Where It's Available
The feature ships across three models, Gemini 3.7 Flash, Gemini 3.6 Flash, and Gemini 3.5 Flash-Lite, and works through both the Interactions API and the GenerateContent API. Developers can enable it by setting the processing configuration to "agentic" in Google AI Studio or the Gemini Enterprise Agent Platform, and it applies to both uploaded video files and YouTube links. There is no separate fee: usage is billed at standard Gemini API token pricing.
Efficiency and Accuracy Gains
The headline number is token efficiency: Google reports up to 88% fewer tokens consumed on long-form content compared to static processing. That reduction compounds into a real cost benefit, with analysis costs down by as much as 66% on standard video benchmarks. Notably, the efficiency gain does not come at the expense of quality. Google says accuracy on video tasks actually improves by up to 7%, since the model can focus its attention on the segments that matter rather than diluting context with redundant frames.
These gains are most pronounced on genuinely long video, the kind that previously forced developers to choose between high token costs and lossy compression techniques that risked dropping important details, such as 10-minute how-to guides, 90-minute lectures, or multi-hour recordings.
What Developers Can Build
Google highlighted several use cases that agentic video understanding unlocks: sub-second moment retrieval for automated video editing, needle-in-a-haystack search across multi-hour footage, anomaly detection that benefits from dynamic resampling around suspicious frames, and precise object or action counting. Early access partners including Ponder, Revyl, Mosaic, and Resemble.AI reportedly saw strong results applying the feature to their own video pipelines.
What's Next
Agentic video understanding is available immediately through the Gemini API, but Google says it plans to bring the same capability to the consumer Gemini app and to power YouTube's "Ask YouTube" feature on the video watch page in the coming months, extending the efficiency gains beyond developer-facing tools.