one year on
Google’s Gemini demo video was staged, not real-time interaction
The company admits the impressive six-minute hands-on video used still images and text prompts, not live voice and video, casting doubt on the model’s claimed capabilities.
Just 24 hours after Google wowed the internet with a six-minute “Hands-on with Gemini” video, the company admitted the demo was not what it seemed. The video, which showed Gemini narrating a duck drawing, recognizing hand gestures, and tracking objects in real time, was created by feeding the model still images and text prompts — not live voice or video. The company added a note that “latency has been reduced and Gemini outputs have been shortened for brevity” in the YouTube description, not in the video itself.
TechCrunch’s Devin Coldewey called the video “faked,” pointing out that the actual testing process, documented in a blog post, required explicit prompts like “What do you think I’m doing? Hint: It’s a game” — a far cry from the seamless wordless gesture recognition shown in the video. Bloomberg’s Parmy Olson first reported the discrepancy. Google DeepMind VP Oriol Vinyals later clarified that the video was meant to “inspire developers” and show what a future multimodal user experience could look like.
The backlash comes as déjà vu for Google, whose earlier chatbot demo earlier this year was also criticized as rushed and botched. The company is racing to catch up with OpenAI’s GPT-4, and the credibility hit from this staged demo could undermine trust just as Gemini Ultra is poised for a 2024 release.
The record
Argued the video was faked, noting that interactions shown were fundamentally different from the actual text-prompt-based testing.
Stated the video illustrates what multimodal user experiences built with Gemini could look like, made to inspire developers.
Told CNBC the video is an illustrative depiction based on real multimodal prompts and outputs, and that they look forward to seeing what people create when Gemini Pro opens on December 13.
First reported the discrepancy between the video and the actual testing method.
One year later — open only if you can handle spoilers
Gemini Ultra eventually launched in early 2024 and performed well on benchmarks, but the demo controversy lingered as a cautionary tale about over-promising. Google later revised its demo practices to include clearer disclaimers.