On Wednesday, Adobe unveiled the Firefly AI video generation tool that will arrive in beta later this year. Like many things related to AI, the examples are equally fascinating and terrifying as the company slowly integrates tools built to automate much of the creative work its valuable user base pays for today.
Repeating the AI salesmanship found elsewhere in the tech industry, Adobe presents it as complementary technology that helps “take the tedium out of post-production.”
Adobe describes its new Firefly-powered text-to-video, Generative Extend (which will be available in Premiere Pro) and image-to-video AI tools as helping editors with tasks like “navigate gaps in footage, remove unwanted objects from a scene, smooth out jump cut transitions and find the perfect B-roll.”
The company says the tools will give video editors “more time to explore new creative ideas, which is the part of their job they love.” (To take Adobe at face value, you’d have to assume that employers won’t increase their output demands from editors once the industry fully adopts these AI tools. Or pay less. Or hire fewer people. But I digress.)
Firefly Text-to-Video lets you create AI-generated videos from — you guessed it — text prompts. But it also includes tools for controlling camera angle, motion, and zoom.
It can take a shot with gaps in its timeline and fill in the blanks. It can turn it into a believable AI video using a static reference image. Adobe says its video models excel with “video of the natural world,” helping to instantly create establishing shots or B-roll without a lot of budget.
Although these samples are curated by a company trying to sell you its products, their quality is undeniable.
Detailed text prompts produce just that, asking for an establishing shot of a fiery volcano, a dog relaxing in a field of wildflowers or (it shows that it can handle hypothetically too) little wool monsters having a dance party. If these results are emblematic of the tool’s typical output (there’s no guarantee), TV, film and commercial production will soon have some powerful shortcuts – whether good or bad.
Meanwhile, Adobe’s example of image-to-video starts with an uploaded galaxy image. A text prompt prompts it to turn it into a video that zooms out from the star system to reveal the inside of a human eye.
A demo of the company’s Generative Extend shows a pair of people walking along a forest stream; an AI-generated segment fills in a gap in the footage. (It was so convincing that I couldn’t tell what portion of the output was AI-generated.)
Reuters reports that the tool will only generate five-second clips, at least initially. To Adobe’s credit, it says its Firefly video models are designed to be commercially safe and only train on content the company has permission to use.
“We only train them on content from the Adobe Stock database that includes 400 million images, illustrations and videos that have been curated to not include intellectual property, trademarks or recognizable characters,” Alexandra Costin, Adobe’s vice president of generative AI, told Reuters.
The company also emphasized that it never trains on users’ work. Whether or not it puts its users out of work, however, is a different matter entirely. Adobe says its new video models will be available in beta later this year. You can sign up to a waiting list to try them out.