Adobe announced a significant expansion of its Firefly generative AI platform, introducing video creation and editing capabilities slated for release later this year. The new Firefly Video Model positions Adobe to compete directly with emerging players in the generative video space, including OpenAI's Sora.
Set to debut in beta form, the video expansion to the Firefly tool will integrate with Adobe's flagship video editing software, Premiere Pro. This integration aims to streamline common editorial tasks and expand creative possibilities for video professionals.
The model boasts several notable features, including the capacity to generate B-roll footage from text prompts, with Adobe asserting that high-quality clips can be produced in under two minutes. This capability mirrors the pure video generation offered by platforms like Sora, Kling, or Dream Machine.
Another new tool, Generative Extend, enables editors to lengthen existing clips, smoothing transitions and adjusting timing to align perfectly with audio cues. Moreover, the AI can address video timeline gaps, helping to resolve continuity issues in editing by contextually connecting two clips within the same timeline—a feature that distinguishes Adobe from its competitors.
The Firefly Video Model also incorporates the ability to eliminate unwanted elements from footage, akin to Photoshop's content-aware fill. Adobe says its generative AI technology edits each frame and maintains consistency throughout the timeline, turning a typically slow, manual process into a faster, automated one.
Additionally, the model can produce atmospheric elements like fire, smoke, and water, thereby enhancing video compositing options. While not revolutionary, this capability does add flexibility to Adobe’s video editing suite. Also, just like other existing generative video tools, Firefly supports various camera movements and angles.
The samples shared in the announcement show a pretty powerful model, capable of understanding the context and providing coherent generations.
Finally, Adobe emphasizes that Firefly is “commercial safe”—trained exclusively on licensed content, mitigating potential copyright concerns. This may be a strategic move considering that Adobe's foray into generative AI has been rocky—to put it mildly.
When the company first rolled out AI features in Photoshop, reactions were somewhat mixed, with some creatives seeing potential and others being more skeptical. But then Adobe stepped in it big time. A license change appeared to give Adobe the green light to use customer data, and all hell broke loose.
Content creators, from YouTube stars to industry analysts, raised their voices against the company—even recommending ditching the suite in favor of less popular (but more pro-creator, anti-AI) competitors.
Those competitors smelled blood in the water. Procreate's CEO didn't mince words, declaring he "fucking hated AI" and swearing that tech would never reach their app. Affinity swooped in and assured its users that there wouldn't be generative AI on their suite of products either.
To clarify, we are not developing any generative AI tech and are against anything which undermines human talent or tramples on artists' IP. Any ML/AI features will be workflow focused. We are 100% a champion of human creativity and always will be. https://t.co/16jq9FM9gy
— Affinity (@affinitybyserif) November 9, 2023
Adobe scrambled to patch things up, tweaking their terms of service, but the damage was done. Their reputation took a beating, especially given the groundswell of anti-AI sentiment in creative circles.
Despite the PR nightmare, Adobe remains firm in its pro-AI vision. Just a few weeks ago, the company introduced Magic Fixup, a technique that applies more sophisticated image editing capabilities than normal image editors after being trained on video instead of still images.
The company has opened a waitlist for the Firefly Video Model beta, though specific release dates have not been announced.