Adobe's NEW AI Video Features are AMAZING! (Letโs Recreate Them)
TLDRThis week's AI film news is packed with exciting updates from the tech world. Microsoft's Vasa 1 tool is set to revolutionize entertainment by converting images and audio clips into realistic videos, with the unique ability to control eye movements and project them onto different characters. Adobe's Premiere Pro is also unveiling new AI video features, including video painting, object removal, and generative expand, which will extend the capabilities of video editing. Meta's new AI assistant, available for free, is making waves with its creative and up-to-date responses, integrated across social platforms and offering real-time image generation. The news also covers an AI creativity tournament in Las Vegas, the potential for an AI creative Esports league, and a look at the latest AI films, showcasing the creative potential of AI in storytelling and film production.
Takeaways
- ๐ฅ Microsoft launched Vasa 1, a tool that transforms images and audio clips into realistic videos with controllable eye movements and character projections.
- ๐ Vasa 1's unique feature allows for nonhuman animations, such as the Mona Lisa singing, in a highly realistic manner.
- ๐ The tool also enables 3D dynamics control, offering more flexibility in character orientation during animations.
- โก Vasa 1 renders animations in real time, which could impact live-action entertainment and various societal applications like AI therapy.
- ๐ Adobe announced new AI video tools for Premiere Pro, including 'video in painting' which allows for content transformation within the video editing software.
- ๐งฉ Pik Labs can be used as an alternative to Adobe's features, enabling video inpainting tasks with a simple interface.
- ๐ Adobe's object removal tool, akin to automatic rotoscoping, was showcased with impressive results, though not without minor imperfections.
- ๐จ Runway offers a similar object removal feature that works in real time, providing a quick solution for video editing tasks.
- ๐ Adobe's generative expand feature extends video clips using AI, but the integration with live-action footage can be static and may require further refinement.
- ๐ Adobe is opening its API to integrate third-party AI models directly into Premiere Pro, expanding creative possibilities.
- ๐ Meta released a free, creative AI assistant that competes with chat GPT, offering real-time image generation and multimodal experiences.
- ๐ฌ AI film news highlighted several AI-generated films, showcasing the growing capabilities and creative potential of AI in the film industry.
Q & A
What is the name of the new tool released by Microsoft that has implications for entertainment and society?
-The new tool released by Microsoft is called Vasa 1. It can turn uploaded images and audio clips into realistic videos, with unique features such as controlling eye movement and projecting the same movement on different types of characters.
How does Adobe's new video in painting feature work in Premiere Pro?
-Adobe's video in painting feature allows users to create a mask and then prompt what they want to see in the masked area. For example, transforming a briefcase from having a few diamonds to having many diamonds inside.
What is the name of the free creative AI assistant released by Meta?
-The free creative AI assistant released by Meta is not named in the transcript, but it is described as a competitor to Chat GPT, integrated within the Meta Suite, including Instagram, Facebook, and WhatsApp, and accessible via a desktop application.
How does the generative expand feature in Premiere Pro work?
-The generative expand feature in Premiere Pro uses artificial intelligence to extend the length of a video clip. Users can select the AI model they want to use for expanding the clip, which can help in creating a more seamless and extended video sequence.
What is the significance of Adobe opening up their API for generative AI video in Premiere Pro?
-Opening up their API allows other tech companies to integrate their AI models directly into Premiere Pro. This means that users could potentially generate videos from platforms like Sora or Runway directly within the Premiere Pro interface, streamlining the creative process.
How does the real-time rendering feature of Vasa 1 impact the future of live-action entertainment and other sectors of society?
-The real-time rendering feature of Vasa 1 allows for immediate playback of the footage after uploading an image and audio clip. This has significant implications for live-action entertainment by potentially enabling real-time animation and storytelling. It could also impact sectors like therapy, where AI therapists could interact with patients in a seemingly human-like manner, or education, where AI tools and avatars could present information more engagingly.
What is the current limitation of Pik Labs for video in painting compared to Adobe Firefly video?
-While Pik Labs can be used for video in painting in a pinch, the fidelity of the results is not as high as that of Adobe Firefly video. Adobe's tool appears to offer more realistic and higher quality outcomes.
How does the object removal tool in Adobe's announcement compare to the capabilities of Runway?
-Adobe's object removal tool, which pairs with a content-aware feature for automatic rotoscoping, is similar to what can be done in Runway. Runway allows users to remove objects from footage and provides a real-time preview of the changes, which can be exported in various formats.
What is the potential issue with the generative expand feature when integrating AI video with live-action footage?
-The generative expand feature may result in a static and slow push-in effect when cutting from the original clip to the AI video, making it feel disjointed. The transition between live-action and AI-generated footage might not be seamless, potentially disrupting the narrative flow of a film.
What was the outcome of the AI creativity tournament held in Las Vegas?
-The AI creativity tournament in Las Vegas was attended by 450 people and featured competitors at the HyperX Esports Arena. The winner of the Apple Vision Pro and a custom crochet trophy from Curious Refuge was Jagger.
How does the new Meta AI tool compare to Chat GPT in terms of creativity?
-The Meta AI tool is about as creative as Chat GPT 3.5 and not as advanced as version 4. It can generate movie ideas and create real-time images, offering a multimodal experience that includes conversation and artwork generation. However, for professional creative assistance, Chat GPT is considered a better option.
Outlines
๐ Microsoft's Vasa 1: AI Video Tool with Real-Time Rendering
Microsoft has introduced Vasa 1, a groundbreaking AI tool that can transform images and audio clips into realistic videos. This tool is unique for its ability to control eye movements and project these movements onto various character types, which could revolutionize entertainment and advertising. Vasa 1 also allows for the creation of non-human animations in a very realistic manner, such as the Mona Lisa singing a song. It can render videos in real-time, which has implications for live-action entertainment and other societal applications, like AI therapy or education.
๐จ Adobe Premiere Pro's Upcoming AI Features
Adobe has announced new AI video tools for Premiere Pro, set to be released by the end of the year. These include 'video in painting', which allows users to create masks and transform objects within a video, such as changing a briefcase's contents or a watch face. Although these features are not yet available, workarounds using existing tools like Pik Labs are demonstrated. Adobe also showcased an object removal tool that performs like automatic rotoscoping, and a 'generative expand' feature to extend video clips using AI. However, the seamless integration of AI-generated content with live-action footage is still a challenge.
๐ Extending Video Clips with AI Tools
The video discusses the ability to extend video clips using AI, as demonstrated by Adobe's generative expand feature. The process involves exporting a frame from the last moment of a clip and using it as a starting point for AI to generate additional footage. This is showcased using Runway's image-to-video feature, which can extend a clip and make it appear quite realistic, although it may not be perfect for high-quality productions. The seamless integration of AI-generated footage with live-action is still a work in progress, with some examples showing inconsistencies.
๐ค Generative AI in Premiere Pro and Meta's AI Creativity
Adobe is set to include generative AI video directly in Premiere Pro, which will significantly speed up the creative process. They are also opening up their API to allow other tech companies to integrate their AI models into Premiere Pro. Meta has released a chat GPT competitor, which is integrated into various platforms and offers real-time image generation. It is compared to chat GPT, showing that while not as advanced, it offers a good free alternative. Meta's tool also provides a multimodal experience, allowing for text-based interactions and image generation, even though the quality is not as high as some niche tools.
๐ฎ AI Film News and Community Updates
The video concludes with updates on AI film news and community events. It highlights an AI creativity tournament held in Las Vegas, with 450 attendees, and the winner of the Apple Vision Pro competition. There is also a discussion about the potential for an AI creative E-sports league. The video showcases three AI films: 'Amy', which has a 1950s TV show aesthetic and tells a story using Dr. Seuss rhyming; 'The Fliff Atrre', an AI documentary about a puffball animal; and a project from PZF AI that creates convincing live-action shots. The host thanks the viewers and invites them to subscribe for more AI tutorials and news.
Mindmap
Keywords
AI Video Features
Premiere Pro
Vasa 1
Video In Painting
Object Removal Tool
Generative Expand
AI Therapist
AI Avatars
Adobe Firefly
Pika
Runway
Highlights
Microsoft released a new tool, Vasa 1, that can transform images and audio clips into realistic videos with controllable eye movements.
Vasa 1 allows for the projection of the same movement onto different character types, impacting entertainment and advertising.
The tool can create animations of non-human entities in a realistic manner, such as the Mona Lisa singing.
Vasa 1 provides 3D dynamics control, enabling more nuanced animation and storytelling.
The tool renders animations in real time, with potential applications in live-action entertainment and AI therapy.
Adobe announced new AI video tools for Premiere Pro, including video inpainting and object removal features.
Adobe's video inpainting allows users to modify specific regions of a video, like changing the contents of a briefcase.
Pika and Runway can be used to achieve similar video inpainting and object removal effects currently.
Adobe's generative expand feature extends video clips using AI, offering new creative possibilities.
Generative AI video within Premiere Pro will streamline the creative process by quickly generating shots.
Adobe is opening its API to integrate other tech companies' AI models directly into Premiere Pro.
Meta released a free creative AI assistant with real-time image generation and up-to-date information capabilities.
Meta's AI tool can generate movie ideas and artwork, providing a creative partner experience.
Researchers have developed a method to turn any video into a video game with interactive scanned environments.
AI Film News showcased a 1950s TV show style film called 'Amy' with a rhyming narrative.
The film 'The Fliff Atrre' is an AI documentary that creatively navigates AI limitations to tell a story.
PZF AI created a realistic AI project with convincing shots and sound design, resembling a live-action film.
AI creativity tournament in Las Vegas saw 450 attendees and the winner received a custom crochet trophy.
The discussion of developing an AI creative Esports league indicates the growing interest in AI and entertainment.