Adobe's NEW AI Video Features are AMAZING! (Let’s Recreate Them)

This week's AI film news is packed with exciting updates from the tech world. Microsoft's Vasa 1 tool is set to revolutionize entertainment by converting images and audio clips into realistic videos, with the unique ability to control eye movements and project them onto different characters. Adobe's Premiere Pro is also unveiling new AI video features, including video painting, object removal, and generative expand, which will extend the capabilities of video editing. Meta's new AI assistant, available for free, is making waves with its creative and up-to-date responses, integrated across social platforms and offering real-time image generation. The news also covers an AI creativity tournament in Las Vegas, the potential for an AI creative Esports league, and a look at the latest AI films, showcasing the creative potential of AI in storytelling and film production.

Takeaways

• 🎥 Microsoft launched Vasa 1, a tool that transforms images and audio clips into realistic videos with controllable eye movements and character projections.
• 👀 Vasa 1's unique feature allows for nonhuman animations, such as the Mona Lisa singing, in a highly realistic manner.
• 🚀 The tool also enables 3D dynamics control, offering more flexibility in character orientation during animations.
• ⚡ Vasa 1 renders animations in real time, which could impact live-action entertainment and various societal applications like AI therapy.
• 🌟 Adobe announced new AI video tools for Premiere Pro, including 'video in painting' which allows for content transformation within the video editing software.
• 🧩 Pik Labs can be used as an alternative to Adobe's features, enabling video inpainting tasks with a simple interface.
• 🔍 Adobe's object removal tool, akin to automatic rotoscoping, was showcased with impressive results, though not without minor imperfections.
• 🎨 Runway offers a similar object removal feature that works in real time, providing a quick solution for video editing tasks.
• 📐 Adobe's generative expand feature extends video clips using AI, but the integration with live-action footage can be static and may require further refinement.
• 🔗 Adobe is opening its API to integrate third-party AI models directly into Premiere Pro, expanding creative possibilities.
• 🏆 Meta released a free, creative AI assistant that competes with chat GPT, offering real-time image generation and multimodal experiences.
• 🎬 AI film news highlighted several AI-generated films, showcasing the growing capabilities and creative potential of AI in the film industry.

Q & A

What is the name of the new tool released by Microsoft that has implications for entertainment and society?

The new tool released by Microsoft is called Vasa 1. It can turn uploaded images and audio clips into realistic videos, with unique features such as controlling eye movement and projecting the same movement on different types of characters.

How does Adobe's new video in painting feature work in Premiere Pro?

Adobe's video in painting feature allows users to create a mask and then prompt what they want to see in the masked area. For example, transforming a briefcase from having a few diamonds to having many diamonds inside.

What is the name of the free creative AI assistant released by Meta?

The free creative AI assistant released by Meta is not named in the transcript, but it is described as a competitor to Chat GPT, integrated within the Meta Suite, including Instagram, Facebook, and WhatsApp, and accessible via a desktop application.

How does the generative expand feature in Premiere Pro work?

The generative expand feature in Premiere Pro uses artificial intelligence to extend the length of a video clip. Users can select the AI model they want to use for expanding the clip, which can help in creating a more seamless and extended video sequence.

What is the significance of Adobe opening up their API for generative AI video in Premiere Pro?

Opening up their API allows other tech companies to integrate their AI models directly into Premiere Pro. This means that users could potentially generate videos from platforms like Sora or Runway directly within the Premiere Pro interface, streamlining the creative process.

How does the real-time rendering feature of Vasa 1 impact the future of live-action entertainment and other sectors of society?

The real-time rendering feature of Vasa 1 allows for immediate playback of the footage after uploading an image and audio clip. This has significant implications for live-action entertainment by potentially enabling real-time animation and storytelling. It could also impact sectors like therapy, where AI therapists could interact with patients in a seemingly human-like manner, or education, where AI tools and avatars could present information more engagingly.

What is the current limitation of Pik Labs for video in painting compared to Adobe Firefly video?

While Pik Labs can be used for video in painting in a pinch, the fidelity of the results is not as high as that of Adobe Firefly video. Adobe's tool appears to offer more realistic and higher quality outcomes.

How does the object removal tool in Adobe's announcement compare to the capabilities of Runway?

Adobe's object removal tool, which pairs with a content-aware feature for automatic rotoscoping, is similar to what can be done in Runway. Runway allows users to remove objects from footage and provides a real-time preview of the changes, which can be exported in various formats.

What is the potential issue with the generative expand feature when integrating AI video with live-action footage?

The generative expand feature may result in a static and slow push-in effect when cutting from the original clip to the AI video, making it feel disjointed. The transition between live-action and AI-generated footage might not be seamless, potentially disrupting the narrative flow of a film.

What was the outcome of the AI creativity tournament held in Las Vegas?

The AI creativity tournament in Las Vegas was attended by 450 people and featured competitors at the HyperX Esports Arena. The winner of the Apple Vision Pro and a custom crochet trophy from Curious Refuge was Jagger.

How does the new Meta AI tool compare to Chat GPT in terms of creativity?

The Meta AI tool is about as creative as Chat GPT 3.5 and not as advanced as version 4. It can generate movie ideas and create real-time images, offering a multimodal experience that includes conversation and artwork generation. However, for professional creative assistance, Chat GPT is considered a better option.

Outlines

00:00

🚀 Microsoft's Vasa 1: AI Video Tool with Real-Time Rendering

Microsoft has introduced Vasa 1, a groundbreaking AI tool that can transform images and audio clips into realistic videos. This tool is unique for its ability to control eye movements and project these movements onto various character types, which could revolutionize entertainment and advertising. Vasa 1 also allows for the creation of non-human animations in a very realistic manner, such as the Mona Lisa singing a song. It can render videos in real-time, which has implications for live-action entertainment and other societal applications, like AI therapy or education.

05:01

🎨 Adobe Premiere Pro's Upcoming AI Features

Adobe has announced new AI video tools for Premiere Pro, set to be released by the end of the year. These include 'video in painting', which allows users to create masks and transform objects within a video, such as changing a briefcase's contents or a watch face. Although these features are not yet available, workarounds using existing tools like Pik Labs are demonstrated. Adobe also showcased an object removal tool that performs like automatic rotoscoping, and a 'generative expand' feature to extend video clips using AI. However, the seamless integration of AI-generated content with live-action footage is still a challenge.

10:03

📚 Extending Video Clips with AI Tools

The video discusses the ability to extend video clips using AI, as demonstrated by Adobe's generative expand feature. The process involves exporting a frame from the last moment of a clip and using it as a starting point for AI to generate additional footage. This is showcased using Runway's image-to-video feature, which can extend a clip and make it appear quite realistic, although it may not be perfect for high-quality productions. The seamless integration of AI-generated footage with live-action is still a work in progress, with some examples showing inconsistencies.

15:04

🤖 Generative AI in Premiere Pro and Meta's AI Creativity

Adobe is set to include generative AI video directly in Premiere Pro, which will significantly speed up the creative process. They are also opening up their API to allow other tech companies to integrate their AI models into Premiere Pro. Meta has released a chat GPT competitor, which is integrated into various platforms and offers real-time image generation. It is compared to chat GPT, showing that while not as advanced, it offers a good free alternative. Meta's tool also provides a multimodal experience, allowing for text-based interactions and image generation, even though the quality is not as high as some niche tools.

20:07

🎮 AI Film News and Community Updates

The video concludes with updates on AI film news and community events. It highlights an AI creativity tournament held in Las Vegas, with 450 attendees, and the winner of the Apple Vision Pro competition. There is also a discussion about the potential for an AI creative E-sports league. The video showcases three AI films: 'Amy', which has a 1950s TV show aesthetic and tells a story using Dr. Seuss rhyming; 'The Fliff Atrre', an AI documentary about a puffball animal; and a project from PZF AI that creates convincing live-action shots. The host thanks the viewers and invites them to subscribe for more AI tutorials and news.

Keywords

AI Video Features

AI Video Features refer to the new capabilities being developed and integrated into video editing software that utilize artificial intelligence to enhance or automate the editing process. In the video, these features are highlighted as game-changers for the entertainment industry, with the potential to transform video editing through tools like object removal, video painting, and generative AI expansion of video clips.

Premiere Pro

Premiere Pro is a professional video editing software developed by Adobe. It is mentioned in the video as the platform that will soon include new AI video editing features. These features are expected to be available by the end of the year and are designed to streamline and innovate the video editing process.

Vasa 1

Vasa 1 is a new tool released by Microsoft that has significant implications for the entertainment industry and beyond. It can turn uploaded images and audio clips into realistic videos, with unique features such as controlling eye movement and projecting the same movement on different character types. This tool is showcased in the video as a significant leap in AI technology for video creation.

Video In Painting

Video In Painting is a feature that allows users to create a mask around an object in a video and then replace the masked area with a different image or scene. The video discusses how Adobe demonstrated this feature by transforming a briefcase from containing a few diamonds to many, and how similar results can be achieved using current technology from platforms like Pika.

Object Removal Tool

The Object Removal Tool is a feature that uses AI to automatically detect and remove objects from video footage, akin to rotoscoping but done automatically. Adobe's demonstration of this tool showed its potential for seamless editing, although the video also suggests that current alternatives like Runway can achieve similar results.

Generative Expand

Generative Expand is a feature that extends the length of a video clip using AI. It is mentioned as a tool that can be used to create longer, more detailed shots without additional filming. The video discusses how this feature can be previewed and customized in real-time, although it also points out that the results may not always be seamless or high-quality.

AI Therapist

An AI Therapist, as mentioned in the video, refers to the concept of using AI to simulate a therapeutic conversation in real-time. This application of AI technology is highlighted as an example of how Vasa 1's real-time rendering capabilities could be used beyond entertainment, indicating the potential for AI to impact various aspects of society.

AI Avatars

AI Avatars are digital representations or characters driven by AI that can interact with humans in various environments. The video suggests that AI technology like Vasa 1 could enable the creation of realistic AI avatars for purposes such as forming relationships or enhancing educational experiences.

Adobe Firefly

Adobe Firefly is a video example used in the script to illustrate the capabilities of Adobe's new AI video features. It is mentioned in the context of video in painting, where a briefcase's contents are transformed from a few diamonds to many, demonstrating the potential for AI to alter video content in creative ways.

Pika

Pika, or Pika.doart, is a third-party tool mentioned in the video that can be used to achieve AI video editing tasks such as video in painting. The video demonstrates how Pika can be used to modify video content, like changing the contents of a briefcase or the face of a watch, showcasing its utility as an alternative to Adobe's upcoming features.

Runway

Runway is an AI platform that enables users to perform tasks such as object removal and generative video creation. The video discusses how Runway can be used to remove unwanted objects from video footage and extend video clips, providing a real-time preview of the changes and offering a current solution while waiting for similar features in Premiere Pro.

Highlights

Microsoft released a new tool, Vasa 1, that can transform images and audio clips into realistic videos with controllable eye movements.

Vasa 1 allows for the projection of the same movement onto different character types, impacting entertainment and advertising.

The tool can create animations of non-human entities in a realistic manner, such as the Mona Lisa singing.

Vasa 1 provides 3D dynamics control, enabling more nuanced animation and storytelling.

The tool renders animations in real time, with potential applications in live-action entertainment and AI therapy.

Adobe announced new AI video tools for Premiere Pro, including video inpainting and object removal features.

Adobe's video inpainting allows users to modify specific regions of a video, like changing the contents of a briefcase.

Pika and Runway can be used to achieve similar video inpainting and object removal effects currently.

Adobe's generative expand feature extends video clips using AI, offering new creative possibilities.

Generative AI video within Premiere Pro will streamline the creative process by quickly generating shots.

Adobe is opening its API to integrate other tech companies' AI models directly into Premiere Pro.

Meta released a free creative AI assistant with real-time image generation and up-to-date information capabilities.

Meta's AI tool can generate movie ideas and artwork, providing a creative partner experience.

Researchers have developed a method to turn any video into a video game with interactive scanned environments.

AI Film News showcased a 1950s TV show style film called 'Amy' with a rhyming narrative.

The film 'The Fliff Atrre' is an AI documentary that creatively navigates AI limitations to tell a story.

PZF AI created a realistic AI project with convincing shots and sound design, resembling a live-action film.

AI creativity tournament in Las Vegas saw 450 attendees and the winner received a custom crochet trophy.

The discussion of developing an AI creative Esports league indicates the growing interest in AI and entertainment.