New Google Images tease the future of AI-generated movies with the DeepMind tool that even creates soundtracks for videos

The next generation of AI-powered videos is about to go public as Google announces a new tool that can automatically create unique soundtracks.

Several AI-generated video creators have impressed users for years, such as OpenAI’s Sora, Runway Gen-3 Alpha, and Luma AI’s Dream Machine.

3

3

But none of these magic makers have managed to generate a decent soundtrack to accompany the videos – until now.

Google announced the new video to audio tool to its DeepMind AI generator on Monday.

“Video generation models are advancing at an incredible pace, but many current systems can only generate silent results. One of the next main steps to bring the generated films to life is to create soundtracks for these silent videos,” Google wrote.

“Today, we’re sharing progress on our video to audio (V2A) technologywhich enables synchronized audiovisual generation.”

“V2A combines video pixels with natural images language text prompts to generate rich soundscapes for the on-screen action,” they explained.

The tool can be combined with video generation templates like Veo to create dramatic soundtracks that align perfectly with any scene.

The AI ​​will produce music that works with character dialogue and other tonal elements to create the right auditory atmosphere.

“It can also generate soundtracks for a variety of traditional footage, including archival material, silent films and more – opening up a wider range of creative opportunities,” said DeepMind.

Google shared impressive examples of the new technology in action, including clips of a Western-style soundtrack that accompanied a cowboy on horseback and a wild wolf howling at the moon.

COMPLETE CREATIVE CONTROL

Google’s new V2A tool will give creators the power to let AI generate a soundtrack based on the clip’s visual input and language instructions, or to create a soundtrack themselves.

Users can provide instructions and editing tips to the tool to guide its output in the desired direction.

One set of instructions read: “Audio request: cinematic, suspense, horror film, music, tension, ambiance, footsteps on concrete.”

The scene shows a man walking through a destroyed building before ending with a vision of the same man on a mysterious bridge.

The AI ​​creates a suitable soundtrack for the clip that matches the tone and pace of the narrative.

ENDLESS SOUNDTRACK OPTIONS

DeepMind’s V2A can also generate an unlimited number of soundtrack ideas.

An example prompt read: “Audio prompt: Spaceship hurtles through the vastness of space, stars passing by, high speed, science fiction.”

The video showed a spacecraft flying through the vast opening of space with the light of a star shining in the distance.

The first soundtrack generated by the V2A tool was an uplifting orchestral piece that matched the image and stimulus.

A second AI-produced soundtrack from the same prompt was darker and slower.

Using “Audio Prompt: Ethereal Cello Atmosphere” changed things even more.

This third soundtrack immediately established a sadder, more thoughtful tone.

JUST IMPROVING

Google said these updates were just the latest attempt to update its full suite of AI-generated content providers.

They hope to improve some issues in the next versions.

“Because the quality of the audio output depends on the quality of the video input, artifacts or distortions in the video, which are outside the model’s training distribution, can lead to a noticeable drop in audio quality,” Google said.

“We are also improving lip sync for videos that involve speech. V2A attempts to generate speech from the input transcripts and synchronize it with the characters’ lip movements.”

“But the paired video generation model may not be conditional on transcriptions. This creates a mismatch, often resulting in awkward lip syncing as the video model does not generate mouth movements that match the transcription,” they added.