Meta, challenging OpenAI, announces new AI model that can generate video with sound

Meta, challenging OpenAI, announces new AI model that can generate video with sound
Meta AI logo is seen in this illustration taken on May 20, 2024.
PHOTO: Reuters

NEW YORK - Facebook owner Meta announced on Friday it had built a new AI model called Movie Gen that can create realistic-seeming video and audio clips in response to user prompts, claiming it can rival tools from leading media generation startups like OpenAI and ElevenLabs.

Samples of Movie Gen's creations provided by Meta showed videos of animals swimming and surfing, as well as videos using people's real photos to depict them performing actions like painting on a canvas.

Movie Gen also can generate background music and sound effects synced to the content of the videos, Meta said in a blog post, and use the tool to edit existing videos.

In one such video, Meta had the tool insert pom-poms into the hands of a man running by himself in the desert, while in another it changed a parking lot where a man was skateboarding from dry ground into one covered by a splashing puddle.

Videos created by Movie Gen can be up to 16 seconds long, while audio can be up to 45 seconds long, Meta said. It shared data showing blind tests indicating that the model performs favourably compared with offerings from startups including Runway, OpenAI, ElevenLabs and Kling.

The announcement comes as Hollywood has been wrestling with how to harness generative AI video technology this year, after Microsoft-backed OpenAI in February first showed off how its product Sora could create feature film-like videos in response to text prompts.

Technologists in the entertainment industry are eager to use such tools to enhance and expedite filmmaking, while others worry about embracing systems that appear to have been trained on copyright works without permission.

Lawmakers also have highlighted concerns about how AI-generated fakes, or deepfakes, are being used in elections around the world, including in the US, Pakistan, India and Indonesia.

Meta spokespeople said the company was unlikely to release Movie Gen for open use by developers, as it has with its Llama series of large-language models, saying it considers the risks individually for each model. They declined to comment on Meta's assessment for Movie Gen specifically.

Instead, they said, Meta was working directly with the entertainment community and other content creators on uses of Movie Gen and would incorporate it into Meta's own products sometime next year.

According to the blog post and a research paper about the tool released by Meta, the company used a mix of licensed and publicly available datasets to build Movie Gen.

OpenAI has been meeting with Hollywood executives and agents this year to discuss possible partnerships involving Sora, although no deals have been reported to have come out of those talks yet. Anxieties over the company's approach increased in May when actor Scarlett Johansson accused the ChatGPT maker of imitating her voice without permission for its chatbot.

Lions Gate Entertainment, the company behind The Hunger Games and Twilight, announced in September that it was giving AI startup Runway access to its film and television library to train an AI model. In return, it said, the studio and its filmmakers can use the model to augment their work.

ALSO READ: Meta releases early versions of its Llama 3 AI model

Source: Reuters

homepage

trending

trending
    Southern Taiwan shuts down ahead of arrival of Typhoon Podul
    Malaysian teachers caught vaping on school grounds may face fines, up to 2 years' jail
    'We both want the best for Singapore': Ex-NMP Calvin Cheng resolves differences with ex-SDP chairman over Gaza comments
    Lee Teng and wife welcome their rainbow baby
    Ninja Van cuts 12% of Singapore workforce after 2 rounds of layoffs in 2024
    Off-duty SCDF officer killed in Punggol crash; 15-year-old taken to hospital
    More than just a bad habit: How Kpods affect youths' physical and mental health
    Luxury items seized in $3b money laundering case handed over to Deloitte for liquidation
    'We have to be ready anywhere, anytime': Off-duty healthcare professionals from Singapore revive elderly man in JB
    Taylor Swift announces new album called The Life of a Showgirl
    Young Malaysian couple's first trip ends in tragedy after motorbike crashes on bend in Cameron Highlands
    Huang Zitao and Xu Yiyang to hold wedding in October with 100 fans in attendance

Singapore

Singapore
    • 'Proof of love between 2 nations': Malaysian man creates SG60 shirt to thank Singaporeans who helped him through hard times
    • Cigarette to blame? Tree in Jurong catches fire after exterminators reportedly remove beehive
    • Vers likely to be launched in next decade: Chee Hong Tat
    • 'A worrying trend': Speeding violations surge 45% in first half of 2025 compared to same period in 2024
    • 'Business has increased by 30%': Local drivers see improvement amid LTA enforcement against illegal ride-hailing services
    • Service on North East Line resumes
    • Singaporean woman deported from Malaysia
    • Singapore delivers ninth tranche of humanitarian aid to Gaza
    • PM Wong to deliver National Day Rally speech on Aug 17
    • More than 53,000 retail workers to see wage increase of at least $130 from Sept 1

Entertainment

Entertainment
    • 'My sweat seeped through the seams': Zhang Zetong on 'suffering' and working with new virtual technology for drama Perfectly Imperfect
    • 'I felt I would die if I closed my eyes': Ada Choi's husband Max Zhang recalls suffering heart attack in April
    • Jet Li's eldest daughter getting married
    • 'Fate is unstoppable': Michelle Chia weds real estate agent boyfriend in whirlwind marriage
    • 'I would be crying every day during lunch': Rebecca Lim recalls feeling inadequate in 1st acting role after childbirth
    • Noah Centineo to star in Rambo origins movie
    • Maluma stops concert to reprimand fan who brought along a baby
    • Tom Holland admits putting on his Spider-Man suit 'feels different this time'
    • Katy Perry shows off bruises and scrapes from her Lifetimes tour
    • Pixie Lott plays her 'last gig', due to deliver second child in early September

Lifestyle

Lifestyle
    • Singapore ranks top in Asia for work-life balance and 25th in the world, according to Remote study
    • Embracing Singlish as part of our identity: Paiseh for what?
    • One-Michelin-starred Restaurant Euphoria shutters, chef-owner looks to 'rethink the future' of his cuisine
    • I try 11 new Michelin Bib Gourmand 2025 eateries to see if they're worth the hype, here's my honest take
    • BYD Atto 2 electric compact SUV launched in Singapore
    • I've lived in Twin Vew for 4 years: What's it like living without an MRT station nearby
    • Even cheaper than Bali: 5 hidden Asian islands you (and your wallet) will love
    • 4 condo layouts and features buyers are moving away from in 2025
    • How to get your driving licence in Singapore - fast
    • 'Last' meals: How durian, chilli crab, and KFC bring comfort to the dying in Singapore

Digicult

Digicult
    • Slim, sleek, but slightly too short-lived: Samsung Galaxy S25 Edge review
    • World's best Dota 2 teams to compete for $1m prize pool in Singapore in November
    • Apple Maps brings 3D landmarks and road-level realism to Singapore
    • The best AI tutor for O-level subjects: ChatGPT, Gemini or The Wise Otter?
    • Vivo X Fold5: A foldable contender with a few class-leading surprises
    • Here's everything in GPT-5 that's new and different than OpenAI's previous AI models
    • Australia regulator says YouTube, others 'turning a blind eye' to child abuse material
    • ZipZap car subscription service launches in Singapore
    • Sony RX1R III brings back the compact full-frame but not the Sony playbook
    • China's Premier Li proposes global AI co-operation organisation

Money

Money
    • Up 4.3%: Singapore's economy grew in Q2 despite US tariff fears
    • Goh Cheng Liang, Nippon Paint billionaire and richest Singaporean, dies aged 98
    • StarHub buys rest of MyRepublic's broadband business in $105m deal; comes after Simba buys M1
    • Keppel to sell M1 unit's telco business to Simba for $1.43b
    • Singapore can deliver and thrive in a fragmented global economy: Morgan Stanley analysts
    • Over 70% of Ang Mo Kio's 4-room million-dollar resales in the past 3 years came from this project
    • DBS beats expectations with $2.82b net profit for second quarter, maintains 2025 outlook
    • Carro targets US IPO with over $3.8b valuation, sources say
    • US companies spending record amounts to protect executives as threats rise
    • Electric car-sharing firm BlueSG to wind down current operations on Aug 8

Latest

Latest
  • 'Cooked alive': Europe's wildfires hit tourism spots and forests
  • Mothers of Gaza hostages fear Israeli offensive will endanger their sons
  • Kenyan activist tries to block new Ritz-Carlton safari lodge opening
  • Democrats fret about which city may be next as National Guard gathers in US capital
  • White House says Trump-Putin meeting is a 'listening exercise'
  • Homeless who refuse to co-operate with Trump crackdown may go to jail, White House says
  • Israel bombards Gaza City; Hamas leader visits Cairo in bid to salvage ceasefire talks
  • Putin holds call with Kim, discusses US-Russia talks, Kremlin says
  • Mexico, under pressure from Trump, transfers 26 more cartel members to US

In Case You Missed It

In Case You Missed It
  • Man remanded after wielding knife, trying to snatch baby in Penang supermarket
  • Malaysia's border control agency gives ICA cake to mark SG60
  • Tourist in Hong Kong killed after cabby, 80, crashes into pillar outside hotel
  • 2 Malaysian men nabbed at Woodlands Checkpoint for allegedly smuggling drugs worth over $150k into Singapore
  • Parents reject $30k settlement from kindergarten in JB after son suffocates to death in school van
  • Pritam gets candid with kids’ questions on his worst subjects and favourite song in radio interview
  • Bro-code before go-mode: Meet the duo leading NDP 2025
  • LTA, Singapore bus operators reviewing Malaysia’s request to start services from JB at 4am
  • Part-time PHV driver who stopped suicide attempt among 38 recipients of MHA’s public spiritedness award
This website is best viewed using the latest versions of web browsers.