Award Banner
Award Banner

ChatGPT-maker OpenAI says it is doubling down on preventing AI from 'going rogue'

ChatGPT-maker OpenAI says it is doubling down on preventing AI from 'going rogue'
Sam Altman, CEO of Microsoft-backed OpenAI and ChatGPT creator listens to Ilya Sutskever, co-founder and Chief Scientist of OpenAI during a talk at Tel Aviv University in Tel Aviv, Israel on June 5.
PHOTO: Reuters

ChatGPT's creator OpenAI plans to invest significant resources and create a new research team that will seek to ensure its artificial intelligence remains safe for humans - eventually using AI to supervise itself, it said on Wednesday (July 5).

"The vast power of superintelligence could ... lead to the disempowerment of humanity or even human extinction," OpenAI co-founder Ilya Sutskever and head of alignment Jan Leike wrote in a blog post. "Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue."

Superintelligent AI - systems more intelligent than humans - could arrive this decade, the blog post's authors predicted. Humans will need better techniques than currently available to be able to control the superintelligent AI, hence the need for breakthroughs in so-called "alignment research," which focuses on ensuring AI remains beneficial to humans, according to the authors.

OpenAI, backed by Microsoft, is dedicating 20 per cent of the compute power it has secured over the next four years to solving this problem, they wrote. In addition, the company is forming a new team that will organise around this effort, called the Superalignment team.

The team's goal is to create a "human-level" AI alignment researcher, and then scale it through vast amounts of compute power. OpenAI says that means they will train AI systems using human feedback, train AI systems to assistant human evaluation, and then finally train AI systems to actually do the alignment research.

AI safety advocate Connor Leahy said the plan was fundamentally flawed because the initial human-level AI could run amok and wreak havoc before it could be compelled to solve AI safety problems.

"You have to solve alignment before you build human-level intelligence, otherwise by default you won't control it," he said in an interview. "I personally do not think this is a particularly good or safe plan."

The potential dangers of AI have been top of mind for both AI researchers and the general public. In April, a group of AI industry leaders and experts signed an open letter calling for a six-month pause in developing systems more powerful than OpenAI's GPT-4, citing potential risks to society. A May Reuters/Ipsos poll found that more than two-thirds of Americans are concerned about the possible negative effects of AI and 61 per cent believe it could threaten civilisation.

ALSO READ: OpenAI rolls out 'incognito mode' on ChatGPT

Source: Reuters

homepage

trending

trending
    From being behind bars to taking the bar: Former drug offender now goes to law school and also runs 2 businesses
    Singapore close to concluding negotiations with Asean partners for upgraded free trade agreement: DPM Gan
    K-pop idols IU, Enhypen's Jungwon and Seventeen join in on cat shadow trend
    'Forever missed. Forever loved': Sister of Singaporean boy who died on Mount Kinalabu finishes climb on his behalf
    'Life without drugs is possible': Singapore commemorates second Drug Victims Remembrance Day
    Heiress Kim Lim 'disappointed' after finding Chanel bag she sold for charity listed on Carousell
    Good Samaritans rush to assist motorcyclist after accident in Woodlands
    Adventure camps, wildlife interactions: A first look at Mandai's ZooSchool, opening in June
    'His criticism has no effect on me': Quan Yi Fong responds to Addy Lee's accusations
    New campaign to urge S'pore home, business, building owners to set air-cons at 25 deg C
    Over 17,900 people caught for use and possession of vapes since January 2024
    Fluff Stack, popular souffle pancake chain, to close all outlets in Singapore due to 'challenging F&B climate'

Singapore

Singapore
    • Singapore's first female DPM? Analysts expect PM Wong to make bold moves in new Cabinet
    • Singapore and Changi cannot be complacent, says PM Wong during groundbreaking ceremony of Terminal 5
    • Fatal Second Link accident: Singaporean pleads not guilty to dangerous driving, lawyers say he lost control of Maserati
    • Punggol cat death: Town council, NParks and animal welfare groups will work to raise awareness, says DPM Gan
    • Daily roundup: Over 17,900 people caught for use and possession of vapes since January 2024 — and other top stories today
    • Gillman Barracks restaurant fined $850 for food safety breaches after 40 gastroenteritis cases reported
    • Singapore must remain 'key node' in global supply chains amidst uncertainty: Chee Hong Tat
    • SIA rewards staff with over 7 months' bonus after record $2.8b full-year profit
    • DPM Heng hopes to spend time with family, travel, 'not do any work' in first year of retirement
    • 1,000 flats at former Keppel Club golf course to be offered in October BTO exercise

Entertainment

Entertainment
    • CL, BabyMetal, Foo Fighters: Singapore concert calendar for 2025
    • Director of K-drama Nine Puzzles 'pulled strings' to get these famous actors to cameo in the show
    • Lee Do-hyun and Monsta X's Hyungwon complete military service, Cha Eun-woo speculated to enlist soon
    • Taiwanese comedian Nono found guilty of attempted rape, sentenced to 2 1/2 years' jail
    • Singer Chris Brown denied bail in UK, throwing upcoming tour into doubt
    • Blake Lively allegedly asked Taylor Swift to delete their text messages in ongoing legal battle with Justin Baldoni
    • Olivia Rodrigo carries cheese in handbag as reminder of childhood
    • Singer Chris Brown charged over alleged assault at London club
    • Sean 'Diddy' Combs' chief accuser challenged over abuse claims, 'Freak Offs'
    • 'I found blood all over my body': Hong Kong former actor Wong He reveals being sexually assaulted twice in 2024

Lifestyle

Lifestyle
    • Ghib Ojisan opens up about birth of baby girl, taking on confinement nanny role to care for wife: 'I want to be there for her'
    • 'A new chapter begins': 8 local indie bookshops unite to launch one-stop online platform
    • Chicken Supremo owners retiring after 34 years, hawker stall to continue under new owner
    • Sizzling exhibits, games and freebies: McDonald's launching first McSpicy Museum at Bugis Junction
    • What to do this weekend (May 16 to 18)
    • Swensen's wedding? Restaurant's buffet concept to open in the west with space for large-scale event hosting
    • Spring in full bloom: Festive fun for all ages in Hong Kong
    • Battle of Middle East budget airlines: Which ones are worth it?
    • The ultimate work-from-home homebuyer checklist (that most people still overlook)
    • 6 inspiring local mum-preneurs in celebration of Mother's Day

Digicult

Digicult
    • A $500 wake-up call: How the Samsung Galaxy Ring made me realise my stress
    • Monster Hunter Wilds producer explains how game has remained unique and fresh over 20 years
    • World's best Dota 2 teams to compete for $1m prize pool in Singapore in November
    • Google Pixel 9a: The best AI-centric phone under $800 in 2025?
    • Western intelligence agencies warn spyware threat targeting Taiwan, Tibetan rights advocates
    • Taiwan says China using generative AI to ramp up disinformation and 'divide' the island
    • Russian court fines Telegram app for refusal to remove anti-government content, TASS reports
    • One Beijing man's quest to keep cooking — and connecting with Americans — on camera
    • Nintendo Switch 2 to launch in June with US$449.99 price tag
    • Games in April: RPGs, racing and Ronaldo in a fighting game

Money

Money
    • Wall Street equity indexes close higher after US-China tariff truce
    • Giant deal: Malaysian company to acquire Cold Storage and Giant supermarket chains in Singapore
    • 6 prime HDB shophouses for sale at $73m in Singapore: A look inside the rare portfolio
    • Croatia's Game of Thrones filming sites face threat from Trump tariffs
    • A new high in Choa Chu Kang: $690k for a 4-room flat
    • Apec warns of tariff impact on trade as members seek deals with US
    • Family of Koufu Group founders to buy Caldecott Hill GCB site for $58m
    • This US-owned factory in China made toys for Walmart. Tariffs put it on life support
    • Are you paying more than you should with dealer financing?
    • Best credit card promotions in Singapore (May 2025): Citibank, DBS, HSBC, UOB and more

Latest

Latest
  • Ukraine calls on allies to keep pressure on Russia after talks yield no ceasefire
  • North Korea's Kim Jong Un oversees air drills, calls for stepped-up war preparation
  • Vietnam steps up talks with US to reduce hefty tariff
  • Judge says Trump administration not detailing efforts to return wrongly deported man
  • South Korea ex-President Yoon leaves conservative party as candidate trails liberal frontrunner
  • Bird flu in top chicken exporter Brazil triggers trade bans
  • In Mexico, first outrage, then victim blaming over murdered TikTok influencer
  • UN aid chief: 'Let's not waste time' with US-backed Gaza aid plan
  • US aid cuts leave food for millions mouldering in storage

In Case You Missed It

In Case You Missed It
  • 'Dog will return soon': GE2025 independent candidate Jeremy Tan wants to contest again
  • Ong Ye Kung leads PAP team to victory while elder brother Howard Ong loses in Australia's election on the same day
  • Tan Kiat How weighs in on viral video of Gan Kim Yong being ignored by passers-by in Punggol
  • PSP's Tan Cheng Bock turns 85; SDP's Paul Tambyah joins celebration at Teban Gardens
  • PM Wong urges voters to 'choose leaders of good character' in PAP's first party political broadcast
  • It is 'important for Singapore's democracy' that WP wins more seats, says Pritam in election broadcast
  • GE2025: PSP, RDU, SDP, PPP, PAR, NSP promise to push for policy changes if elected to Parliament in first political broadcast
  • 'Everyone has the right to express their feelings': WP candidates address four-cornered fight in Tampines GRC
  • PAP's Desmond Lee responds to opposition's calls for GST exemption, says 'we want to make it progressive'
This website is best viewed using the latest versions of web browsers.