Initiative by IMDA, AI Verify Foundation tests AI accuracy, trustworthiness in real-world scenarios

Initiative by IMDA, AI Verify Foundation tests AI accuracy, trustworthiness in real-world scenarios
Earlier in 2025, IMDA and the AI Verify Foundation rolled out an initiative focused on real-world uses of gen AI to encourage the safe adoption of AI across various industries.
PHOTO: Reuters

SINGAPORE – Doctors at Changi General Hospital (CGH) are testing the use of generative artificial intelligence (GenAI) to summarise medical reports and provide recommendations on clinical surveillance.

But are these recommendations accurate?

Meanwhile, regulatory technology firm Tookitaki uses GenAI to investigate potential money laundering and fraud cases.

Are its findings trustworthy?

Earlier in 2025, the Infocomm Media Development Authority (IMDA) and the AI Verify Foundation rolled out an initiative focused on real-world uses of GenAI to encourage the safe adoption of AI across various industries.

The AI Verify Foundation is a not-for-profit subsidiary of IMDA that tackles pressing issues arising from AI.

Between March and May, 17 organisations across 10 different sectors – including human resources, healthcare and finance – had their GenAI applications assessed by specialist GenAI testing firms.

The findings were published on May 29, marking Singapore’s commitment to spearhead the development of global standards for the safe deployment of GenAI apps.

The Global AI Assurance Pilot, as the initiative is called, has allowed organisations to see how their GenAI applications perform under practical conditions, said Senior Minister of State for Digital Development and Information Tan Kiat How on May 29.

He was speaking on the last day of the Asia Tech x Singapore conference, held at Capella Singapore.

Clinical Associate Professor Chow Weien, chief data and digital officer at CGH, told The Straits Times that taking part in the initiative helped the hospital design a more robust and reliable way of testing its AI models.

“For example, we could assess whether our GenAI application was extracting the clinical information accurately from the doctor’s colonoscopy report, and if the application was providing the correct recommendation, in line with the clinical guidelines,” he said.

Tookitaki founder and chief executive Abhishek Chatterjee told ST the experience helped make the firm’s AI model more auditable and allowed the company to incorporate guardrails against AI hallucinations.

These are inaccurate or nonsensical results generated due to factors such as insufficient training data.

Read Also
'Safety labels' in AI apps to clearly state risks, testing in discussion: Josephine Teo
singapore
'Safety labels' in AI apps to clearly state risks, testing in discussion: Josephine Teo

While earlier initiatives had focused on the testing of AI models, the Global AI Assurance Pilot aimed to test the reliability of GenAI in real-world scenarios, said AI Verify Foundation executive director Shameek Kundu.

This is important as the information fed to AI can be flawed, he said, giving the example of poor-quality scans from a patient provided to a hospital’s AI.

The aim is to make the use of GenAI “boring and predictable”, to ensure the technology’s reliability for day-to-day use, he said.

In a statement, IMDA and AI Verify Foundation said the initiative also showed that human experts were essential at every stage of testing, from designing the right tests to interpreting test results.

While the technology may improve in the future, a human touch is still needed for now, said Mr Shameek.

“The technology is not good enough for us to blindly trust and say it’s working,” he said.

A report detailing the findings is available on AI Verify Foundation’s website.

In line with the pilot, a testing starter kit for GenAI applications has also been developed, serving as a set of voluntary guidelines for businesses that want to responsibly adopt GenAI.

“It draws on insights from the Global AI Assurance Pilot, tapping the experience of practitioners to ensure the guidance is practical and useful,” said Mr Tan.

He added that the kit includes emerging best practices and methodologies for testing GenAI applications, as well as practical guidance on how to conduct such testing.

The guidelines will be complemented by testing tools to help developers conduct these tests, which will be made progressively available via IMDA and AI Verify Foundation’s Project Moonshot, a toolkit targeted at AI app developers.

IMDA is conducting a four-week public consultation on the starter kit, which can be found online. The consultation will end on June 25.

Feedback can be e-mailed to aigov@imda.gov.sg with the e-mail header “Comments on the draft Starter Kit for Safety Testing of LLM-Based Applications”. 

Mr Tan also announced that AI Singapore (AISG) – a national initiative to build the Republic’s capabilities in AI – will sign a memorandum of understanding with the United Nations Development Programme (UNDP) to advance AI literacy in developing countries.

This partnership will see AISG’s AI for Good programme, launched in 2024 to bolster national AI capabilities, expand to an international scale, he said.

“AISG and UNDP will explore initial AI for Good pilots in South-east Asia, the Caribbean and the Pacific Islands, so that we can support more inclusive participation in AI-driven growth together,” he added.

Read Also
digicult
Embracing the future: A personal guide to how to get started with artificial intelligence

This article was first published in The Straits Times. Permission required for reproduction.

homepage

trending

trending
    Vape disposal bins available at 24 locations; users who surrender devices will not face penalty
    Motorcyclist, 23, dies in accident involving lorry near Woodlands Checkpoint
    Ghim Moh hawker responds to 'unfair' one-star rating over tray incident, calls reviewer 'petty'
    'It was not how I planned to be a dad': Terence Cao speaks about fatherhood
    Long-time tech executive and Microsoft Singapore managing director Lee Hui Li dies
    Etomidate found in blood of 2 people involved in fatal Punggol Road accident in May: HSA
    Nasi padang chain Hjh Maimunah to introduce colour-coded price labels
    Raymond Lam takes selfies with fans while touring Singapore with family
    'I'm so angry': Christopher Lee shares thoughts about son Zed sharing bed with Fann Wong
    'It was the right thing to do': SBS bus captain returns missing wallet containing $2,400, work permit to distressed passenger
    'Fella really enjoying': Monitor lizard chows down on stingray at MacRitchie Reservoir
    'He asked why I was so selfish': Kym Ng recalls emotional conversation with husband about afterlife plans

Singapore

Singapore
    • Ex-inmate featured in documentary on Changi's maximum security prison returns to court to face fresh charges
    • Teenager caught with vapes 5 times sentenced to 2 years at Singapore Boys’ Home
    • Singapore-registered cars must have COE of over 6 months to apply for VEP, says Malaysian transport minister
    • 'Please help us seek the truth': Sister pleads for eyewitnesses after brother dies in accident on CTE
    • Company director fined for collecting $112k in kickbacks from migrant workers
    • Khatib Camp to make way for housing, with its functions moving to Amoy Quee Camp
    • 1,300 names, addresses of motorists published online; police investigating
    • 7 weeks' jail for male cook who molested drunken man sleeping on bench at shopping mall
    • Male victim of fatal Toa Payoh fire was known to keep many things, say residents
    • Police warn of scammers impersonating staff from National Crime Prevention Council

Entertainment

Entertainment
    • Shinee's Taemin wants to 'create new memories' with fans at Waterbomb Singapore 2025
    • 'I want to be an independent senior': Xiang Yun on ageing positively and not relying on children
    • Chinese actor Zhang Yiyang revealed to have been executed for murder of 16-year-old girlfriend
    • Coldplay might have exposed tech CEO's alleged affair during concert
    • Cai Xukun, I-dle, Jam Hsiao: Singapore concert calendar for 2025
    • Florida police launch medical investigation into Hulk Hogan's death
    • Vanessa Kirby on filming Avengers movie while pregnant: 'The coolest thing'
    • Trump, McMahon and Undertaker pay tribute to Hulk Hogan
    • Grammy-winning jazz musician Chuck Mangione dies at 84
    • Hulk Hogan, who helped turn pro wrestling into billion-dollar spectacle, dies at 71

Lifestyle

Lifestyle
    • Wedding ang bao rates in Singapore (July 2025): How much should you give?
    • From Paris to Raffles Place: Singaporean-founded deli Nonette opens pop-up, here's what you should try
    • Digital nomad in JB: Can Singaporeans really work from across the Causeway?
    • Books Kinokuniya's new outlet at Raffles City opens, features reading room and outlet-exclusive items
    • Prostate cancer among most common cancers in Singaporean men - is it preventable?
    • New glamping experience in Mandai starts in August, here's what Colugo Camp has to offer
    • 'How dare you': YouTubers call out London stall for selling 'fake Malaysian food'
    • Queen Elizabeth's fashion looks to go on display in new London exhibition
    • Cat A COE prices remain unchanged in second bidding exercise for July 2025
    • Singaporeans continue to hold world's most powerful passport in latest ranking

Digicult

Digicult
    • Slim, sleek, but slightly too short-lived: Samsung Galaxy S25 Edge review
    • World's best Dota 2 teams to compete for $1m prize pool in Singapore in November
    • Elon Musk's Starlink network suffers rare global outage
    • Spy cockroaches and AI robots: Germany plots the future of warfare
    • 'Give a positive review': Hidden AI prompt found in academic paper by NUS researchers
    • 'Report 1 shop, another 10 appear': Hoyo Fest artists on copyright struggles
    • NTU penalises 3 students over use of AI tools; they dispute university's findings
    • Australia social media teen ban software trial organisers say the tech works
    • Disney, Universal sue image creator Midjourney for copyright infringement
    • Initiative by IMDA, AI Verify Foundation tests AI accuracy, trustworthiness in real-world scenarios

Money

Money
    • Up 4.3%: Singapore's economy grew in Q2 despite US tariff fears
    • Britain and India sign free trade pact during Modi visit
    • HDB launches 10,209 BTO and balance flats, as priority scheme for singles kick in
    • US-Philippines trade talks yield modest tariff shift after Trump-Marcos meeting
    • Indonesia to cut tariffs, non-tariff barriers in US trade deal
    • US, China to discuss tariff deadline extension as Trump reaches Philippines deal
    • Trump says trade deal struck with Japan includes 15% tariff
    • 10 best savings accounts in Singapore with the highest interest rates (July 2025)
    • US Senate passes aid, public broadcasting cuts in victory for Trump
    • Beneath China's resilient economy, a life of pay cuts and side hustles

Latest

Latest
  • Daily roundup: Nasi padang chain Hjh Maimunah to introduce colour-coded price labels — and other top stories today
  • Philippine Supreme Court voids impeachment complaint against VP Duterte
  • Thailand rejects international mediation to end fighting with Cambodia
  • YouTube generation propels Japan's anti-foreigner politics into the mainstream
  • Hong Kong issues arrest warrants for 19 overseas activists, offers bounties for 15
  • School roof collapse in India's Rajasthan kills 4 children, media reports
  • Why are Thailand and Cambodia fighting along their border?
  • Thailand, Cambodia exchange heavy artillery fire as fighting rages for second day
  • 9 people killed in bus crash in Vietnam

In Case You Missed It

In Case You Missed It
  • Toddler wanders out of home in Selangor, mauled by stray dogs
  • 'I was embarrassed': Malaysian security guard in viral knockout by MMA coach 'thankful' he wasn't fired
  • Mid-air brawl erupts on AirAsia X flight from KL to Chengdu over loud conversation
  • Robber drops gun and misfires after failed clinic robbery in JB
  • $30 one-off cash handout, petrol price cut: Malaysian PM Anwar Ibrahim announces measures to tackle living costs
  • 'I felt helpless': Female tourist claims she was sexually harassed by ice cream vendor in Turkey
  • Unhealthy air quality in Malaysia, NEA warns of potential transboundary haze
  • School van in JB overturns after hitting uncovered manhole, 16 students injured
  • Thai woman allegedly extorts $15m from senior Buddhist monks over 3 years, gambles away almost everything
This website is best viewed using the latest versions of web browsers.