International Fact-Checking Day: How an experimental Google app lets newsrooms bust fake images

April 1 may be a day where we can close an eye to fraudulent claims and charlatan stunts, but the day after is when things get super serious about truthiness. Today (April 2) is, after all, International Fact-Checking Day. 

Checking the veracity of statements is especially critical at this point in time while the world grapples with a constant stream of information regarding a deadly global pandemic. Compounded, of course, by an upcoming general election in Singapore during a time when deliberate fake news might be engineered to influence voting results. 

But as regular internet users and journalists have found out, it’s not that easy and straightforward to determine what is real and what is not when it comes to deciphering images online. It’s for this reason that Source was created, an app that Google created together with fact-checking organisation Storyful which can analyse the data of any image file, including its origins, presence of manipulation, and even translate the text within. 

The app itself was conceptualised during a design sprint (a technique often used at Google to build products and solve problems) as part of the 2018 Trusted Media Summit run Singapore that brought together journalists, fact-checkers, technologists —including user interface experts  — and user research experts from Singapore newsrooms. 

After one and half days, a working prototype was built after a consultation process with journalists and fact-checkers while incorporating different technologies such as Cloud Vision API (which offers powerful image analysis via machine learning).

The project was taken further with help from Storyful to develop Source, which currently remains in alpha testing phase, though interested journalists and fact-checkers can sign up to test the app here. 

AsiaOne spoke to Google News Lab Lead for Asia Pacific Irene Jay Liu to get more verified facts about the experimental tool as well as the threats that Singapore and the rest of the region could face if misinformation goes unchecked. 

---

HOW DOES SOURCE WORK?

Irene (I): Source uses Google's AI technology to give instant access to an image's public history, allowing you to sort, analyze and understand its provenance, including any manipulation.

That’s already useful but it goes a step further. Source helps detect and translate text in images too, which is especially useful for journalists cataloguing or analyzing memes online.
 
If a journalist or fact-checker has a photo they want to verify, what they will do is: 

• Take the photo, upload it into the source app. 
• They can rotate the image, flip it, crop it to remove the random text in the picture and focus on only the image after they have narrowed it down to the portion they wish to verify. 
• Press analyze and the app will search for appearances of that image - whether it is an exact copy or a partial match. 

What's really exciting about this app is that it not only returns appearances of a photo on other websites, it will also extract the text from the image should you need to copy and paste it. 

We worked with @Storyful to build an AI-powered app to better detect manipulated images and counter misinformation. Hear more about it here: https://t.co/mZLQ2EpWpw pic.twitter.com/RL6bC0bCtf

— Google News Initiative (@GoogleNewsInit) February 26, 2020

The app can also do live translation, which is helpful for the multilingual world we live in. This is essential as one thing we found with misinformation is that a video that was filmed in Thailand will spread to Spain and vice versa, so having that multilingual capability is important in ensuring that misinformation is curbed on a larger scale as well. 

WOULD AN AI-DRIVEN PROCESS BE COMPLETELY FOOLPROOF WHEN IT COMES TO VERIFYING FAKE IMAGES?

I: Google’s AI technology does allow for identical and partial matches to images to help you get some of the clues on its origins. But as any fact-checker will tell you, this is only one step of many. News verification is detective work and Source is often only one of many tools that is used to ensure the accuracy of an image during the fact-checking process. 

If the image has been heavily manipulated, journalists will often need to go to other tools to do more reverse image searches on other platforms as well. 

WHY IS THE DETECTION OF FAKE IMAGES PARTICULARLY IMPORTANT IN ASIA?

I: Around the world, images and memes flood our feeds and chat groups. Social media is a visual medium, and it is just too easy to share on impulse or “just in case it is true.”  Bad actors know this and bet on people’s emotions — whether it is their biases or their goodwill. They’re willing to exploit the reach of social platforms and pollute conversations with false facts and narratives, including extremist content. 

In Asia, where there are eight times more social media users than in North America, the issue of misinformation is magnified. While there are existing tools that Asian journalists can use to discover the origins and trustworthiness of news images, they’re relatively old, inconsistent and for the most part only available on desktop. That’s a barrier for fact-checkers and journalists in countries where most people connect to the internet on their mobile. 

HOW ARE MEMES PLAYING A PART IN THE SPREAD OF MISINFORMATION?

I: We see memes a lot nowadays on social media — while they may be funny, they can sometimes also be a source of misinformation as they often provide exaggerated accounts of an issue or event. They often incorrectly attribute quotes to certain individuals and include text that apparently describes the photo.

WHAT ARE THE SPECIFIC DANGERS OF MISINFORMATION FOR THE MARKET IN SINGAPORE THESE DAYS?

I: The types of misinformation in Singapore is synonymous to those that we are observing globally - this can range from stories with false facts and narratives created by bad actors, out-of-context images and videos, to seeing ‘shallow fakes’ online, which are very simple image manipulations such as flipping a photo, adding a person in, taking out a person - things that are quite simple and crude. The other issue is that people often get fooled by photos and videos that are used out of context. For example, there could be a video of a flood in my neighbourhood today, but the video was actually from three years ago.

To prevent this, Google’s algorithms ensure that credible information and authoritative news sources are weighted more heavily when people search for information. For example, in the face of the current global pandemic, we connect people with accurate and updated information related to Covid-19 through our Knowledge Panel in Search and YouTube, surfacing the latest news plus safety tips and links to more authoritative information from the World Health Organization (WHO) and Singapore's Ministry of Health.

WHO ARE PARTICULARLY VULNERABLE TO ONLINE MISINFORMATION IN SINGAPORE?

I: Everybody is increasingly getting their information online and has a chance of encountering misinformation. Beyond having newsrooms generate accurate and authoritative news, fact-checking should also be something that every individual actively practises in their everyday lives - being aware of how to verify information online goes a long way in ensuring that we do not fall victim to fake news. 

ASIDE FROM BEING A HUGE HELP IN NEWSROOMS, HOW WOULD SOURCE BE ABLE TO BENEFIT EVERYONE ELSE?
 
I: For regular users, unfortunately, they won’t be able to access the app. It really is built for journalists at the moment, and meant to be part of their sleuthing process. What we are learning from this app and from fact-checkers who are using it is that there is potential to apply some of this technology into a tool for regular people who just want to know more about what they're seeing online. Our intention is to make this open-source which means that anyone who wants to purpose-build a tool for different audiences can do it. 
 
One of the other things that I do in my work at the Google News Lab and at Google is to focus on news literacy and media literacy among users, educating on the ways to source for trusted news. This includes using Google’s existing tools that give users access to accurate, and useful information such as the fact check label and knowledge panels that provide more useful context for people as they explore information online, or turning to YouTube’s Top News and Breaking News shelves to view relevant videos from trusted sources. 

ilyas@asiaone.com