Singtel fined $1m for outage that lasted over 4 hours, disrupted calls to emergency services

Singapore's largest telco Singtel has been hit with a $1 million fine for a nationwide landline outage that disrupted calls to public institutions and emergency services on Oct 8, 2024. 

The Infocomm Media Development Authority (IMDA) said on Thursday (Dec 11) that it has concluded its probe into the incident that affected about 500,000 of Singtel's residential and corporate users. 

For about four hours that day, about half of the calls made to public service hotlines — including government agencies, banks and hospitals — could not be connected. The police and Singapore Civil Defence Force had urged the public to contact them via SMS instead. 

IMDA said the disruption had the potential to seriously impact the safety and security of the public. 

It also said the incident was within Singtel's control to prevent and was not caused by a cyberattack. 

The maximum fine of $1 million was imposed after the regulator considered the scale and impact of the disruption, and the time taken to restore the voice network. 

What happened

At about 2pm on Oct 8, 2024, online reports about the landline outage surfaced. Singtel said at 5pm that it was progressively restoring fixed-line voice services, and updated at 8.25pm that full service had been restored. 

IMDA on Thursday said its investigations found that the telco hosted the virtualised firewalls of its fixed-line voice system as well as the monitoring system for home broadband routers and Pay TV set-top boxes on the same hardware. 

When the monitoring system's virtualised firewall experienced a surge in traffic intensity on Oct 8, 2024, the hardware memory resources that both firewalls shared became overwhelmed. 

This caused the virtualised firewall of the voice system to malfunction. 

"Voice traffic from the affected voice system should have been fully and seamlessly redirected to a separate voice system at an unaffected site through an automatic failover mechanism," said the infocomm regulator. 

But the failover was partly disrupted because of the intermittency of the affected voice system's virtualised firewall, causing calls to be dropped intermittently as voice traffic alternated between the voice systems. 

The outage was fixed after Singtel fully swung all voice traffic to the unaffected voice system. 

The telco has since implemented separate hardware for its voice system and monitoring system, and an intervention mechanism to stop network traffic from alternating between its systems during failovers. 

IMDA said telcos that are key service providers are held to a high level of service reliability. 

It has also required other key service providers to conduct checks on their systems to avoid similar configuration weakness and to remediate where necessary. 

"IMDA will not hesitate to take strong action under the Telecommunications Act, including imposing financial penalties, should any lapses be identified," it added. 

[[nid:723095]]

lim.kewei@asiaone.com