Singtel breach (2021) – case study

What happened Singtel?

Singtel, in a report, released a statement that they are currently investigating a data breach involving customer data. For those who aren’t familiar, Singtel is a Singapore based group of telecommunications companies around Asia, as well as a telco licensee in Singapore.

Singapore was notified by Accellion that the data breach occurred due to its file sharing system. The system was breached by unidentified threat actors (aka hackers). Singtel explains that it’s a standalone system and its used to share information within and with external parties.

Singtel explains that the use of Accellion FTA product was legitimate and had support running till April 2021. In mid-December 202, Accellion had issued a patch within 72 hours of the zero day notification. Accellion had noted attacks based on the reported zero days till end of January 2021.

What about Accellion?

Accellion, through its own website had a press release on the matter.

Interestingly, Accellion made a clear note that the product affected was a 20 year old “approaching end-of-life” product. Typical corporate sales techniques, Accellion uses this opportunity to urge its customers to migrate to its newer platforms. Interesting to note that Accellion makes it clear that the FTA platform is “legacy” and implies that, while the product is under support, organizations should either have migrated across to newer platforms or start doing so (preferring to “upgrade” to its own new version).

Analysis of the incident

Lets look at each part of this and the claims made by the respective organizations.

  1. The FTA system is a standalone system.

My assessment? True and False.

Lets look into the function of the FTA. Essentially its an FTP (file transfer protocol) server used for transferring files in and out of the organization. There seems to be some issues with this setup. Singtel further explains that the platform is used by both internal and external parties.

Did anyone notice a huge blinking red flag here? No? I’ll explain why.

In a typical telco setup, these FTP servers are crucial part of the equation. CDR (call data records) are often put into FTP servers before it gets passed to mediation and eventually billing and charging. Again, big red blinking light – CDR!!!

Why would file transfer be needed for external parties?

It’s used for many reasons, i’ll outline 2 as example. Firstly is bill payments. Some bill payments use REST API for immediate settlement, while others use bulk payment (aka batch) which uses file transfer via FTP. a bank may receive payments from respective customer and does update every night at 3am. Another scenario would an outsource arrangement involving a third party to perform corporate account provisioning, and then doing bulk activation based on the files provided.

Good hygiene practice, the file transfer platform should be completely separate and  isolated between internal and external parties.

Next, the question of whether the system is isolated or not. For me, an isolated system is a system that doesn’t have connectivity to any other systems, like a Windows 10 PC at home only connected to the internet. But a file transfer system? You can see that the system/network/security admins would have punched holes on the firewall in order for the system to be able to receive and transfer files. Yes, it is interconnected, but whether it can access the other interfaces (both ethernet and 3G specific) depends on what ports are open.

2. Usage of legacy platform.

This is where both parties seem to have differing views. Singtel seems to think that the product is supported (noting that EOL is around the corner), hence safe to use. Accellion however minces no words and blatantly put legacy tag to the platform.

Logical ensuring question – why didn’t Singtel migrate their platforms to a newer one? (This is the part where i throw theories into the equation, only Singtel would know the real reason)

Firstly, don’t fix what’s not broken. Remember it’s a 20 year old platform, and assuming that Singtel had used for half of it’s useful lifetime, that’s easily 10 years! The folks who provisioned and configured the platform may have moved on, or even retired! So, it works, it continue to work hence don’t touch!

A system migration can make or break a CIO/CTO’s career. We look back at statements made by Singtel. The FTA platform is used internal and external parties. This means firewall rulesets needs to be migrated. New service accounts need to be created. Permissions need to be mapped. Application ID’s need to be created. Batch jobs or cron jobs running in the server modified. God knows what else needs to be done! Now that’s just the internal parts. Minus the system, you’d have internal application owners screaming blood at you due to KPI missage!

The next big headache is coordinating initiatives with the external parties. I’ve had experience during migration where one of the external parties wanted to bill me for their migration! We, of course, declined politely and said that migrations are handled by individual organizations at their own cost (providing timelines to migrate across).

3. Why didn’t the patch work?

Singtel seem to indicate that the patch provided by Accellion didn’t work. Noting what Accellion mentioned, the patch was produced within 72 hours. One has to wonder if proper regression and quality checks were performed before patches were released. Reminds of Microsoft, who previously released a patch for a patch (in their credit, they’ve come a long way).

Conclusion

Tech debt is real, and in Singtel’s case just hit them with a huge interest. While one can argue its a zero-day issue, it is without a doubt that the legacy platform should have been managed out. Reminds me of the switch issue in MAHB? From a glance, seems like Singtel has lots of work ahead of them. They are moving in the right direction, I only hope they take a comprehensive look at their environment and not “scope down” into just the FTA.

Reference

  1. ZDNet: Singtel breach – https://www.zdnet.com/article/singtel-hit-by-third-party-vendors-security-breach-customer-data-may-be-leaked/
  2. Singtel Release: https://www.singtel.com/personal/support/about-accellion-security-incident
  3. Accellion Press Release: https://www.accellion.com/company/press-releases/accellion-provides-update-to-recent-fta-security-incident/
  4. MAHB Airport Case Study – http://www.drsuresh.net/2019/09/mahb-case-study-aug2019/
  5. Tech Debt – http://www.drsuresh.net/2019/08/cyber-tech-debt/

 

e-Pay data breach – a case study

Introduction

e-Pay is a solution part of GHL group of companies. Based on their website, e-Pay is was founded when Malaysia’s telco industry was just emerging in the late nineties. We have been providing top-up services ever since prepaid mobile plans became popular. Since our simpler beginnings, e-pay has expanded to include a host of other e-payment services, allowing us to drop our earlier moniker “One Stop Prepaid Reload” to adopt the now more accurate “One Stop E-Payment Service Provider”. Our network has grown fast, as we continue to build bridges between Product Partners, Merchants and Consumers across the entire nation, and now expanding out into Asia Pacific.

If you visit e-Pay’s website, it seems like business as usual. They have the promos on their website on their product and services. Everything seems normal. But is it?

On Feb 2021

@bank_security revealed that data breach occurred, revealing personal details of over 300,000 E-Pay customers exposed online. A threat actor was spotted selling a database of 380,000 customers on an data sharing forum for USD 300 (about RM1,215), which translates to be about 0.32 sen per user.

Further looking into the metadata reveals that a wealth of personal information was made available. The usual stuff (login, password) but what caught my attention was data of birth (?), nationality (?), which seems to indicate that the breach data isn’t just payment information, but seem to indicate information that is typically captured during a signup process.

These type of data are common in breaches. You’ll find past breaches having similar types of data being exposed.

What did e-Pay say?

TheEdgeMarket took a step further and contacted GHL on the matter. GHL released a statement saying that “currently investigating these serious allegations and are checking our system”. GHL asserts that the allegations are limited only to the e-pay online reload and bill payment collection system (E.V.E) and does not impact other e-pay and GHL businesses and operations

For a company that’s investigation a supposed breach, they seem to know exactly where the breach may have happened. Good sign of system awareness I suppose.

GHL adds further that the E.V.E system operates on an independent stand-alone system which does not interfere with the technical operations of other e-pay and GHL merchant acquiring systems and servers.

“Investigations are still underway and we will continue to update on the progress and any new findings. In the meantime, we would advise E.V.E users to go to our official website and change their passwords as precautionary measures.”

“E.V.E users should not click on unverified e-mail links urging them to update their credentials but to do so only on our official website,” it said.

Nothing out of ordinary, but i’m curious as to why a bill payment system needs data of birth or even nationality?

It would have seen passable as any other breach, but then a new piece of puzzle emerged.

What happened in 2020?

Kela, an organization that performs dark net monitoring issued a startling statement in their social media account.

According to Kela, based on the posting above, the breach data has been around for some time.  Specifically the same breach data has been published before, namely March 2020 and August 2020. The reporting on the breach by @bank_security was on February 2021.

The is raises some fundamental question about the whole incident.

  1. Was e-Pay aware that they’ve been breached at 2020?
  2. Was e-Pay  only made to be aware of the breach because of the recent reporting?
  3. By making a statement that only one system (EVE) was affected, how did they excluded others? It seems that either they know more about the breach, or trying to stem bleed from casting doubt on other systems.
  4. Will they release a full statement on this matter (have to wait and see)
  5. Instead of making a blanket statement, have GHL/e-Pay reached out to their customers regarding  this matter?
  6. As this is a PII breach, I’m wondering if PDPC (Personal Data Protection Commission)/MCMC (Communications and Multimedia Commission)  to issue a statement on this matter?

Conclusion

While this incident highlights the issue with one organization, it exposes a larger problem. There is no framework that compels organizations to publicly disclosure breaches. This is due to 2 factors. Lack of laws governing data breaches and shame factor. I’m reminded of an organization (believe was a financial) had huge IT meltdown causing their operations to be severely affected which was rumoured to be a ransomware attack but downplayed it as an IT glitch. After that, no further reporting or coverage was seen on that matter.

I lauded the efforts of FireEye for coming up front about Solarwinds attack. Malaysia needs to normalise breach reporting and notification, especially so when personal information and in this case payment information experiences data breach.

Reference:

  1. E-pay website – https://www.e-pay.com.my/
  2. SoyaCincau website – https://www.soyacincau.com/2021/02/04/e-pay-customer-database-breach-380000-sale-forum/
  3. The Edge Market: https://www.theedgemarkets.com/article/ghl-system-investigating-epay-data-breach-claims
  4. KELA – https://ke-la.com/