Singapore to propose Infosec tech rating – a review

BlackHat Asia recently hosts Singapore’s Deputy Chief Executive Brigadier General Gaurav Keerthi. Gaurav Keerthi was speaking on Singapore’s initiative for a voluntary “Cybersecurity Labelling Scheme” that is aimed at rating consumer’s broadband gateway.

In his speech, Gaurav Keerthi draws parallel between the importance of public utility such as water supply & sewerage, focusing on the aspects of fresh water and ensuring good governance. This is crucial for public health and safety. On this note, he equates infosec/cyber security to having access to clean water and proper sewerage. He opines that the citizens can be “scolded”  into better behavior.

Today, Singapore already offers Singpass to its citizens, an authentication/authorization services which allows the citizens to access services in a secured manner. This service is also offered to corporatations such as banks and financial sector to prevent having islands of authentication services.

To make the Cybersecurity Labelling Scheme wholesome, Gaurav Keerthi explains that it will expanded to connected devices. The first phase would be focused on ISP provided gateways and smart hubs, using a 4 star rating. Details on how the devices will be rated is going to be released during Singapore International Cyber Week which starts on October 5th, 2020. Being the Asian leader, Gaurav Keerthi mentions that Singapore plan to share the labelling scheme to other countries in the notion of public good.

My take on the matter

It’s a laudable effort from Singapore, taking a forefront on consumer devices. It’s no secret that consumer devices have wrecked havoc due to poor security. Cheap embedded devices such as IP CCTV Cameras have been known to cause issues, and was primed at source of Mirai botnet.

Firstly, lets understand why the choice of smart hubs and ISP gateway as a first stab into the scheme. ISP gateways are the most common and necessity for a household to get internet access. The device is either provided by the Service Provider as a bundle, or purchased by the end user. Today, these devices, from a physical perspective will be required to go through a type approval, which is a mandatory process. Such processes are for example CE marking from EU to ensure interoperability and ensure no conflict of operations against other devices while conforming to the requirements such as operating frequency/spectrum. Some might even say this will be a similar attempt, but from the cyber security perspective.

Analysis and Questions

To dissect this further, and raise some pertinent questions about the implementation of the Cybersecurity labelling scheme, I have grouped it into 8 categories.

Devices

Physical devices are often manufactured by OEM and sold by ISP. Often, we see that OEM also sells these devices in the open market. Bundled packages would include devices being thrown in as a freebie as part of the offerings. These devices have a lifetime, and some ISPs do not refresh these devices after its End-Of-Life, but at times position themselves to offer new package in order for the consumer to get new devices. Consumer may opt for a new package, or be happy with the current offerings and only choose to update the devices by themselves by purchasing from the retailer. This creates the responsibility conundrum between the ISP and the OEM provider. Reason, some models of these devices may be specific to an ISP (due to exclusivity contract nature).

Question:

  1. 1. Who is responsible in ensuring that the devices are certified under the scheme? The ISP? The OEM? The reseller? The consumer? What governs the relationship? Contractual? Explicit requirements?

Responsibility

Each device has 4 parties associated to it. The end-user, usually the consumer. The retailer, if its purchased from them. The reseller, in this case ISP, who pre-selects and provides the device. And the manufacturer, who produced the device. Each party is part and parcel of the whole value chain will have, some roles to play. The assessor will be the one conducting the scheme audit and certification.

Question

  1. Are these lines of responsibilities clear?
  2. Are they defined and the relationship in the ecosystem explicit, outlining their roles and responsibility?

Process

The process of this scheme defines how the whole thing works. It starts by establishing the requirements, roles and responsibilities, liabilities and limitations. While the scheme is voluntary, in the subsequent section, you will see how it can become a precursor to other things.

Questions

  1. Probably the most easiest question to answer – What is the certification process?
  2. Is the certification process a one time?
    1. Does it recur?
    2. How often is it retested to ascertain continuous compliance
  3. Once a device is certified, is the rating lifetime?
  4. How does device obsolescence affect rating?
  5. Is there a requirement on minimum device lifetime support as part of this scheme?
  6. How much does the process cost?
  7. Is there a requirement to the parties responsible in ensuring that the devices continue to serve at the rating provided?
    1. If so, what do they need to do? (I.e. timeframe to issue patches, automated patching, etc)
    2. If not, is there a downgrade process and how is that communicated?

Scope of Scheme

There are many variables in having this scheme. While the device is one, the moving parts (metaphorically) varies, depending on the complexity of the device. Hence, it is necessary to see what is being certified.

Questions

  1. Is the hardware itself certified?
  2. Are the supporting cables and peripherals certified?
  3. What are the determinant factor for a peripheral or supporting items such as cables be required for certification?
  4. Is the software/firmware certified?
    1. Is the certification based on version?
    2. Is the certification based on family of hardware support?
    3. A user compiles his/her own firmware (aka Tomato) for his/her router. Are those certificates? If so how? Codebase? Version? And back Roles & Responsibility
  5. Some devices have embedded micro codes. Does the scheme cover micro codes? SOC? Chip level instructions?
  6. How does the star scoring work? Equal weightage for all components, or in parts? Weighted? Percentage? Traffic lights style?

Let’s look at more complex aspects of this venture. While some of this may sound hypothetical, history has shown that anything and everything will eventually take place.

Vulnerabilities

Anything that has a piece of code, may also have a piece of error or bug.  It can be at a compiler level, code level, or even binary level. The security of the device ultimately defined on how well the code executes and is resistant against attacks. Devices with code often has 3 types of code (i) Specific code (usually due to hardware) (ii) Common Code (Free or Open Source) and (iii) Proprietary Code. The term bug is loosely used in this article to refer to vulnerability or software defect.

  1. Hardware bugs have been quite common, with the recent Intel CPU based hardware vulnerability. Does the scheme take into account hardware based vulnerability checks? This will have impact on even laptops as most Intel CPUs  still have some vulnerability to these types of attacks.
  2. Hardware bugs are often resolved with updated drivers (though drivers themselves pose security risks). Are these components part of the scheme? Drivers and even hardware/firmware are often guarded with strong agreements.
  3. I am a gateway developer, who purchases a hardware solution and develop an application on top of the platform. I do not have access to hardware layer information but have the application code for it. Can I get star rating for the scheme?
  4. Open Source Code is a common sight in any commercial product. Is the testing of these code done as part of the product or evaluated separately?
  5. What’s the impact when a common Open Source component (for e.g. OpenSSL) has a vulnerability? Does this warrant a review of the component as its own ? If so, will the scheme cover all Open Source components?

Pricing

Any product, given the time and effort to produce it will formulate the pricing strategy of that device.

  1. Will higher rated device be priced higher?
  2. Will the pricing also cause product isolation (cheaper devices will not run secure firmware?)

Consumer View – Simple vs Complex

At the end of the day, its the consumer who will take the simplified look. “I will only go for 4 star rating because i want to be secure” Herein lies the problem

Ah Chong (A local food court hawker) walks to the shop, wants to buy a gateway. He looks at this set of requirements, which his friend advice meets the scheme 4-star requirements.

  • CPU – MCX1234 – 4 star
  • Hardware – Cap Ayam revision 2.2.5 – 4 star
  • Cables – Xiao Kia  power code with Xiao Kia Ethernet cable – 4 star
  • Firmware – Potato Router version 1.2.99 – 4 star

Ah Chong checks every single component to make sure its 4 star before buying it, by going to the Scheme websites, putting each item one by one.

Is this simple? Or complicated?

2 days later, Potato Router firmware using OpenSSL 0.9.8h has a Remote Code Execution vulnerability and there is active exploitation (for this scenario). The router has a firmware update page, but requires Ah Chong to login, download the latest firmware.

Is this simple? Or Complicated?

Ah Chong buys the router 4 days after the vulnerability was announced. Being non-technical, he has no clue about the matter and is not aware of the latest tech/cyber security development. The packaging shows the information as 4 star validation, which is accurate as of the point of manufacturing.

Is Ah Chong deceived? Is he still confident of his purchase decision?

Future of the Scheme

For all intents and purposes, these schemes are beneficial and brings good to the society. It helps raise the bar for social and economic improvements through cyber security enhancements in total. However, future considerations need to be taken into account

  1. Today the scheme is voluntary. Once the scheme proves itself, its only natural for the governments to make it mandatory. Will it stay voluntary?
  2. Once it becomes mandatory, its easy to include requirements such as TLS MITM, traffic snooping, etc as a requirements for these devices (especially for gateways). Will this be an eventuality at the name of cyber security?

Moving Forward

There is no doubt that this initiative is necessary. Long has it been that consumer devices suffer from insufficient security controls. However the issue isn’t that simple. I fear that CSA Singapore may have oversimplified the matter. Oversimplification is a problem in Cyber Security and will due be a problem by itself. I understand drawing the parallel between the 1800 water issues and cyber security, but the issue and complexity is far from it. This is indeed an interesting development, and I am curious to know how CSA intends to address the concerns raised above.

P/S: Fellow journalist who happens to use these materials, do give me some credit, thanks!

Reference

  1. The Register – https://www.theregister.com/2020/10/01/singapore_infosec_strategy/
  2. Mirai Botnet – https://www.csoonline.com/article/3258748/the-mirai-botnet-explained-how-teen-scammers-and-cctv-cameras-almost-brought-down-the-internet.html
  3. Spectre and Meltdown – https://meltdownattack.com/

Privacy – White elephant in the room with COVID-19?

If it’s one thing life has taught me, “almost” everything has a price. For a good sum, you can get a person to sell his phone. For others, something else. It’s known fact that we live in the world of data.

Everything we do today generates data. Every step you take, every move you make (no its not a song), every interaction. Our lives have become a digital data lake, filled with details of what happens. Data can be in many forms. Audio – forms of conversations. Video – CCTV footages. Logs of transactions, usage, and patterns that is formed based on behavior.

In 2018, Strava, a company that produces fitness tracking solution had inadvertently revealed secret military base due to its users heatmap. The visualization component of the app provided heat maps on user clustering, which indicated secret military presence. This wasn’t the outcome Strava had foreseen, but undoubtedly become prevalent.

I have a saying about data. “Once you create data/information, you are forever doomed to tend to it till it ceases to exist”. Something like Sisyphus, who was condemned to rolling the stone up the mountain, only to find it back down the very next day.

If you attend a conference and visit the booths just for a “look-see”, you most often find a simple glass bowl. In it, a stack of name cards. Name cards are wealth of personal information. One could argue that its corporate information, which is apt. However you’d also find mobile phone numbers. Unless if those are company issued (remember the good ol days of Blackberry?), you’ve just handed over your personal mobile phone number to (not one) but countless number of individuals who will have access to that information. Ever wonder how a completely unknown sales person calls you up for similar products…. *cricket sound*

Ironically all that for a “possibility” of winning <insert the latest gadget name> or a booth token/premium. I remember the time when we went to a week long debate about managing personal information in the form of name cards in context of whether it is a business or a personal venture during the implementation of Malaysia’s PDPA for a telco, together with then the Commissioner of PDPC.

With MCO, contact tracing became a “new normal” (see I can also do buzzwords). Contract tracing is when the outlet you visit requires you to put your details such as name, phone number and temperature. It’s implemented quite simply, using a piece of paper or a book with the visitor jotting down his/her details. Just hypothetically, if you see a person of interest walking up to the same outlet, all one has to do is glean over and note down the number that was written on the contact log. There’s 2 school of thoughts on this matter. First, the contact details given, in some instances, are bogus to prevent such exact situation, which sadly defeats the purpose. Second, it’s a requirement, hence the burden of protecting such information belongs to the establishment collecting that information…. *cricket sound*

Point to note, what happens after MCO? Is the log book going to end up in a dump somewhere with all of the contact details?

What about contact tracing apps? I’d like to cite the example of AarogyaSetu app from India. When it was initially launched, the creators were barraged with queries of privacy and surveillance, which eventually lead to the app being open sourced. While the code was open sourced, upon closer inspection reveals that it has a few critical missing parts, and also found that it was retaining logs of other devices the app had come into contact with ( a database inside the app stores all of the Bluetooth addresses). The internet community celebrated its victory, having to compel the authors to publish the codes on Github.

The importance of having such applications being code published are a few. The codes allow the collective hive of internet to find any potential bugs or issues which allows the app to be improved and become safer over time. The transparency of code helps to allay fears of surveillance and privacy concerns. There’s research done on privacy preserving scheme which can be used to ensure that the app only captures relevant information. In an increasing rise of police state, such as the Black Lives Matter movement (re: George Floyd) in the US and worldwide, having such steps shows commitment of the respective nation states to their rakyats (meaning citizen in Malay). It’s been seen that data leakages happen due to poor security on the backend (such as exposed data buckets on the internet).

There is no doubt, the new normal has everyone getting adjusted into doing things differently. But that doesn’t mean privacy needs to take a back seat. Things can be done in proper manner, just needs some serious thought through. Age of smart phones has made it much easier for anyone and everyone to do contact tracing easily, but it also comes with serious fore-thought for it to be effective.

In Malaysia, we have a number of mobile apps. State government of Selangor published the “SeLangkah” app to do simple contact tracing. Malaysian central government introduced MySejahtera and MyTrace for COVID-19 tracking. MySejahtera has been seen to be adopted as part of the wider strategy while SeLangkah seems to be most retailer’s choice.

With these apps in place, in Malaysia, there are questions left hanging

  1. What are the security considerations and controls put in place to ensure that the mobile application is secure?
  2. Will the codes be published (quoting the Minister of MOSTI who made the statement on 10 May 2020) ?
  3. Where is the data captured by these apps stored? Are those storage secure? Who has access to those data? What type of data is secured?
  4. How is the security of the backend servers and services of these mobile applications?
  5. Has the mobile app undergone necessary security validation (i.e. vulnerability assessment/penetration testing/code audits)?
  6. What happens after the Movement Control Order (MCO) dismantled? What happens to the application and the data being captured? Whose responsible in enduring that the data is not kept beyond its use and disposed securely?

References

1. Strava fitness band gives up military presence – https://www.theguardian.com/world/2018/jan/28/fitness-tracking-app-gives-away-location-of-secret-us-army-bases

2. Myth of Sisyphus – https://en.m.wikipedia.org/wiki/The_Myth_of_Sisyphus

3. AarogyaSetup Android app Github page – https://github.com/nic-delhi/AarogyaSetu_Android

4. DP3T – Decentralised Privacy-Preserving Contact Tracing – https://github.com/DP-3T/documents

5. Minister allys privacy fears in contract tracing – https://www.thestar.com.my/news/nation/2020/05/10/khairy-allays-privacy-concerns-over-contact-tracing-app

 

Hackers for Hire – The case of Dark Basin

Mad kudos to Toronto based Citizen Labs for this excellent work!

Citizen Labs just published (about 13 hours ago) an expose of an Indian company, dubbed as ‘Dark Basin’ which is responsible for hacking thousands of individuals over six continents. The victim list isn’t just random joes, but public figures, rich and the affluent, NGOs including Electronic Frontier Foundation (EFF).

I wasn’t really surprised when an Indian company was coined for this. India is known as tech factory, producing software development and technology talents which went all over the world. And this included the dark side of technology.

Not too long ago, I was involved in forensic investigation of a high level intrusion, affecting Board of Directors and Senior Management of European  region telecommunications provider. Working closely with the law enforcement agencies, we were able to trace and eventually find out that the perpretrators were from India and had vast infrastructure for such clandestine operations. The Norman Hangover report was published, detailing the bits and bytes of the attack.

Back to the recent expose; the company used a variety of methods to target their victims. Primary mode of attack is through phishing. Their effective rates were high, simply because they were extremely persistent. They would do intelligence gathering of the clients, and attempt multiple times from different angles until the clients fall prey to the attack. On the background, a server is set up to masquerade as valid login pages, such as Google, or Facebook. Once a victim enters their password, their credentials are exposed to the attackers and its used for whatever other purposes deemed fit. In some attacks, these attackers were seen using these illegally obtained credentials to send out phishing email to other related entities, making them also fall prey to the attack.

These series of attack is attributed with high confidence towards Belltrox InfoTech Services. The attribution is made based on a few factors. The domain previously used by Belltrox – belltrox.org was registered by the email address from Yahoo, which was also used to register other phishing sites. Eventually this email address was changed. Operating hours of which the phishing emails were sent correspond to IST – GMT+0530. References to Indian festivals were made on their URL shortener (powered by phurl). Incidentally the same URL shortener was used by the attackers to link back to their CV. The founder of the company – Sumit Gupta (named as Sumit Vishnoi in DOJ documents) were previously indicted on hacking-for-hire scheme. In short, they were identified due to severe lack of opsec (I think the staffs didn’t know that they were suppose to keep it hush hush).

LinkedIn provides wealth of information about Belltrox and its circle. Based on the recommendations received by Belltrox and its staff, its clear that Belltrox has been working with private investigatiors and government agencies. This includes Canadian government officials, local law and state enforcement agencies and former intelligence agency staffs who are most likely gone professional.

Victimology indicates a large pool of diverse targets, which shows that the nature of business is not specific, but demand driven. This includes NGO that goes after large corporations, such as the #Exxonknew campaign. Interesting targets to note including friends and family members of those involved in the campaign, including the legal counsel.

At this point, it is certain that Belltrox is the source of the phishing campaigns. Who hired them still remains unknown. Sumit Gupta, when contacted, denied of any wrongdoing and stated that his firm assists his clients to retrieve emails for private investigators based on the credentials provided. (Yup…. eyes rolling here).

Belltrox has a wide range of industries where their target resides. This includes short sellers, hedge funds, financial journalist, global financial services, legal services, Eastern and Central Europe, Russia, government agencies and even individuals involved in private dispute.

Tools Techniques and Procedures – aka Tradecraft

Key modus operandi of Belltrox is phishing. They deploy a number of phishing kits (which they even leave it open/available). To power these phishing kits, a URL shortener is used. The URL shortener is based on a package called phurl, which creates a sequential numbered shortened URL, which makes it easy for the good guys ™ enumerate and identify what are the actual long URLs. Through this, the list of domains used for phishing is identified.

While phishing isn’t really new, this revelation strengthens the idea that phishing is very much relevant and effective. Login pages of commonly used services such as Google/Facebook is hosted, creating the opportunity for the attacker to capture credentials.

Hacking-as-a-Service (HaaS) – Global issue

HaaS is becoming a global thorn in the cyber realm. Emergence of such players, including reports on the based on Dark Matter highlights a lucrative market for such services, and that while the service remains clandestine, demand and need for such services continues to thrive. Legal frameworks are still developing over the need to handle and dismantle such services.

HaaS also presents an issue for attribution of attack. In this case, Belltrox was coined as the attacker, but the actual puppet master remains hidden. This can apply for nation state sponsored attacks, completely washing their hands away which engaging a contractor to do the dirty work for them.

Protecting Yourself

These attacks highlight a key need for 2 factor authentication. Worthy to note that any security control put into place makes it harder, but not impossible for attackers to get through.  The Dark Basin attacks runs on the premise that the victims did not secure their Google account with 2FA, making it easy for the attacker to use their ill gotten credentials to gain access.

References

  1. Norman Hangover Report – https://paper.seebug.org/papers/APT/APT_CyberCriminal_Campagin/2013/Norman_HangOver%20report_Executive%20Summary_042513.pdf
  2. Citizen Labs – Dark Basin – https://citizenlab.ca/2020/06/dark-basin-uncovering-a-massive-hack-for-hire-operation/
  3. EFF phishing attempts – https://www.eff.org/deeplinks/2017/09/phish-future

 

Sg Buloh hospital – Jan 2020 case study

Introduction

The Malay Mail reported that Sungai Buloh hospital (SBH) was recently hit with IT failures. Sg Buloh hospital is quite well known to the denizens of Klang Valley, being a governmental hospital of choice to many. I personally find the service is very good, doctors are friendly, professional and I don’t spend much time waiting as the processes are quite efficient.

The news report highlights difficulties in retrieving medical and investigation reports of patients. This problem was pinned at the hospital using Windows XP as the Operating System. Reports also mention that the hospital main servers were down for sometime. It was also quoted that the issue also stemmed due to limited storage space due to Windows XP OS limitation.

Some facts for consideration in this article, which to me sounded odd, but plausible. Lack of clarity on the matter, unfortunately fuels to the speculation (of which some are discussed in this article)

What about Windows XP?

Firstly the use of Windows XP. Microsoft positioned Windows XP as an end-user operating system, targeted to be installed on desktops and laptops. It was never meant to be used in server environments, though I have noticed small organization turns a desktop into a file share for the organization. The OS was never meant to be a server platform (though it can be at a minuscule deployment), and surely not for a hospital the size of SBH.

Microsoft declared Windows XP obsolete starting April 8, 2014. That means from the EOL (end-of-life) date, there will be no support. Meaning, if there is a bug or vulnerability in the OS, it will not be fixed. This also implies that the software ecosystem, such as anti-virus, endpoint protection and other critical software will also not made available as the efforts to maintain the software will be focused towards supported operating system. So, not only you have an OS that doesn’t have any updates, you also lose the updates for other software that runs on that ecosystem. Essentially a ticking time bombs.

Without any further information, one of the points of issues faced by SBH was the server failure. This can potentially be attributed to the use of Windows XP as the server OS (god forbid, but based on experience can/may happen), or a general failure at the server, be it at the OS, Application or Hardware level (may even be network, the server might be running on a 10Mbps network card, supporting the whole hospital). Again, without any further details, one can only speculate on this matter.

Another interesting point to note that the report also pointed out that Windows XP has a storage limitation. A quick check shows that the first limitation is at the memory level, and its due to the 32bit architecture. There are also issues at the file system level, as XP primarily supports FAT (File Allocation Table) format, which has a hard limit of 32GB. Worth to note that Microsoft also released a version of Windows XP for 64 bit. Windows XP supports FAT32, however the format tool natively supplied with XP doesn’t allow creation of FAT32 partitions (Primer: FAT/FAT32 creates an index of where files are located, based on free space available, which gives the OS a location of the files in the disk).

Software Obsolescence

Software obsolescence isn’t new, but its worth revisiting to understand how it can contribute to this situation.

When a software companies declare that a product is at End-Of-Life, its a statement to inform customers that they will no longer support that product, and that the customer should opt for newer software. While at face value it looks pretty simple, the impact is far reaching.

Hardware Compatibility

Firstly, upgrading an OS requires the hardware the be compatible. This means, if my father had a PC that he uses at home, he needs to first check if the PC can be upgraded. At times, the change in the OS can be drastic. Meaning, that the PC may not be compatible due to several reasons. For example, moving to a new PC hardware because the processor and it’s architecture is also obsolete. In this case, a new OS may only support 64bit architecture and may not have backwards compatibility to a 32 bit architecture. Windows XP supported both 32 bit and 64 bit architecture, paving way for the move from 32bit hardware to 64 bit hardware, which offers better scalability and flexibility. While this implies the link between OS and hardware, some subtle changes at software may have similar effect. Apple introduced MacOS Catalina which effectively prevented running on 32bit applications on it, making it a pure 64bit based Operating System. Many users reported the issue due to application availability, and some users ended up reverting back to MacOS Mojave.

Secondly, while the hardware becomes a factor to look in, another point to consider is the hardware compatibility towards the OS, from the point of drivers. Drivers are software that allows the OS to “talk” to the hardware. Without the drivers, the hardware remains useless. I remember the time when we had a spectrum analyzer in one of my previous roles, and the spectrum analyzer on worked on Windows 3.11 for Workgroups due to limited driver support. We couldn’t move it to the latest OS as the manufacturer had stopped support, and their recommendation was to spend equal amount of money to get a new set of equipments, that had the latest software and drivers. The organization, of course, chose the path of least expenditure, had to isolate the PC so that it was only a single use, making it a standalone independent analyzer. I still remember pulling the data off the analyzer using floppy disk. The same can be said to medical equipments. Imagine an MRI that was build using Windows XP as it’s operating system, cannot be migrated as the hardware support is no more there. No “sane” hospital would buy a new MRI machine just because the OS is outdated. As some IT experts will tell you – “If It ain’t broken, don’t fix it…”. The whole security industry was built around managing security risks, mind you.

Thirdly, organizations do not actively manage obsolescence. This is because obsolescence is an expensive affair. A cost conscious organization would do its best to “sweat its assets” and make their investments live past zero net book value. It creates additional cost to business when it comes to obsolescence management, and we potentially see it happen previously at the MAHB issue.  There is cost in replacing equipments, there is cost in migrating the data as well as time and resources required to carry out the project, not to mention training to all those involved so that they know how to use the new system (which may come with new UI/UX and workflow). It’s a very daunting affair, and one that has far reaching effect throughout the organization.

Software Dependencies

We depend on a number of software for our systems to work. Having a computer and the OS alone doesn’t do much, business runs based on its Line-Of-Business applications. Example, ERP (Enterprise Resource Planning), CRM (Customer Relationship Management) and many more. When the OS becomes deprecated, software developers also stop developing functionality for a now defunct OS, moving their codebase to a new platform. This means the code, depending on the extensivity of the change, may render the old code useless. Some platforms provide some degree of cross version support, but that’s usually limited, more of in favor of newer platforms. Remember that the same support required by the user is also required by the software developers to build their code on. There are some cross platform frameworks available, but even these frameworks may stop supporting older, deprecated OS.

The OS also comes with SDK (Software Development Kit), crucial for software development teams to harness the power of the OS. Just like how the OS gets deprecated, so will the platform SDK’s be.

Moving Forward

I discussed technology debt in greater details before, and it seems to be a recurring theme in large organizations in Malaysia. Obsolescence is a huge debt, which most organizations overlook, eventually coming back to haunt them. It’s not a discussion that any CEO/CFO would ever like to have, especially when the cost balloons and creates a huge dent in the balance sheets. The same can be seen in Governmental departments, where focus on maximizing the taxpayers money can be seen as a prudent outcome of well oiled administration.

Non-technology organization fail to grapple the complexities of technology. It took us long enough to start trusting automation and computing, and now this becomes another headache to manage. Some organizations even opt not to capitalize on the computing/internet era, effectively creating a barrier to efficiency and economies of scale (when it comes to data/records management). Having focus and attention to IT helps to alleviate such issues.

Most organizations establish an IT Steering Committee, comprising of senior leadership team to address such risks. Strategic discussion on project prioritization, maximizing annual budgets and looking at technology risks becomes a staple periodic discussion to look at IT and it’s associated risks/debts.

If at all, the business is faced up to the wall with no option but to manage its business with the existing infrastructure, there are some mitigation’s that can be applied. Ideally these systems should be run in isolation (similar to the spectrum analyzer case I spoke earlier). If it has to be networked due to the nature of the application, thorough backup and restoration procedures need to be established, run and periodically tested. I’ve seen organizations that pride on doing backups, but had never tested their backups, nor even tried to restore the backup (there’s a reason why those systems are called backup software and not restore software). Having a recovery plan helps, but the plan is as good as being tested periodically. Worst case scenario needs to be tested, break glass type procedures, and how business can run, in the event that complete failure is imminent. All else fails, start saving for a new system, maybe look at cloud as an alternative?

References

  1. Malay Mail (22 Jan 2020)- https://www.malaymail.com/news/malaysia/2020/01/22/report-sungai-buloh-hospital-hit-by-it-breakdown/1830364
  2. Microsoft XP EOL statement – https://www.microsoft.com/en-us/microsoft-365/windows/end-of-windows-xp-support
  3. MAHB Case Study – https://www.drsuresh.net/2019/09/mahb-case-study-aug2019/
  4. Technology Debt – https://www.drsuresh.net/2019/08/cyber-tech-debt/

Malaysian Airport Incident – A case study

Last updated: 4 September 2019

Acknowledgement

The information provided in this post was through crowdsourcing, thanks to the IT Security SIG set up by Nigel Rodrigues, contributed by many, with candid discussion which inspired me to write this article.

As this incident is still developing, this article will be updated with the latest information, and what you see here is a snapshot in time at the point.

The incident

On 21 August 2019, KLIA/KLIA2 airports begin to experience system and technical difficulties. The failure affected check-in counters, flight information display systems (FIDS), baggage handling, its airport mobile app, as well as payment systems which rely on its networks. Ostensibly, tempers and dissatisfaction among airport users were high.

Timeline

20 August 2019 – MAHB signed a MOU with Huawei on technology modernization.

21 August 2019 – KLIA/KLIA 2 reported system/technical issues affecting multiple systems in the airport. Initial news indicates a failure at the network equipment.

22 August 2019 – The Star reported that MAHB had informed that the situation will be resolved by 23 August 2019, as it has received new equipments to replace the existing ones and testing to be conducted on the same night.

23 August 2019 – MAHB updated their website (as at 6am) explaining that they are in the midst of stabilizing their system and had deployed additional buses to ferry the passengers to their respective terminals.

24 August 2019 – The Malay Mail reported that the situation had improved, passenger flow has been reported smooth with intermittent disruptions.

24 August 2019 – NACSA issued a statement to affirm that there were no cyber attacks which resulted in the network issue at KLIA/KLIA2.

25 August 2019 – The Malay Mail reported that KLIA/KLIA2 Operations has been restored to normal  based on a check by BERNAMA at 0930.

26 August 2019 – Ministry of Transport announces a panel to investigate the system failure of TAMS (Total Airport Management System). NACSA is one of the members who forms part of the committee.

26 August 2019 – MAHB in a statement was quoted saying that they are not dismissing the possibility of malicious intent that may have caused the incident.

26 August 2019 – Airport passengers were stating that its not a full service recovery, the information system was still down and the airports were operating at partial system availability.

27 August 2019 – Airlines seeking compensation from MAHB due to airport system down.

27 August 2019 – MAHB lodges police report over possibile malicious intent being cause of downtime.

28 August 2019 – PM orders probe to the airport downtime incident.

29 August 2019 – PDRM said to be probing 4 in relation to airport system failure, based on the report made by the IT division senior general manager.

30 August 2019 – AirAsia, a malaysian carrier is said to confirm not to sue MAHB due to the recent airport system failure.

2 September 2019 – Police said to have recorded 12 MAHB staff statements over the system failure incident.

3 September 2019 – 4 MAHB pioneer IT officers lodge counter police reports against MAHB. They were suspended, and claimed false accusation.

The cause

The details are vague, however the incident was pointed out to a faulty IP network switch which caused the IP network traffic to get to a grinding halt. The switch in question seems to be the core switch which processes all the network traffic for the airport.

Core switch is usually responsible for traffic between each segments, also acting as an aggregation point. Each area is connected via a smaller switch, point to an intermediate or aggregation switch which leads to the core switch. In this case, since the core switch is down, the segments are disconnected, through each machine shows as network connection as connected. Access to upstream, such as Internet, which is used for credit card payment gateway, is also interrupted as the traffic stops at the core since its not working.

Social Media Buzz

It was noted that a user, claiming to be a subcontractor to MAHB said that the network switch had been 17 years old and had not been changed since. This is unconfirmed, pending official statement from MAHB.

A report from Utusan Malaysia also have mentioned something similar, an excerpt mentioned here.

Related news

Just one day before the incident, on the 20 August 2019,  MAHB signed an MOU with Huawei “to drive MAHB’s digital transformation framework by enhancing connectivity and real-time information by connecting all stakeholders in one fully integrated digital ecosystem. The collaboration would also seek to set up a fully integrated network communication managed platform to manage above technology and integrated data to enable future big data analysis throughout the entire airport, further improving airport operation efficiency and reduce overall ICT cost.”

It’s probably sheer luck, the network equipment failed the very next day, seemingly catapulting the priority of this initiative.

Assessment

At this point, lack of official news seems to lead to multiple speculation. The first would be that the airport was under a cyber attack. This news was quickly quashed by NACSA, confirming that there were no attacks.

Another discussion lead to the belief that there should have been sufficient DR (Disaster Recovery) infrastructure to ensure business runs as usual. Assuming the social media news was right, most networks designed at that time would have had a typical star topology, whereby layer one connectivity would cascade back to a single core switch. Using Cisco as example, the spine and leaf architecture would have allowed the network to be redirected to a different core, should that had been the architecture. Spine and leaf is still a new concept, there may be others which any organization can adopt.

The Good

MAHB had been mobilizing their own staff, by recruiting and promoting initiatives to get them to assist the passengers during these trying times.  A poster was seen circulating on social media dated 22 August 2019 asking to assist the situation at KUL during peak hours (12 – 2pm & 4 – 10pm).

MAHB had exhibited strong understanding of the airport processes, being able to manage with manual processes and having pure manpower to handle the airports operations while the system was down.

Flipside

Assuming the theory about 17 year old network equipment is true, there can be 2 possible outcomes. The first, an overzealous CIO might end up saying “We should sweat our assets more, make sure you don’t buy anything new for the next 15 years! (BTW are we using the same brand as the airport?)”. Scary, to say the least! Worthwhile to remember that computer/network hardware are susceptible to degradation over time, even to the network copper wire, hence some data centers make it a point to “re-cable” their infrastructure periodically! Other views include “we’re not an airport, we wont need to worry about it”.

The second outcome is that investment on IT now becomes justifiable, as part of technology refresh. More prudent approach to technology life cycle emerges and that the MAHB story becomes a talking point at the Board level, raising the question of whether the assets in use are still (1) maintained, with necessary support and (2) prior to End-of-Life/End-of-Support. This is in line with managing tech debt, ensuring that such compounding interest doesn’t suddenly pop up!

Lessons learnt – so far

1. Have manual processes that will stand in if something fails. Can you operate without technology?

2. Understand the implications of tech debt. It’s a matter of time before it catches up and as an organization then pays the compounding interest. Reputational damage becomes severe and takes time to recover.

Reference

  1. Malay Mail – https://www.malaymail.com/news/malaysia/2019/08/23/mahb-network-failure-caused-systems-disruption-at-klia/1783638
  2. MAHB Official PR – https://www.malaysiaairports.com.my/media-centre/news/klia-network-disruption
  3. NACSA PR – https://www.nacsa.gov.my/doc/Press_Release_MAHB_KLIA_English.pdf
  4. TheStar MAHB Huawei MOU – https://www.thestar.com.my/business/business-news/2019/08/20/huawei-malaysia-to-support-mahb039s-digital-transformation
  5. Cisco Spine & Leaf Architecture – https://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/white-paper-c11-737022.html
  6. Copper degradation – https://www.quora.com/Does-a-signal-sent-over-a-cable-network-degrade-over-time
  7. Potential malicious intent – https://www.thestar.com.my/news/nation/2019/08/26/mahb-not-ruling-out-malicious-intent-behind-klia-glitch
  8. The Star (22 Aug 2019)  – https://www.thestar.com.my/news/nation/2019/08/22/mahb-expects-klia-glitch-to-be-resolved-by-friday-morning-aug-23
  9. MAHB update (23 Aug 2019) – http://www.malaysiaairports.com.my/media-centre/news/latest-update-systems-disruption-klia-0
  10. The Malay Mail – Day 3 – https://www.malaymail.com/news/malaysia/2019/08/24/klia-systems-still-crippled-but-operations-improving-on-third-day-video/1783804
  11. The Malay Mail – Day 4 – https://www.malaymail.com/news/malaysia/2019/08/25/klia-operations-back-to-normal-after-system-outage/1784006
  12. The Star – https://www.thestar.com.my/business/business-news/2019/08/27/airlines-to-seek-mahb-compensation-for-delays-losses
  13. https://www.nst.com.my/news/nation/2019/08/516444/mahb-lodges-police-report-klia-systems-disruption
  14. https://www.thestar.com.my/news/nation/2019/08/28/pm-wants-probe-into-klia-systems-malfunction
  15. https://www.malaymail.com/news/malaysia/2019/08/29/report-police-to-probe-four-over-klia-systems-disruption/1785325
  16. https://www.nst.com.my/business/2019/08/517398/airasia-wont-sue-mahb-system-glitches-klia-and-klia2
  17. https://www.malaymail.com/news/malaysia/2019/09/02/klia-systems-disruption-police-record-statements-from-12-mahb-staff/1786552
  18. https://www.nst.com.my/news/crime-courts/2019/09/518461/4-mahb-it-officers-lodge-police-reports-against-their-employer-over

IT vs Cyber Security – Technology Debt

Where are we today?

Almost on a daily basis, we are bombarded with news of cyber attacks, breaches, data leaks and more. It’s as if cyber related issues are becoming a norm, so much so someone was quoted saying “ There are 2 types of organization; the ones that has been breached, and the ones that have yet to be”. As such, all organizations are putting emphasis on spending for continuity, and one question gets asked quite frequently. How much is enough? Is there a magical percentage that a CEO needs to consider as part of healthy spending to ensure that the safeguards are sufficient to manage today and tomorrow’s risk?

While there are research done on average spend buy organization, that is not an accurate reflection of what a particular organization’s spend pattern for protection of its assets. This article aims to demystify the topic on technology debt, using Security as a factor,  in order to identify right-spending for an organization. Technology debt is just one of the consideration to put into place when evaluating tech spend vs. security spend as a consideration.

What is a technology debt? 

A debt is defined as owing. When someone borrows money, they are obliged to return in (in most instances with an interest). A technology debt concept is no different than a conventional debt, however it is in the form of considerations, protection and governance aspect in rolling out current and new technology.

The concept of interest in technology debt is the occurrence of an event which creates an additional burden to the organization. Example, a cyber breach causes additional overheads from manpower utilization, engagement of relevant third party for services such as recovery, forensics as well as additional expenditure incurred.

Does Technology incur a debt? How does it work?

To illustrate technology debt, there will be 3 examples on how technology debt is incurred.

Scenario 1

An end user procures a computer for home use. He gets the operating system installed and starts using. He/She finds the computer very useful and engaging, starts using it for not just work/assignments, but also for personal content consumption such as videos, websites and even social media. One day, the user encounters a phishing email, which leads to downloading an attachment which infects the user with a ransomware. As his work is important and needs to be sent to the customer, the user ends up paying the ransom.

For this case, the tech debt was incurred at the point of starting the use the computer. The debt was to ensure that the computer was secured, had the necessary protection in place, such as endpoint protection and phishing alert. Because the debt as been incurred, the user ends up paying with interest, i.e. the ransom in order to retrieve the data.

Question: does the debt end here? Yes and no. While the ransom is paid (interest), the debt (principal) is still there. The debt goes away when the user secures his endpoint/laptop/machine and removes the “debt” altogether.

Scenario 2

A hardware store has purchased a Point-of-Sale (POS) terminal for use, primarily to ensure sales tax calculations are done and the reports made available for submission to the authorities. There is a thermal printer to print out the receipts, with the computed tax value as per regulations. A barcode scanner is attached to make it easy to input item code for the data capture during checkout. It became very convenient, so much so that even the inventory was managed effectively. Life seems to have been easier, thanks to the new technology. The POS came with 1TB hard drive, which makes it almost impossible to fill it up.

One day, for some unfortunate reason, the hard drive in the Point-of-Sale machine crashed. This resulted in some inconvenience as the the items have to be manually computed. Because of the convenience of the POS system, the prices are no longer printed as the reliance is towards product barcode. A manual list with the prices had to be derived after calling the vendors for price confirmation. What made it worse? The taxation department decides to show up for an audit, demanding to see the taxation report that was suppose to be produced adhoc as part of system requirement for taxation.

The technology debt in this case is the inability to backup and restore the system. While the reliance of the system is good, the debt (backup/recovery) had been incurred, and the user ended up paying interest (fines due to non compliance, additional recovery services, manual process institution, time wastage).

Scenario 3

A mobile app development firm has purchased a server  to store their source codes. The server is backed up daily using DVD and a copy is kept in a separate site. The server is configured with detailed access control list to ensure only the right people have access to the right set of codes.

A disgruntled employee decided to take matters into his/her own hands and deleted portion of the codes on the day he/she was leaving. The manager discovered the issue when reviewing the CI/CD logs during build failure and found files missing. Upon inspecting the version control software, identified the malicious action that has taken place. The manager proceeded to recover the part of the tree that was lost, and compared it against the backup that was kept to ensure that the changes made were consistent.

This case shows zero debt scenario. While deploying the solution, the IT team took into consideration requirements for backup, audit logs and continuity plan. When a potential “interest” scenario came up, because the debt was zero, there was none/minimal impact to the organization.

How does tech debt influence budgeting?

As technology gets deployed, as illustrated above, debt starts coming in. In some organizations, the debt is addressed up front as technology is deployed to avoid interest. Some organizations pan out the debt over time, in hopes that the interest will never come up.

How does this influence budgeting? The budget to manage security will include ensuring that the debt is being addressed timely. For organizations that has incurred debt, in order to zeroize the debt, expenditure needs to be done. As budgets are usually one line item for an organization, this then is seen in the percentage split between IT spend vs security spend.

Hence, for some organization which heavy tech debt, the budget will be more towards resolving the debt rather than expansion of IT. The percentage split will be skewed as the debt now influences the spend percentage.

Another reason why the spend will be skewed is when the interest come into play. Due to an incident, the interest becomes mature and payable. This creates additional expenditure which eats up into the budget. Post incident usually sees organization putting more emphasis into governance and control, almost having a blank cheque to show commitment, including in most instances, hiring a CISO that reports directly to CEO and Board.

The result, difference in spend percentage compared to overall budget based on level of debt resolution, depending on the state of the organization. Mature organization depends on resolving debt as the technology is incorporated, while other play catch up, due to business and budget limitation. What’s important is to be mindful that the debt may spring an interest at any time, causing organization to end up spending more. Delayed investment may result in heightened expenditure.

While the scenarios presented above may be simplistic, it is worth remembering that technology debt is often multi-dimensional and require an in-depth study to ascertain the respective areas of protection required. In the future article, we can discuss about this aspect of multi-dimensional tech debt and how to look at resolving the debt and preventing interest.

Moving forward

The crux of this article was to make a clear distinction between why different organization had different budget spend split. Though a baseline of spend helps CEOs identify whether the spend is healthy, understanding the technology debt help to justify why the spend needs to be more for some organization. While most organizations look at analyst report on average security spend, it is wise to ensure that technology debt is kept at check to ensure lack of interest popping up.

Perhaps if there is enough interest, then I can write up on identifying and resolving technology debt.

Geopolitical considerations as part of Technology risk

This thread started off as a discussion at the local Mamak (the Malaysian colloquial terminology for your cafe). A bunch of security and tech folks meet up to ponder upon the world and business woe.

The discussion started off with the question “How do you decide on your tech purchase? What are your consideration factors?”

Our conservative buddy came up and said “You can never go wrong with Brand X! Tried and tested”. That seems to indicate that the selection criteria is based on market presence, branding and prominence. As well as adoption.

The bleeding edge/challenge the status quo person came up and said “Why not Open Source?” Its mature enough for adoption and more organisations are cozying up to the idea that Open Source will work, provided that support is available.

Then comes in the CIO, whom made it clear that his/her choice will be cost based. Why bother paying premium and consider alternatives when you can get a good bargain at a reasonable choice? Pricing would be the ultimate deciding factor, provided that it meets bare minimum.

I had to open my mouth and ask, ”what about geopolitical consideration?” Everyone had a flustered look, some in amazement and some pretended that was not even the case. Geopolitical? Is that even necessary?

What is geopolitical consideration/risk?

This is a consideration when you view the origin/source country of technology and consciously make a decision to use technology from another country. Example, if the first tier of firewall originates from US, the second tier of firewall may be purchased from Russia (ignoring the underlying hardware may all originate from China, the consideration here is based on vendor origin, not part origin, although that may be a severe version of geopolitical based risk separation).

History Lesson – PGP

A little bit of history lesson on technology, starting from cryptography. PGP was created by Phil Zimmerman in 1981. PGP was created with the intention of securing communications between activists and to prevent snooping. The software was free to use, as long as its not for commercial use. Eventually PGP ended up on the Internet, being adopted for widespread use as an added encryption layer on top of emails.

In 1983, Zimmerman became target of prosecution. Cryptographic capabilities above 128 bits became subject to export restriction and Zimmerman’s PGP was using keys with defaults of 1024. Zimmerman became a target, due to violations on “munitions export without license”. Definition of munition includes “guns, bombs and even software”. For unknown reasons, the case never proceeded and was eventually dropped without any criminal charges filed.

Zimmerman was determined to make his software public. He identified a loophole, in which the First Ammendment, protects the export of books. Through MIT Press, Zimmerman published the source codes of PGP. One had to simply procure the book, scan the contents and make it digital using OCR (Optical Character Recognition); or simply type the code into a program.

More challenges on export

A similar situation happened to D.J. Bernstein. He wanted to publish the source code of his Snuggle encryption system. Together with EFF, DJ Bernstein challenged the export ruling. After 4 years and one regulatory change, Ninth Circuit Courts of Appeal ruled that software source code is protected by the First Amendment, and government preventing the publication is unconstitutional.

Why geopolitical risk?

The world is already borderless. Technology crosses boundaries easily without much hassle. However, G2G relationships are never that smooth (G2G – Government to Government). Technology sold by a company is governed by the laws in which that company is HQ’ed. Hence indirectly, law of the land plays an important role in ensuring that governments play an indirect crucial role in determining the availability of technology.

The most common technology denominator is the USA. USA produces majority of technology innovations which the world uses. An example used in the earlier part of this article is encryption/cryptography technology. As algorithms become prevalent, the use of these algorithms often become subject of export restrictions.

The rise of nation states

Borderless world creates borderless problems. The hacking scene (not the “Texas Chainsaw Massacre type”) used to be fueled by hormone-raged idealistic filled teens, or just curious cats trying to learn tech. But today, dominance in cyber space is seen as a sign of “Cyber-sovereignty” and arms race towards cyber dominance becomes imminent. (Man I really abuse the cyber word this time…)

As explained earlier, the battle ground has shifted into the cyber world. Corporates are becoming the unwilling victims in the fight towards dominance. Nation-states may infiltrate large corporate organizations in order to further their agenda, by implanting their tech folks which directly influence the product build. This means that product that gets shipped out may potentially be inhibited with malicious code, backdoors or even intentional vulnerabilities in order for nation state actors to freely abuse.

Export laws, sanctions and politics

Open any news site right now and you’d hear about trade wars between government. In the recent news, one government has stood firm and taken actions against another country for alleged espionage. This resulted in key companies in the country being denied business and imposed high levies and taxes. The situation created a “tit-for-tat” reaction, causing a downward spiral of impact towards other organizations which forms part of the ecosystem.

Standards and tech volition

If export restriction becomes apparent, in a new twist to the developing stories, standards organisation are now becoming subject of such ruling. One standards body which is referred to worldwide has stepped up and imposed bans towards researchers from a said country from being moderators or participating in standards building. This has far reaching impact to the global community.

Firstly, other countries who are not part of the trade war are now unwilling victims as the standards body align themselves towards the country stance. Secondly, the countries now have to re-evaluate and establish their own standards, or subscribe to a common standard which all vendors should be given a chance to participate. ISO (International Standards Organization) is a global standards body which prides on being independent from country level politics (while the standards being voted are based on country lines and affiliations).

In one hand, you need a standards body as reference point, and in another you’ll need to start excluding standards body which shows affiliation towards country level policies. Aligning standards into a country specific set will be another arduous task.

Long story short

Countries today can no longer exclude geopolitical factors of risk. This is prevalent today, looking at the recent developments in the international arena and current trade wars and Brexit. While moving towards Industry Revolution 4.0, it is important to no longer be in a nutshell, but understand that borderless is a reality and new sets of regulations are emerging to govern tech and it’s use.

Insider Threat – A look at AT&T incident

In a recent expose published by SecureWorld through court documents seen, this issue has suddenly hit the spotlight.

The damning question, can your employees be bought?

Lets look at the reported news on the incident experienced by AT&T Wireless. The A&T& Wireless call center in Bothell, Washington is where this had begun. Call center employees knowingly shared their credentials with the cybercriminal, in exchange for money. According to DOJ based on the indictment documents, one call center employee who made the most had paid “$428,000 over 5 years scheme”.

There were 3 things that the employees did

  1. The employees were instructed to install malware in their machine.
  2. The employees installed unauthorized access points, hardware devices to create a backdoor into the network.
  3. The employees installed a specialized malware that performs phone unlocking through AT&T’s internal network using valid AT&T credentials that were obtained from the call center agents.

The objective of the “intrusion” was to create unlocked phones. Phones that are sold in the US are carrier locked, meaning once the phone is provisioned, only AT&T services can be used on those phones. Having the phones carrier unlocked creates a huge market, selling them on eBay and other online stores.

This begs the question, why would the phones require to be unlocked in the first place? The phones are locked to a carrier because it is subsidized and requires a contract. When a user travels overseas, the phones may require unlocking for roaming purposes, hence unlocking becomes a legal function of the call center.

This racket had netted more than 2 million phones to be unlocked and sold. At the rate of an iPhone price, one can only imagine how much money is there to be made.

According to the official documents, the scheme  began somewhere around 2012, and around October 2013, AT&T discovered the unlocking malware. When questioned, the AT&T staff in question left the organization. The criminals were determined, recruiting new insiders in the same call center on the subsequent year. Recruitment happened through Facebook (surprise, surprise, and not LinkedIn) and the bribes were made in-person. The cybercriminal, known as Muhammad Fahd, is now in jail.

A breakdown of this issue

  • Call center agents sold their access and performed illegal acts in payment for money. Insider threat will remain a key issue, and it becomes a challenging issue to tackle. While a potential solution can be to look at a “lifestyle audit”, getting trustworthy staff will always be a challenge, in a market where skills are limited.
  • Valid access used for illegal activities – this may be potentially addressed by monitoring activities performed with a certain ID. This means that there is sufficient logs in place and systems to correlate and analyze system usage behavior and look at baselining activities to identify anomalies. If someone does extremely too many unblocking, a check on what is actually done is performed. This review process (often slow, painful and most of the time even manual) is usually avoided due to unnecessary workload, though i am sure AT&T would enforce this as a requirement now.
  • Installation of malware – call center agents should never ever have administrative access. Ability to install or run application should be limited, through the use of application whitelisting. However, there will still be an issue of a malicious IT Technician, which may have been possible in this scenario.
  • Installation of access points – rogue access points can be detected with Wireless Intrusion Prevention systems. However, WIPS presents different set of problems, as it may potentially deny the use of wireless due to neighboring building APs having signal spillover, effectively causing a denial of services attack.

Taking the original question into perspective, can employees be bought? The answer to this question is multi-faceted while the technical challenges can be addressed

  1. Getting more money is always appealing to everyone around. Looking at the money being made, a call center agent would have jumped to the occasion because of the sheer amount to be made.
  2. Moral obligation of doing the right thing. In any such cases, you’d hear many reasons why the staff did what he/she did. From the point of making ends meet, to doing something that didn’t hurt anyone, moral standing has always been on shaky grounds.
  3. Economics of organizations also play a part. Income disparity, job satisfaction vs load becomes a talking point. Most call center agents bear the brunt of the customers, and often, even yelled at. Hence call centers become a churning pot for most organization, and those who stay are often resilient, understanding that it is a thankless job.
  4. Making examples – some organizations motivate employees to do the right thing by (1) having a whistleblowing policy to aid reporting and (2) showing examples of action taken against wrong-doing. Denying someone their livelihood has always been a key motivator to do the right thing.

Reference

SecureWorld – https://www.secureworldexpo.com/industry-news/insider-threat-at-att-wireless-activated-by-a-cybercriminal

 

Capital One – The Breach

Capital One (PRNewsFoto/Capital One Financial Corp)

The incident

Capital One issued a press release on 29 July 2019 that there was an unauthorized access by an outside individual who obtained access to it’s customer information. The information that was obtained were credit card application information, for applications between 2005 to early 2019. Information breached includes

– Name

– Addresses, ZIP/Postal Codes

– Phone number

– Email addresses

– Date of Birth

– Income information (self reported)

– Status information – credit scores, credit limits, balances, payment history, contact information

– Transaction data from a total of 23 days during 2016,2017 & 2018

– Social Security Number of 140,000 and 80,000 linked bank account of secured credit card

The existing customers doesn’t seem to be affected as the system in question was only specific for credit card application facility.

What happened?

According to CapitalOne, the “highly sophisticated individual” was able to exploit a certain configuration vulnerability. NYT added that it was a misconfiguration of a firewall on a web application, and this echoes the court documents pointing to a misconfigured firewall on Capital One’s Amazon Web Services cloud server. The information was accessed between March 12 to July 17.

More than 700 folders of data was stored on the server.

The hacker

FBI has arrested Paige A. Thompson, going by the nick “erratic”, according to Justice Department. Ms Thompson made appearance in Seattle District Court on July 29, 2019 and was ordered to be detained pending hearing on August 1,2019.

Ms Thompson posted in GitHub regarding the information theft, which was reported by a GitHub user to Capital One on July 17, 2019. Capital One contacted FBI on July 19, 2019 after confirming the breach to be legitimate. FBI confirmed the identity of the attacker.

Ms. Thompson has worked with Amazon Web Services before. It was also evident that Ms Thompson left online trails of her hacker activities. She is listed as an organizer for “Seattle Wares Kiddies”, a group on Meetup, which lead to her online identities at other social media such as Twitter and Slack. The nick “erratic” was identified back to Ms Thompson as she had previously posted a photograph of an invoice for a veterinarian care services.

Ms. Thompson was quoted saying that “I’ve basically strapped myself with a bomb vest” in a related Slack posting, according to the prosecutors. If convicted, Ms Thompson will face the possibility of a USD250K fine and up to 5 years jail term.

The victim (?)

Capital One had anticipated that they would be incurring loss of up to USD150 million, which includes paying for the customer’s credit monitoring services. The credit monitoring services and identity protection services is offered as part of compensation for those affected.

Capital One may also be facing potential regulatory fines/sanctions, which at this point of time is still undetermined, as well as lawsuits.

New York Times was also seen to report that Amazon has refused any blame as part of the incident. Amazon told Newsweek that “this type of vulnerability is not specific to the cloud“. Misconfiguration, be it at the application or data bucket layer seems to be leading cause of data theft from cloud infrastructures, as seen the past such as Attinuty. Amazon maintains that “you choose how your content is secured“.

Situational Analysis

SocMed seems to be abuzz about whether the focus should be on the attacker, since its a criminal offense, while Capital One walks free. While the attacker may have done crime, the question is, could it have been prevented?

From a criminal aspect, what Ms Thompson did is illegal. The proof, which seemly handed by Ms Thompson herself, due to number of posts/articles, as well as poor opsec due to posting of the invoice. The public persona of Ms Thompson indicates her leaning towards hacking, and postings on the social media channels indicate admission. The prosecutors would have all the necessary evidences to convict Ms. Thompson, following the digital trail. In my opinion, seems like the prosecutors have an open/shut case in their hands.

Capital One was also dissected on social media for it’s role in the incident. The question remains if Capital One had done everything it possibly could to ensure such issues do not occur. Reading from the press release, it seems that Capital One looks to “augment routine automated scanning to look for this issue on a continuous basis”. Not sure how to interpret that, whether a routine automated scan has been recently introduced, or whether the scan itself was enhanced to include misconfiguration related issues.

What’s next?

Companies with cloud presence has a different set of security concerns to address. While traditional on-prem presence seems to indicate better control. Some quick action items to be done for organizations concerned with such issues

I. Train your staff on cloud security. It can be provider specific as well as provider agnostic.

II. Providers such as Amazon/Azure has configuration templates which can be used to securely roll out services. These configurations are secure by default and will not allow any insecure setup. Insecure setup should be reviewed and follow internal process for deployment and approval.

III. Deploy tools to check for misconfiguration on a periodic basis.

IV. Separate instances based on type environment – Development/Testing/Production

V. Enforce strict IAM/PAM (Identity Access Management/Privilege Access Management) to ensure access is managed effectively

References

  1. New York Times – https://www.nytimes.com/2019/07/29/business/capital-one-data-breach-hacked.html
  2. TechRadar – https://www.techradar.com/sg/news/capital-one-hit-by-major-data-breach
  3. CNET – https://www.cnet.com/news/capital-one-data-breach-involves-100-million-credit-card-applications/
  4. US Dept of Justice – https://www.justice.gov/usao-wdwa/pr/seattle-tech-worker-arrested-data-theft-involving-large-financial-services-company
  5. Capital One – http://press.capitalone.com/phoenix.zhtml?c=251626&p=irol-newsArticle&ID=2405043
  6. NewsWeek – https://www.newsweek.com/amazon-capital-one-hack-data-leak-breach-paige-thompson-cybercrime-1451665
  7. US Department of Justice – Case details – https://www.justice.gov/usao-wdwa/press-release/file/1188626/download

Do you need BCP for Cloud?

I woke up feeling very warm. I thought I missed the alarm, but its just 3:23 am. Very sure I don’t need a potty break, extremely sleepy and obviously upset. Leaned over to see the AC (air-condition), and I found that it was off. I’m very sure its too warm and by now the AC should have kicked in. Mumbling, I woke my already tired and weary body and walked towards the thermostat to see what’s happening.

After blinking a few times to get my sight back to normal, I found that the Nest thermostat isn’t working. Walking back to my bedside table to grab my phone (I know, it’s a bad habit), I checked to see if the internet was down. WiFi seems up, checked my public IP (instead of good ol’ ping), everything seems okay. Google search shows up okay. Still with sleep in my head, I rummaged through my bedside drawer for the remote and turned it on. “This is too much work” – grumbled my half sleepy head. That’s enough for the night.


Woke up in the morning with a sleep hangover (yes, its possible, when you don’t have enough sleep), I was trying to figure out what happened. Turned on twtr and true enough, reports on Google Cloud services failure starts trickling in.

URL: https://status.cloud.google.com/incident/compute/19003

The horror! Google Cloud services went down?

*My panicked head screaming – The sky has fallen! The Sky has fallen!*

This pretty much explains why the thermostat went down. I wondered how may threat actors lost their C2 hosted on Google Services, how many IOT devices like the Nest Thermostat stopped working and other dependent service. If as an end user I am grumbling on the service availability, how about corporate organisations relying on Cloud services ?


Today’s organization rely heavily on cloud. Business today runs on cloud. Social media runs on cloud. Almost everything runs on cloud. Whether it’s server/virtual servers, serverless, functions (you name it), runs on cloud. (Disclaimer, most of my stuff also runs on cloud…)

But, is cloud outage a rarity? Well it depends on what you deem as rare. The Internet forgives, but never forgets. In August 25, 2013, AWS suffered an outage, bringing down Vine and Instagram with it. March 14, 2019, Facebook went down, bringing WhatsApp together in an apparent server configuration change issue.

The impact is obvious, business will lose revenue when the services goes down. Local franchise such as AirAsia, runs their kit mostly on Cloud. The impact is devastating, imagine booking of flights goes dark. So does a lot of other business. Hence this brings an interesting point: What is your business continuity plan if cloud goes down?

When I had this conversation a few years ago, most CIOs I spoke to boldly claim that their BCP is the cloud (we never reached the part about cloud and security because its most often dominated by the cost debate). There is no need, due to the apparent global redundancies of cloud infrastructure. The once-sleeping-soundly-at-night CIOs are now rudely awaken (just like me, due to the broken thermostat) that cloud no longer offers the comfort they can afford, after investing years of CAPEX (capital expenditure) and happily paying cloud services their monthly dues to show that their services are up.

Few points to note for those interested in even thinking about Cloud BCP. Yes, its time we take the skeletons out of the closet and start talking about this.

Firstly, can your application and services run a completely different cloud provider? Let’s look at the layers of services before we answer this question.

XKCD - The Cloud

If you are running server images (compute cloud), it’s completely possible to run in a different cloud provider. You’ll need to be able to replicate the server image across cloud provider. You can archive the setup of your cloud server via scripts, create a repository to host your configuration files and execute the setup script to bring up the services in a separate cloud provider. The setup and configuration can be hosted in a private git/svn repository and called up when needed.

What about data? Most database services provide for replication and data backup services. For “modern” database services, data can be spread across multiple database for better data availability and redundancies.

The actual stickler for hybrid cloud is serverless/function based hosting. If the organization invests heavily in one particular cloud provider’s technology (without naming any particular provider), then it depends on the portability of that technology. If something common such as Python is used, the portability is pretty much assured. Technologies that are exclusive for a cloud provider will have issues of portability across different cloud providers.

Another question that needs to be answered is, how would you “swing” your services across different cloud providers? A common approach for internet availability is to use DNS services. Using DNS, the organization can change the location of services by changing the DNS records. This would allow seamless failover without having to change the URL. However, speed of failover will be determined based on the DNS TTL (time-to-live) configuration of that record. Too low, your DNS will be constantly hit with queries, but changes are almost instantaneous (usually a low TTL is around 15 to 30 minutes). Too high, your DNS infrastructure will have low traffic, but takes a long time before the failover actually happens. DNS based failover also creates administrative headache for firewall administrators as they have to change their approach from IP based to a DNS based access control list.


All of cloud isn’t just hot air. Moving towards Industry 4.0 (now I’m just throwing buzzwords around), Cloud adoption is definitely a core component of the technology strategy that each organisation needs to have. As times goes by, we find that even cloud is fallible, hence a proper approach towards Cloud is key in business continuity.

So, what’s your approach towards Cloud Services BCP?