Meltdown / Spectre Mitigation Is a Work in Progress
Last updated on: September 6, 2020
Since researchers disclosed the Meltdown and Spectre vulnerabilities on Jan. 3, vendors and IT departments have been consumed trying to figure out how to properly address the potentially devastating effects of these kernel-level bugs.
By now, one thing we know for sure is that dealing with the vulnerabilities is a moving target. This situation is compounded by the fact that they have broad implications and that every day seems to bring new, relevant information that must be factored into ongoing mitigation efforts.
Thus, it’s important to stay on top of the latest developments, so we’re providing a snapshot of what we know to date, how Qualys can help and and what InfoSec teams can do. We’re also tracking a list of Qualys resources.
Meltdown and Spectre basics
If exploited, these vulnerabilities can give hackers unprecedented access to compromised systems and widespread liberty to steal a broad variety of confidential, sensitive data. Their severity, complexity and scope puts them among the most dangerous ever.
Namely, the vulnerabilities affect virtually all processors made by Intel dating back to 1995. Some chips from ARM Holdings and from Advanced Micro Devices are also affected.
As Qualys’ Product Management Director Jimmy Graham explained in a blog post, Meltdown, which requires OS patches, and affects primarily Intel chips, allows any application to access all system memory, including memory allocated for the kernel.
Meanwhile, Spectre allows an application to force another application to access arbitrary portions of its memory, which can then be read through a side channel. Underlying the vulnerability is a chip technique called “speculative execution,” devised to optimize performance. This vulnerability will require microcode updates and software patches in order to fully mitigate, according to Graham. In addition to Intel chips, Spectre also impacts AMD and ARM chips.
No easy fixes
What’s clear is that dealing with Meltdown and Spectre is a learn-as-you-go situation, as evidenced by multiple missteps in remediation efforts so far, with the aggravating factor that the stakes are sky high.
The IT industry may be dealing with these vulnerabilities for years, as many think, including U.S. CERT, that the only way to fully eliminate the risk is to replace the faulty processors.
“It’s kind of a worst nightmare from a security-response perspective,” Paul Kocher, one of the independent security researcher who discovered the vulnerabilities told The Wall Street Journal. “There is going to be vulnerable hardware around for a very long time.”
There are multiple vectors for the vulnerabilities, including browsers, operating systems, applications, and processor firmware. Thus, in addition to Intel, there are many vendors involved in the patching and mitigation efforts, including Apple, Microsoft, Mozilla, Amazon, IBM, VMware, Linux OS vendors, and Intel OEM partners, among many others.
As Wired explained in an article: “Lots of vulnerabilities require large-scale patches. But Meltdown and Spectre are unique in that they involve overhauls of both standard operating system software, and more rare updates to the firmware and microcode that coordinate and control hardware.”
IT departments have their work cut out for them. “This is a nasty one,” Harjit Dhaliwal, a senior systems administrator in the higher-education sector who handles patching for his environment, told TechTarget. “This is not one of your normal vulnerabilities where you just have a patch and you’re done.”
After the first wave of patches, we’re already starting to see attempts to refine the mitigation efforts, such as Google’s Retpoline binary modification technique. A bit of good news is that there have been no reported attacks exploiting the vulnerabilities in the wild yet.
Performance slowdowns, bricking
Some of the main issues that have been reported so far include:
- Performance impacts caused by patches, ranging from mild to severe
Last week, Microsoft acknowledged that Windows OS patches it had released for Meltdown and Spectre slowed down PCs and servers where they were installed.
Meanwhile, Intel said it had received reports from some customers — including data center operators — of a reboot problem after installing firmware updates for Broadwell and Haswell CPUs. Update Jan. 22: After investigating the issue, Intel told OEMs, cloud service providers, system manufacturers, software vendors and end users on Monday Jan. 22 to stop deployment of current versions of the firmware updates until further notice, as they may introduce higher than expected reboots and other unpredictable system behavior.
Microsoft also has acknowledged a “compatibility issue” with some anti-virus products. VMware pulled its ESXi patches after problems were reported.
Meanwhile, reports from the field are starting to crop up about the performance impact of the patches in real world scenarios, including from SolarWinds, EpicGames, a tech journalist who owns a Microsoft Surface Book, and several industrial equipment manufacturers.
- Full system disruption after patches
This happened to users of PCs running AMD Athlon CPUs after installing a Windows patch, as well as to users of Ubuntu systems.
- Concern about public cloud platforms
There is special worry about the cloud platforms that collectively host mission-critical workloads from millions of businesses, and data from hundreds of millions of consumers. First, the vulnerabilities can allow an attacker to bypass virtualized partitions, making it possible to steal data from all virtual machines on a single server. Second, there’s concern that in massive data center environments, the performance degradation from the patches would be exponentially replicated, leading to serious slowdowns of applications and web services.
- As would be expected, cyber criminals have started to attempt to trick users into installing fake Spectre and Meltdown patches that are really malware, as Malwarebytes Labs recently warned.
Best practices from Qualys
At Qualys, our vulnerability management perspective is that the patches that have been issued so far by the OS vendors amount to mitigations and workarounds. The patches themselves are complex, and compatibility issues should be expected. For example, anti-virus software is deeply embedded in systems and kernels. A change in how kernel memory is stored will certainly affect anti-virus products.
Every company will have their own risks in terms of operational risk versus security risk. For that reason, Qualys believes that it may be better for some organizations not to patch and instead use a different compensating control to mitigate exposure as much as possible.
There’s a greater sense of urgency with Spectre, because exploiting Meltdown requires having a foothold on the targeted system, as Qualys’ Graham told ThreatPost. Spectre opens up certain types of remote attack scenarios, which could result in compromising credentials and session keys, allowing hackers to bypass many security protections.
As Graham noted in a blog post, Qualys has released several QIDs that detect missing patches for these vulnerabilities across several operating systems. A list of currently-released QIDs is being maintained in this Qualys Support article. The QIDs are supported by both authenticated scanning and the Qualys Cloud Agent.
In addition, Qualys AssetView now has a dashboard with preloaded widgets that can help track remediation progress as customers patch against Spectre and Meltdown. These widgets were built with out-of-the-box functionality, and can be imported into any Qualys subscription. The dashboard file and instructions on how to import the dashboard are in this Qualys Community post.
Stay tuned for future updates, recommendations, and best practices related to Meltdown and Spectre, and for information about how Qualys products can help. View the recorded webcast or read the summary to learn more.
Spectre / Meltdown Resources
- How to Inventory Systems and Track Patching Progress
- January 17 Webcast Recording and Summary
- Visualizing Impact and Remediation Progress with AssetView
- Configure Spectre/Meltdown Dashboards in AssetView
- List of QIDs for Meltdown & Spectre
- Spectre/Meltdown Search Lists, Scan Option Profile, Remediation Tracking and Patch Reports
- Protecting the Qualys Cloud Platform
- March 30: Patch for the Meltdown Patch
- March 13: Patch Tuesday includes Meltdown / Spectre Patch
- February 16: Patch Tuesday and Meltdown / Spectre Keep IT Departments On Edge
- February 9: Intel Makes Spectre Patch Progress
- February 2: New Concerns Over Intel Patches, as Hackers Test Exploits
- January 26: Intel Nixes Patches, Tech CEOs Questioned on Information Blackout
Kernel level bugs? I always referred to Meltdown & Spectre as silicon level bugs!
Thanks for your wonderful article, This Spectre, and Meltdown problem has not ended even after a series of patches from Linux, Microsoft updates, and Intel microcodes. Also, some versions of an ARM are affected. https://www.infosecuritycenter.com/data-protection/meltdown-and-spectre-bugs-against-healthcares-data-security-thrive/