Despite the huge advantages that containers offer in application portability, acceleration of CI/CD pipelines and agility of deployment environments, the biggest concern has always been about isolation. Since all the containers running on a host share the same underlying kernel, any malicious code breaking out of a container can compromise the entire host, and hence all the applications running on the host and potentially in the cluster.
That fear of container isolation failing to hold up turned out to be true yesterday when a vulnerability in runC was announced. runC is the key and most popular software component that most container engines rely on for spinning up containers on a host. The announced vulnerability allows an attacker to break out of the container isolation through a well-crafted attack (technical details of the vulnerability and the exploit are at https://seclists.org/oss-sec/2019/q1/119) and compromise the entire host. The vulnerability is particularly nasty because it is not covered by the default AppArmor or SELinux kernel-enforced sandboxing policies.
What can you do to protect your containerized applications?
Even though the exploit is tricky to execute, the exploit code will be released publicly on February 18, so it’s best to protect your container environment by doing the following:
- Know which nodes (Docker hosts) you are running the containers, and if you are running a vulnerable version of Docker Engine. If you are a Qualys customer, you can use AssetView to get that information. Docker has released the patch in version 18.09.2.
- Upgrade your Docker hosts to version 18.09.2.
- For hosts managed by public cloud service providers, please keep a close watch on how they are addressing the issue.
GCP – https://cloud.google.com/kubernetes-engine/docs/security-bulletins
AWS – https://aws.amazon.com/security/security-bulletins/AWS-2019-002/
- Qualys is working on releasing the following detections (QIDs), and more vendor-specific QIDs will be launched in the coming days.
237121 : Red Hat Update for docker (RHSA-2019:0304)
237120 : Red Hat Update for runc (RHSA-2019:0303)
351500 : Amazon Linux Security Advisory for docker: ALAS-2019-1156
371641 : Runc Container Breakout Vulnerability
You can get more details at Qualys Threat Protection.
What to do in the future?
It’s good to be concerned about any new technology while it matures, but it’s equally important to harden the application build and deployment workflows in order to prevent the attacker from getting an easy lead into exploiting the deployed containers.
- Ensure that only those container images that have gone through the defined compliance checks (related to vulnerabilities, packages, etc.) are deployed in production. As an example, you can use the Qualys Container Security solution to promote only those built images that pass the compliance checks on the build nodes.
- Privileged containers, if compromised, can bring down the entire container cluster. Hence, keep a close watch on all privileged containers running in your environment.
(Asif Awan is CTO for Container Security at Qualys)