Built-in Runtime Security for Containers

Security teams struggle with visibility into behaviors inside their running containers. Qualys is today announcing general availability of Container Runtime Security (CRS) to provide industry-leading visibility for running containers using an approach that is container-engine agnostic and layered into the container image. This provides runtime visibility and enforcement in containers through policy-based control of system calls (File, Network, Process behaviors). These capabilities can be used to address various use cases for container runtime security like file access monitoring, network micro-segmentation, vulnerability mitigation and virtual patching. CRS is an add-on to Qualys Container Security and is immediately available.

The Challenge of Securing Running Containers

The rise and adoption of containers has been rooted in the promise of workload isolation, application abstraction and immutability. While these specific aspects of containers do help in reducing the attack surface, a single insecure image could be instantiated many times over as separate running containers and thus create a widespread attack surface. The following diagram provides a high-level overview of the attack surface for containers.

The attack surface at a host level can be secured via traditional host based solutions (e.g. Host level agents). The in-container attack surface needs to be handled with a defense in depth approach that consists of scanning across the build-ship-run pipeline, fixing the related security findings and runtime security that protects the remaining attack surface.

There are various approaches to container runtime security.

• In-Container Approaches: Operating systems have security features like Linux’s AppArmor, SELinux, and SecComp, and Windows’ EMET, which can be used to enforce application behavior and reduce the workload’s attack surface. Some of these can be used inside containers as well. However, customers must devote considerable time, effort and specific knowledge to build workflows around these capabilities and operationalize a security program for container runtime security.
• Node-Based Approaches: There are several approaches to runtime security that leverage node-based solutions, such as at the orchestration layer (e.g. Kubernetes admission controller), and node-based (Node level agent, privileged container). However, these solutions are either too narrow (e.g. deployment only checks via Kubernetes admission controller) or they’re intrusive with a lot of overhead with regards to operational security requirements (e.g. security privileges, performance overhead for a node-level Man-in-The-Middle privileged container). More importantly these node based approaches don’t translate easily to newer Container-As-A-Service environments (e.g. AWS Fargate, Azure Container Instances, Google CloudRun) as there is no access to deploy an agent at the host/node level. Some Container-As-A-Service environments are starting to provide security hooks but these are still at an early stage and not standardized across your container environments.

The opportunity here is to provide a turnkey solution that conveniently allows customers to monitor and enforce container behavior and works across all types of container environments including newer Container-As-A-Service environments.

The Qualys Solution

Qualys advocates for a defense in depth container security approach – consisting of scanning the build pipeline, container registries and running containers with its cloud-native container sensor and its unparalleled vulnerability knowledge base. Once the container attack surface is minimized via this scanning approach – a lightweight container friendly runtime security solution can be leveraged to protect the remaining attack surface. The figure below summarizes this approach.

The Qualys Container Runtime Security solution leverages a lightweight instrumentation approach that embeds security into the container image and ensures built-in security across many types of container infrastructure. The security instrumentation provides for a function-level firewall capability which allows for inspection, and gating of system calls based on user-defined granular behavioral policies.

Here’s a high-level overview of the Qualys Container Runtime Security workflow.

1. Image Instrumentation

The workflow for Container Runtime Security starts with instrumentation of the target container image. Qualys provides a lightweight microservice that can be leveraged in a customer environment to instrument application containers with Qualys’ security instrumentation. The Qualys instrumentation is lightweight, intercepts system calls in the container, evaluates if the system call should be allowed/blocked/monitored based on the assigned runtime policy and communicates accordingly with the Qualys Cloud backend.

2. Behavioral Policies for Container Runtime Security

Application containers spun up from instrumented application container images register with the Qualys Cloud Platform and obtain runtime policies and instrumentation configurations. These runtime policies and the Qualys instrumentation autonomously drive container behavior visibility and enforcement.

The Qualys solution allows users to specify policies to enforce in-container behavior. This involves specific rules as part of an overall policy. Each rule allows for allow/deny/monitor action type. The Qualys in-container instrumentation takes the action specified by the matching rule when evaluating a system call against the overall policy. Additionally, there is a default allow/deny policy setting for various types of un-matched behaviors – which allows for allowlist/blocklist type of use cases. Our solution provides the following rule types as part of a runtime security policy.

File Rule

As part of a file rule, users can specify program (process) name, path/file name and action type. The action type specifies what enforcement action to take when a system call with the specified file access parameters (process, path/file name, specific file access system call) is encountered.

Example use cases that can be addressed with a file rule:

• A policy to monitor access of ‘authorized_keys’ file to prevent SSH access by any process. This allows monitoring of private keys which are used for authenticating remote access services like SSH.
• A policy to monitor the read/write access to files associated with cron jobs. This would allow prevention of local cron job scheduling. Cron jobs are used to schedule the execution of programs either at system startup or after a specific time interval, for persistence.

Network Rule

This rule consists of two types:

• An inbound network rule that governs whether processes can run external-facing network services on a particular port, such as an ssh process running as a daemon on port 22 providing ssh inbound connectivity into the container. The following parameters need to be provided for this rule – port number, remote IP address, protocol and action type.
• An outbound network rule that allows for control of outbound network communications. The following parameters need to be provided for this rule: IP address, port, protocol and action type.

Example use cases that can be addressed with a network rule:

• Block any program/IP address from binding to port 22
• Block TCP communications on port 80 to a particular IP address

Application Rule

Lastly we provide an application rule type that allows for specifying a particular system call with associated arguments including program/process name. This rule type is a superset of the previous two rules and is intended for advanced users who understand the internals of Linux system calls.

Example use cases that can be addressed with an application rule:

• A policy that checks for the ‘write’ event on either of the specified files related to DNS resolution (hosts, resolve.conf files). This is useful in preventing modifications to ‘hosts’ and ‘resolve.conf’ files which can result in resolution of domain name to a malicious IP address.
• Prevent execution of the capsh Linux program. This policy will prevent any user from identifying the capabilities the container is running with. It can be leveraged for reconnaissance and in turn privilege escalation.

Combining Rule Types

Here are some examples of use cases where multiple rule types can be combined into a single policy.

• Block the LibMiner cryptomining malware in containers. LibMiner downloads and executes the components from an attacker’s server. It also drops the bash script ‘symcfget’ for persistence and registers it as a service. This policy consists of a number of file, network and application rules, such as rules to block communications to C&C servers, and to lock down malicious file access and execution. This blocks the execution and C&C communications of LibMiner.
• Isolate a container for quarantining purposes by enforcing network rules denying execution of various network system call operations

Thus, with these rule types, the Qualys solution allows for higher level formulation of rules, such as for file access and network communications, as well as more granular individual rules driven by system calls, such as application rules. The Qualys Cloud Platform translates these policies into a firewall rule table (of system calls) that’s used by the Qualys in-container instrumentation to enforce behaviors.

3. Event Management for Container Runtime Security

Once containers are spun up from instrumented container images, the Qualys instrumentation connects to the Qualys Cloud Platform and updates policies and configurations, if required. The instrumentation functions autonomously and sends telemetry back to the cloud. Users can view events in the Qualys Container Security module via their UI and also obtain them via an API for integration into their SIEM/SoC workflows. Additional filtering capabilities via event metadata are available via QQL (Qualys Query Language) queries.

Comprehensive Container Security Program

The diagram below shows how customers can put together a comprehensive container security program with scanning and runtime security across the build-ship-run container pipeline.

To summarize, the Qualys Container Runtime Security solution can be utilized by customers to put in place a standardized container runtime security program across many types of container infrastructure including newer Container-As-A-Service environments and address the following use cases:

• Enforce security best practices inside containers (e.g. No ssh access into running containers; not allowing the httpd process to run as a webserver on port 80 inside a database container)
• File access monitoring
• Network access control
• Mitigatory virtual patches for certain types of vulnerabilities and misconfigurations
• Monitoring of CRS telemetry for Indicators-Of-Attack (IoA) and Indicators-Of-Compromise (IoC), such as files accessed, processes run, and IP addresses communicated with.

More information on Container Runtime Security can be found in the user guide and the API guide.

Please contact your Qualys Technical Account Manager (TAM) to get access to a free trial of this new offering!

For more information on Qualys Container Security solutions and for a demo of Container Runtime Security, please register for the virtual Qualys Security Conference (Nov 9-24).