Capital One: Building Security Into DevOps
Last updated on: September 6, 2020
Capital One prides itself on staying at the forefront of IT innovations to give its business a competitive edge.
For example, it adopted Agile software-development methodologies years ago, and uses artificial intelligence and machine learning. It was the first bank to implement a mobile wallet with “contactless” NFC payments, and to offer voice-activated financial transactions using Amazon’s Alexa. When 2018 ends, Capital One expects 80% of its IT infrastructure to be cloud based, allowing it to go from seven to two data centers.
Given its tech transformation track record, it’s not surprising that Capital One has embraced DevSecOps, embedding automated security checks into its DevOps pipeline. This effort has dramatically accelerated the process of assessing vulnerabilities and mis-configurations in its virtual machine images and containers.
As a result, the code created in the DevOps pipeline is certified as secure and released to production without unnecessary delays. This allows Capital One — one of the United States’ 10 largest banks, based on deposits — to consistently boost its business across the board by quickly and continuously improving its web properties, mobile apps, online services and digital offerings.
“This has provided a huge benefit to the entire company,” said Emmanuel Enaohwo, Senior Manager for Vulnerability/Configuration Management at Capital One, a Fortune 500 company based in McLean, Virginia that offers a broad spectrum of financial products and services to consumers, small businesses and commercial clients.
Read on to learn how the bank has automated vulnerability and compliance checks in its CI/CD software pipeline, helped by Qualys.
Building a secure AMI bakery
Initially, Capital One’s process was manual: After creating an Amazon Machine Image (AMI), the DevOps team emailed the vulnerability and configuration management team, requesting a security assessment.
The security team would perform the assessment and create a report, which the DevOps team would use to carry out remediation. Then it would re-send the AMI for another check, which usually uncovered new issues. They’d go back and forth in a “fix / find / verify” loop, which typically took up to two weeks to complete.
Capital One wanted to shorten this security certification process for AMIs. The bank found the solution without having to look very far: The DevOps team was given API access to the security team’s Qualys vulnerability management and policy compliance tools.
This allowed developers to run the scans themselves, get a report instantly, start remediating immediately, and re-scan as needed, without involving the security team. Everything sped up dramatically.
“As soon as we introduced Qualys’ APIs into the environment, we cut the time to less than 24 hours,” Enaohwo said.
Capital One also seeds the Qualys Cloud Agent on every AMI that’s going to be deployed to production, so it’s alerted immediately about newly-discovered security and compliance issues on live instances.
After doing an initial comprehensive assessment of the IT, security and compliance status of each AMI, the agent from then on only reports changes, and it does so as soon as it detects them.
“In traditional VM programs you’re lucky if you scan once a week. Most people scan once a month. With the Qualys Cloud Agent, it’s almost real time,” Enaohwo said.
The Cloud Agent is lightweight, consuming negligible computing resources, and its OS support includes Windows, Linux, MacOS, and “cloud native” platforms, including AWS. It works in assets that are on premises, in clouds and remote endpoints.
There are no scan windows, since it’s always collecting data on assets it’s installed on, even when assets are offline. Since it only communicates outbound to the Qualys platform, there’s no need for credential and firewall management.
By installing the Cloud Agent on almost every AMI that passes through its DevOps “bakery” — excluding some third-party AMIs — Capital One has achieved 95% assessment coverage of its IP addresses.
“That’s something we couldn’t do before Qualys,” he said.
The agent has also boosted accuracy of detection of vulnerabilities and mis-configurations, slashing the number of false positives, and quickened scan data availability.
“All these KPIs (key performance indicators) are met because of the integration with DevOps using the Qualys Cloud Agent and APIs,” he said.
Next: Securing containers
Like other IT innovators, Capital One uses Docker containers to add speed and flexibility to its application development and delivery.
Containers are lighter and more ephemeral than virtual machines because they can be spun up without a guest operating system for each one. Applications can be smaller and more portable.
Containers have helped popularize microservices, a new architecture where applications are structured as independent, small, modular services, and which Capital One is leveraging.
However, containers bring security challenges, such as the use of un-validated software pulled from public repositories, which often contains unpatched vulnerabilities and weak configurations.
Recognizing the importance of securing container environments, Capital One is automating security and compliance checks on them similarly to how it’s doing it for AMIs.
Capital One chose Qualys Container Security, a cloud app that gives organizations continuous discovery, tracking and protection of containers in DevOps pipelines and deployments. Qualys Container Security’s native sensor is available as an image and is deployed as a ‘side-car’ unprivileged container on hosts.
Capital One is using Qualys Container Security’s plug-in for the Jenkins CI/CD (continuous integration / continuous delivery) tool, so that DevOps teams can scan container images themselves, and fix them immediately.
It was also familiar for members of the DevOps team acquainted with the AMIs’ automated security checks, because the same workflow and QIDs (Qualys IDs) are involved.
The role of the security team
Delegating vulnerability and compliance assessments to the DevOps team not only accelerated the AMI and container pipeline: It also has allowed the Capital One security team to focus on higher-level tasks, such as discovering assets and improving the accuracy of its CMDB’s data.
The team also establishes the security and compliance standards and thresholds that AMIs must meet before going live, and retains oversight of the entire process, including configuration-hardening procedures and vulnerability identification.
Enaohwo’s team also drafts reports from the scan data, customized for different constituencies, including the board of directors. It helps business units analyze scan findings, and generate remediation strategies and plans. The team also tracks remediation progress.
On the governance side, it’s responsible, among other things, for interfacing with compliance and audit, and handling documentation and process mapping.
“I always tell my team we’re not meant to do operations. We’re meant to drive change,” he said.