Qualys Performance Tuning Series: Remove Stale Compliance Data for the Best Performance

In our first post in the Performance Tuning Series, we talked about removing stale assets to improve performance. In this installment, we will address the benefits of removing data once it becomes stale.

Why does data become stale?

The IT environment of any enterprise is very dynamic, and more so with increased usage of virtualization and cloud services. Additionally, the compliance environment is continuously changing, with newer frameworks, policies, and controls being applied to the IT assets. A typical list of such changes is:

• New assets are provisioned, or existing assets deprovisioned
• New technologies are installed on an asset, or existing technologies are uninstalled or updated.
• Technologies or operating systems may be upgraded
• Applications or assets may move across environments or data centers
• The association of assets to policies may change
• Controls may be added to a policy or removed from a policy

Why should you worry about stale data?

All these changes pose a unique asset and policy compliance challenge. Stale assets and the resulting stale compliance postures are something that we encounter regularly when working with our customers. The most significant issue caused by stale data is the decline in data accuracy, which affects your reports and dashboards.

Stale data:

• Decreases accuracy
• Impacts your compliance posture
• Reduces Performance
• Causes data bloat in customers’ data stores

Reporting on assets and postures that no longer exist in your environment causes IT teams to chase misconfigurations that aren’t there anymore (which impacts mean time to remediation (MTTR)), obscures an enterprise’s overall compliance posture, and results in management losing confidence in and starting to question the data. This is an easily avoidable problem, with the automated purge features available in Qualys able to remove this stale data automatically.

Qualys’ Stale data utility

To make it easier for customers to track stale data in their subscriptions, we have created  an enhancement in the platform that automatically purges stale data from instances and technologies that have not been scanned in a customer-defined period of time. This configuration can be specified via the UI, as shown below

This utility reports on the following types of stale data:

• Scan data from assets that have not been refreshed
• Evaluated postures for assets that have not been updated
• Exceptions that have expired prior to the configured duration

Customers can review this report of stale data and proceed with purging it. During the initial rollout, the Qualys Support team will work with the customer to provide the report on stale data and handhold the purging process. Once the initial data purge is done, we will set this up as an automatic purge job to regularly perform the housekeeping task of cleaning up this old data.

Purge rules and recommended configuration

Qualys recommends that customers set this configuration to a value greater than five times their scan frequency or 30 days, whichever is lower. The purge rules implemented are:

• Qualys scans all assets via scanner or agents at a configured frequency ranging from four hours to sometimes as much as a week. Hence, it is expected that each asset will report back on compliance data in this interval. Compliance data related to the asset, technology, or control will not be received in situations where:
  • The asset is decommissioned.
  • A technology on the asset is uninstalled or upgraded.
  • The asset is removed from a policy.
  • A control is removed from a policy.

In such cases, after waiting for about 30 days, the compliance scan and evaluated posture data will be purged

• When exceptions are configured with a defined expiry, after an additional safety period of the defined duration, the expired exceptions will be purged

This Purge Stale Compliance Data article is part of our Best Performance Series. This series is intended to show how to optimize and effectively manage your Qualys subscription and use it in process automation. For questions and further assistance, schedule a time to meet with our Solutions Architects and Product Managers; we are here to help.