Unified Vulnerability View of Unauthenticated and Agent Scans
Last updated on: June 13, 2023
Table of Contents
Vulnerability scanning has evolved significantly over the past few decades. But the key goal remains the same, which is to accurately identify vulnerabilities, assess the risk, prioritize them, and finally remediate them before they get exploited by an attacker.
Therein lies the challenge. It is easier said than done. There are multiple ways to scan an asset, for example credentialed vs. uncredentialed scans or agent based vs. agentless. Regardless of which scanning technique is used, it is important that the vulnerability detections link back to the same asset, even if the key identifiers for the asset, like IP address, network card, and so on, have changed over its lifecycle.
The next few sections describe some of the challenges related to vulnerability scanning and asset identification, and introduce a new capability which helps organizations get a unified view of vulnerabilities for a given asset.
Update: Recording available on demand for the webinar on February 17, 2021: New Unauthenticated and Agent-Based Scan Merging Capabilities in Qualys VMDR.
Vulnerability Scanning Challenges
Unauthenticated Scanning
In the early days vulnerability scanning was done without authentication. Unauthenticated scanning provides organizations with an attacker’s point of view that is helpful for securing externally facing assets. However, it is less helpful for patching and remediation teams who need to confirm if a finding has been patched or mitigated. Unauthenticated scanning also does not provide visibility when an attacker gains unauthorized access to an asset. In addition, these types of scans can be heavy on network bandwidth and cause unintended instability on the target, and results were plagued by false positives. Over the last decade, Qualys has addressed this with optimizations to decrease the network and targets impact while still maintaining a high level of accuracy. Finally unauthenticated scans lack the breadth and depth of vulnerability coverage that authenticated scan results provide, so organizations began to use authenticated scans.
Authenticated Scanning
Now let us compare unauthenticated with authenticated scanning. The screenshots below show unauthenticated (left) and authenticated (right) scans from the same target Windows machine. Based on the number of confirmed vulnerabilities, it is clear that authenticated scanning provides greater visibility into the assets.
Diving into the results from both scans, we can quickly see the high-criticality vulnerabilities discovered.
Although authenticated scanning is superior in terms of vulnerability coverage, it has drawbacks. Due to change control windows, scanner capacity and other factors, authenticated scans are often completed too infrequently to keep up with the continuous number of CVEs released daily. These point-in-time snapshots become obsolete quickly. In addition, routine password expirations and insufficient privileges can prevent access to registry keys, file shares and file paths, which are crucial data points for Qualys detection logic.
Agent-Based Scanning
Agent-based scanning solves many of the deficiencies of authenticated scanning by providing frequent assessment of vulnerabilities, removing the need for authentication, and tracking ephemeral and moving targets such as workstations.
However, agent-based scanning has one major disadvantage: its inability to provide the perspective of the attacker. These network detections are vital to prevent an initial compromise of an asset. Although agent-based scanning is fast and accurate, it lacks the ability to perform network-based checks and detect remote vulnerabilities identified by unauthenticated network scans. As a result, organizations have begun to use a hybrid approach of agent-based and unauthenticated scans to scan assets.
Asset Mapping Challenges
Agent-based scanning had a second drawback used in conjunction with traditional scanning. It resulted in two sets of separate data because there was no relationship between agent scan data and an unauthenticated scan for the same asset. The duplication of asset records created challenges for asset management, accurate metrics reporting and understanding the overall risk for each asset as a whole.
Historically, IP addresses were predominantly static and made for an easy method of uniquely identifying any given asset. With the adoption of RFC 1918 private IP address ranges, IPs are no longer considered unique across multiple networks and assets can quickly change IPs while configured for DHCP. MAC address and DNS names are also not viable options because MAC address can be randomized and multiple assets can resolve to a single DNS record.
The below image shows two records of the exact same asset: an IP-tracked asset and an agent-tracked asset. Two separate records are expected since Qualys takes the conservative approach to not merge unless we can validate the data is for the exact same asset.
Introducing Unified View and Hybrid Scanning
Given the challenges associated with the several types of scanning, wouldn’t it be great if there was a hybrid approach that combined the best of each approach and a single unified view of vulnerabilities?
To resolve this, Qualys is excited to introduce a new asset merging capability in the Qualys Cloud Platform which just does that. It allows users to merge unauthenticated scan results with Qualys Cloud Agent collections for the same asset, providing the attacker’s point of view into a single unified view of the vulnerabilities. This new capability supplements agentless tracking (now renamed Agentless Identifier) which does similar correlation of agent-based and authenticated scan results. Now your agent-based, unauthenticated and authenticated scan data is merged for a comprehensive view of the posture of each asset without asset duplication.
As seen below, we have a single record for both unauthenticated scans and agent collections.
Merging Unauthenticated and Scan Agent Results
Here’s How it works:
Qualys Cloud Platform:
- Accept the Agent Correlation Identifier and the Qualys Cloud Platform will merge results from unauthenticated scans and agent collections for the same asset using a Correlation ID to uniquely identify the asset record to merge scan results.
- Once the results are merged, it provides a unified view of asset vulnerabilities across unauthenticated and agent scans.
Qualys Cloud Agent:
- Update or create a new Configuration Profile to enable Agent Scan Merge allowing the Qualys scanner to collect the Correlation Identifier during an unauthenticated scan.
- During an unauthenticated scan using the Qualys scanner, the Cloud Agent will return its Correlation ID to scanner over one of the Agent Scan Merge ports (10001, 10002, 10003, 10004, 10005).
Qualys QIDs:
- Qualys has released an Information Gathered QID (48143 Qualys Correlation ID Detected) that probes the agent on the above-mentioned Agent Scan Merge ports, during an unauthenticated scan, and collect the Correlation ID used by the Qualys Cloud Platform to merge the unauthenticated scan results into the agent record.
Step-by-step documentation will be available.
As technology and attackers mature, Qualys is at the forefront developing and adopting the latest vulnerability assessment methods to ensure we provide the most accurate visibility possible. Qualys is actively working to support new functionality that will facilitate merging of other scenarios. Unifying unauthenticated scans and agent collections is key for asset management, metrics and understanding the overall risk for each asset. We hope you enjoy the consolidation of asset records and look forward to your feedback.
Learn More
Webinar February 17, 2021:
New Unauthenticated and Agent-Based Scan Merging Capabilities in Qualys VMDR
Frequently Asked Questions (FAQ’s)
Does Agent Correlation Identifier replace Agentless Identifier (previously known as Agentless Tracking)?
No. Agentless Identifier behavior has not changed. In fact, these two unique asset identifiers work in tandem to maximize probability of merge.
Under what scenarios would merging of unauthenticated scan results into the agent record fail?
If the scanner is not able to retrieve the Correlation ID from agent, then merging of results would fail. This could be possible if the ports listed above are not reachable by the scanner or a scan is launched without QID 48143 included in the scan. Agent Scan Merge Cases documents expected behavior and scenarios.
Customers should ensure communication from scanner to target machine is open.
What are the timelines for this functionality to be available?
The solution is dependent on the Cloud Platform 10.7 release as well as some additional platform updates.
As of January 27, 2021, this feature is fully available for beta on all Qualys shared platforms.
Is this feature enabled by default at time of release?
The feature is available for subscriptions on all shared platforms. Customers need to configure the options listed in this article by following the instructions in Get Started with Agent Correlation Identifier.
How would a user configure the new merging option using Correlation ID identifier?
Customers can accept the new merging option by selecting “Agent Correlation Identifier” under Asset Tracking and Data Merging Setup.
Does each Option Profile need to be reconfigured to use Agent Scan Merge ports?
No. Once Agent Correlation Identifier is accepted then these ports will automatically be included on each scan. This is not configurable today.
Can this feature be enabled for only certain assets?
Yes. To enable this feature on only certain assets, create or edit an existing Configuration Profile and enable Agent Scan Merge. Then assign hosts based on applicable asset tags. It’s also possible to exclude hosts based on asset tags.
How can customers secure access to the ports enabled by Cloud Agent?
The agent can be limited to only listen on the ports listed above when the agent is within authorized network ranges. Qualys is calling this On-Premises Detection and can be configured from the UI using Configuration Profiles.
What happens if Agent Scan Merge ports (10001, 10002, 10003, 10004, 10005) are all in use by other services?
Qualys believes this to be unlikely. In the case this does occur, Agent Scan Merge ports are customizable and user should select an alternative. Note: which ever service binds first will be available. If the Correlation Identifier binds first then the other competing service will not be allowed to bind to the port.
How would merging results for agentless assets work?
Customers should leverage one of the existing data merging options to merge results from assets that don’t have agents installed. In addition, we are working to support new functionality that will facilitate merging of data based on custom correlation rules.
Which versions of Cloud Agent are required to have this functionality?
- Windows Agent version 4.2 or later
- Linux Agent version 3.1 or later
- Only Linux and Windows are supported in the initial release.
What happens to my current duplicate records?
- The merging will occur from the time of configuration going forward.
- Qualys will not retroactively clean up any IP-tracked assets generated due to previous failed authentication.
- See how to identify stale IP-tracked assets.
Can I give feedback on this feature?
Yes. Please fill out the short 3-question feature feedback form.
Or participate in the Qualys Community discussion.
This is a great article thank you Spencer,
Where cloud agent is not permitted in our environment, QID 90195 is a routine registry access check within our environment. Whilst authentication may report successful, we often find that misconfiguration on the device may cause many registry keys to be inaccessible, esp those in the packages hives.
It’s also very true that whilst a scanner can check for the UUID on an authenticated scan, it cannot on a device it fails authentication on, and therefore despite enabling the Agentless Tracking Identifier/Data merging, you’re going to see duplicate device records. It’s therefore fantastic that Qualys recognises this shortfall, and addresses it with the new asset merging capability.
Can’t wait for Cloud Platform 10.7 to introduce this.
Excellent post. The question that I have is how the license count (IP and VM licenses used with the agent) are going to be counted when this option is enabled?
@Alvaro, Qualys licensing is based on asset counts. Merging records will increase the ability to capture accurate asset counts
Also for the ones that are using authenticated scanning (or plan to) would this setting make sense to enable or if there is a reason why we should not if we have already setup authenticated scanning.
Hi Alvaro,
You can enable both (Agentless Identifier and Correlation Identifier). These two will work in tandem. It will increase the probability of merge
Agent based scans are not able to scan or identify the versions of many different web applications. In fact, the list of QIDs and CVEs missing has grown.
If you want to detect and track those, you’ll need an external scanner. And an even better method is to add Web Application Scanning to the mix.
Unfortunately, once you have all that data, it’s not easy at all to compile, export, or correlate the data from within Qualys.
Hi Dev Ops,
Happy to take your feedback. You can email me and CC your TAM for these missing QID/CVEs.
Hello Spencer
after enabling this in at the beginning of march we still see 2 asset records in Global asset inventory (one for agents and another for IP tracked records) in Global IT asset inventory. is that the correct behaviour? my expectaiton was that when i search for assets i shold only see a single record
Hi Alvaro, I have followed up via email.
Hello Spencer / Qualys team – on article https://qualysguard.qg2.apps.qualys.com/qwebhelp/fo_portal/host_assets/agent_correlation_identifier.htm is mentioned “Note: Qualys does not recommend enabling this feature on any host with any external facing interface” = can we get more information on this, what issues might cause and such? Is a bit challenging for a customer with 500k devices to filter for servers that has or not external interface :)