Optimizing a Web Application Security Scan for bWAPP  

Today almost all organizations have an online presence, with more information accessible at the click of a mouse, making customer experiences much more frictionless. Yet the delivery of great experiences also opens the door to potential hackers intent on compromising the website and its APIs. Luckily there are myriad resources available that help protect web applications by finding vulnerabilities (such as insecure coding errors) in either a manual or automated manner.  

Qualys Web Application Scanning (WAS) is designed to automate vulnerability scanning. However, there are configurations that may be better suited for certain apps than others. That’s why security testers often use open source applications to consistently evaluate the results of automated tools including Qualys WAS. One of these open source applications is bWAPP. 

bWAPP stands for “buggy web application.” It’s a free, open source, deliberately insecure web application. bWAPP helps security enthusiasts, developers, and students to learn web vulnerabilities using a hands-on approach.  

In this blog, we cover recommendations on configuring a Qualys WAS scan for bWAPP and provide reasons why each configuration is best suited for bWAPP. All these tips and tricks can be used for any application to help optimize results, improve coverage, and decrease scan time.   

Strengths of Qualys Web Application Scanning 

This blog assumes that the reader is familiar with running a Qualys WAS scan, but for those who aren’t, here are some reasons to try Qualys WAS for dynamic application security testing.  

Some of the strengths of Qualys WAS include:  

• Tests multiple parameters and forms at a much faster speed 
• Finds logic and forms based on the page source, which might not be as simple to find manually 
• Discovers multiple vulnerabilities in an automated manner 
• Uses multiple techniques to test web vulnerabilities 

Steps for Configuring Qualys WAS for bWAPP

Below we’ll cover the following configurations and topics: 

1. Setting asset details  

2. Selecting explicit URI 

3. Configuring an option profile  

4. Authentication of bWAPP 

5. Review results and diagnostic Qualys vulnerability IDs (QIDs) 

Note: This blog will not cover every possible configuration combination. It’s implied that if the configuration is not mentioned below, the choices are to leave it ‘as-is’ or adjust it according to your business requirements.  

Let’s start by creating a new web application profile in Qualys WAS: 

Setting Asset Details  

Once a new profile is started, the first configuration is Asset Details, which includes the Target Definition. Here are some points to remember while choosing a link to add to the Target Definition: 

• Qualys WAS will use this as one of the first requests when crawling is initiated 
• This link can be any link on the application that is intended to be scanned, such as a valid directory or a valid filename 
• The link should ideally respond with no errors, since Qualys WAS uses what is in the response to continue crawling 
• Make sure requesting this link does not cause a loss of authentication  

Taking the above points into consideration for bWAPP, there are a couple paths to get to all the vulnerable links, or ‘tree’, of the application. These are two URL options: aim.php (see Figure 1) and portal.php. It is recommended to use aim.php in the Target Definition (Fig. 2), as it will cover all points above and help bypass any optimization that Qualys WAS is performing. More later about why we want to bypass the optimization… 

bWAPP has been developed so it will generate links and actions that are considered redundant in Qualys WAS. This will lead to the exclusion of links that are required to be scanned in order to report specific vulnerabilities.  

Referencing Figure 3, this is where optimizations can be applied. The drop-down menu will generate the same form (same URL) multiple times with the same body; only the bug value will change. Qualys WAS will consider these the same action and only choose a few options. The Qualys team has encountered very few applications structured similar to bWAPP; this is based off reviewing multiple applications including real world customer apps. This link does not follow the same logic as http://x.x.x.x/bWAPP/aim.php and should allow the scanner to extract all the unique vulnerable links.

Note: With aim.php in the source, there are unique links in the href tag, so Qualys WAS will parse all of these and navigate to the vulnerable links. This is one of the techniques that the scanner uses to crawl.  

POST http://X.x.x.x/portal.php  Body bug=14&form=submit 

POST http://X.x.x.x/portal.php  Body bug=13&form=submit 

POST http://X.x.x.x/portal.php  Body bug=17&form=submit 

Now let’s move to the second step of the configuration.  

Setting Application Details 

When setting application details, there are a couple of different optional configurations. We’ll focus on Explicit URL to Crawl. See example in Figure 4. Some things to note when adding explicit links: 

• Qualys WAS will request the link as provided, so the link should ideally respond to a GET method 
• URLs added will be prioritized after the Target definition during the crawl and test 

• Adding an explicit URL will make sure those links are prioritized and are tested first, after authentication has been successful 

Qualys WAS will continue to extract links from the source as the crawl continues, through both actions and AJAX links (if SmartScan is on). The scanner will only be able to reach links where there is a crawl path from the target definition or explicit URLs.  

In the case of bWAPP, it helps to prioritize the SQL links to make sure these links are tested first. During a scan, there are thousands of requests with payloads being sent, so there is a possibility that vulnerable links may go later in the scan. Prioritizing these links prevents inconsistent responses and leads to more consistent reporting. Qualys WAS is dependent on responses from the application to report most vulnerabilities. Applications can sometimes respond differently during a scan as opposed to testing manually. 

Qualys WAS uses a phased approach to test, which means it will test all links during a ‘phase’, which is defined as a specific grouping of QIDs. Once it tests all links for that phase, it will move to the next phase (another set of QIDs) and test all links. This means the prioritized links will be scanned first in each phase. In the WAS report, QID 150021 will summarize the details of phases run during the scan.  

In real-life applications that have multiple modules, adding these respective links should ensure that important links are crawled first. Adding links to explicit URLs will bypass any redundancy configuration or blacklist set up in the profile.  

Setting the Option Profile 

The next step is to set the Option Profile (Fig. 5). For this scan, let’s focus on all the SQL and XSS findings that can be detected by Qualys WAS on bWAPP. Most important is to check Include form action URI in form uniqueness calculation. The bWAPP application does have some forms that may submit the same set of parameters to different URLs. Selecting this option allows this optimization to be turned off.  

Qualys WAS will only crawl links that are discovered during the crawl phase, either through source or actions performed. The scanner does perform Enhanced Crawling, which helps with more paths to be crawled. If WAS finds the link as shown below, then it will only crawl this link but not try and navigate to the individual paths.  

http://x.x.x.x/path1/path2/path3/file1.html  

When enhanced crawling is selected, Qualys WAS will also try and find valid responses at the following links below, even if they are not found during regular crawling.

http://x.x.x.x/path1/path2/path3 

http://x.x.x.x/path1/path2 

http://x.x.x.x/path1 

For bWAPP, this should be turned off to try and prevent any errors and optimize scan time. The Qualys WAS analysis team has seen scenarios where this technique of enhanced crawling can lead to errors in applications. It can even cause a web application firewall or blocking devices to drop Qualys WAS requests. It is always highly recommended that Qualys IP ranges are whitelisted, so valid responses can be generated.  

Select the  medium scan intensity to also help prevent errors or network delays. As long as the application is responding accordingly and has the properly allocated resources, Qualys WAS will receive appropriate responses to detect vulnerabilities. A majority of QIDs are dependent on the response and/or response time from the application in order to detect vulnerabilities. Scan intensity can be lowered or increased to help control scan time. If the server can respond at high speeds, then a high scan intensity can be useful to get faster results.  

Since the focus is on XSS and SQL-type vulnerabilities, the categories option is selected (Fig. 6). This same configuration will help if there is an application that errors out after a couple of thousand requests from the scanner. 

Configuring Authentication 

Now let’s cover the last required configuration. Qualys WAS supports multiple techniques of authentication. For bWAPP, both Standard Auth and Selenium have been tested multiple times and proved to be successful. However, due to the configuration being used and during the review of 150115 Authentication Form found, there are multiple forms being identified, so Standard Auth is not a logical option since it is dependent on 150115. In this case, the best option is Selenium (Fig. 7). 

Launching a Scan 

At this point, the remaining options can be left ‘as-is’ and a discovery scan or vulnerability scan can be launched (Fig. 8).   

Reviewing Qualys WAS Results Report 

Once the results report is generated, it should show multiple XSS and SQL-related vulnerabilities. Here’s an example of results for a bWAPP scan (Fig. 9):  

When reviewing some of the 150003 SQL Injection and 150012 Blind SQL Injection findings, remember that there are three different techniques that are being used to determine the SQL vulnerability:  

1. Error  

2. Delayed  

3. Response Differences 

Figure 10 shows an example of the error-based technique. In the “Response” section, it shows a snippet of the response that led to this detection. It was detected with a payload that is not usually used for SQL testing. Note the payload being used at the bottom of the Request section, highlighted in red.  

Figure 11 shows an example of the delayed technique. Qualys WAS will send a couple of rounds of delayed and non-delayed payloads to make sure the delay is not due to network latency. This prevents false positives from being reported.  

Finally, the last method is based on differences in response (Fig. 12). Here, there are two payloads sent. One is a true case, and the second is a false case. This also runs through multiple rounds to prevent any false positives.  

The examples above are examples of high quality bWAPP scan reports with valid findings. As the configurations are changed or adjusted, there may be more vulnerabilities found and even better coverage.   

Now let’s list some of the areas to review in a Qualys WAS report, if the same results are not reported for your bWAPP installation.  

Reviewing Report Diagnostics  

If the detections above were not reported with the configuration provided, it’s best to run a full discovery scan. Qualys WAS has a variety of QIDs to help understand what is going on during the scan. Listed below are some of the QIDs that give an overview of what happened during the scan:  

• 150018 Connection Error Occurred During Web Application Scan - This will provide details of any potential links running into timeout issues. If a large number of links are reported here, review the details in this QID. It will provide which phase may be causing an issue. 
• 150009 Links Crawled and 150152 Forms Crawled - All links and forms found will be reported. If a link is not reported in these QIDs, then it was not tested. If links are added as explicit links, these will still be reported in these QIDs regardless of the response of the link. Meaning, if the link is responding with an error, it will still be tested but does not guarantee the expected response is received or the expected vulnerability will be reported.  

• 150094 Selenium Web Application Authentication Was Successful - Make sure the authentication was successful, since this is required for some of the vulnerabilities to be detected. 
• 150202 Missing header: X-Content-Type-Options - Multiple QIDs in the Security Weaknesses section provide status codes for the links crawled. Under the following QID for example, if it’s reported that expected SQL links were responding with a 404 (see below), then that hints that the links were no longer responding or the configurations for bWAPP need to be reviewed.   

GET http://X.x.x.x/bwappp/sqli_1.php response code: 404 
GET http://X.x.x.x/bwappp/sqli_2.php response code: 404 
GET http://X.x.x.x/bwappp/sqli_3.php response code: 404 

There are multiple ways that these open source applications can be deployed. The report itself returns an abundance of information on what is going on with the application during the scan. Keep a look out for upcoming releases for Qualys WAS, as new diagnostics QIDs are added based on need and demand from customers.  

Disclaimers 

Most vulnerable open source applications are designed for manual testing practices. Sometimes when a vulnerability is easy to find manually, it may be more challenging to exploit in an automated manner and vice versa. Not all vulnerabilities in bWAPP will be detected, but anything that can be automated by WAS should be reported.