Web Application Penetration Testing: Minimum Checklist Based on the OWASP Testing Guide

Without a doubt, web applications have to be thoroughly protected from hackers. Unfortunately, the security of most web applications is still questionable. The total number of vulnerabilities discovered in 2018 was 23% higher compared to 2017, according to the 2019 Imperva Report. So it isn’t overreacting to say that ensuring web application safety needs to be a high priority for developers and testers in 2019.

The major goal of penetration testing or pen testing is to find and fix security vulnerabilities, thus protecting the software from hacking. To do so, a QA specialist has to conduct simulated cyberattacks on the web application.

Pen testing helps QA specialists to:

• identify previously unknown vulnerabilities
• check the overall efficiency of security measures
• test the configuration of components that are exposed publicly
• detect security loopholes that can potentially lead to the compromise of sensitive data

This article will be useful for QA specialists who are carrying out penetration testing for web applications. Even those who have a decent tech background may find it hard to fully understand which application’s parameters have to be checked and which don’t.

In this article, we’ll tell you about the most significant elements to include in your penetration testing checklist. We took the OWASP Testing Guide as a basis, highlighted the most critical steps, and added comments about the tests you can skip under certain circumstances.

Quick overview of the OWASP Testing Guide

The OWASP Testing Guide v4 leads you through the entire penetration testing process. Each test contains detailed examples to help you comprehend the information better and faster. This guide is suitable for different web applications and is a perfect choice for deep assessment.

The OWASP Testing Guide is the most detailed and extensive, and it’s considered one of the best options to help you conduct thorough penetration testing.

All of the recommendations in this post are based on optimizing the stages mentioned in version 4 of the OWASP Testing Guide. We also include a couple of tests from version 3. You can also check out our OWASP mobile security guide to audit protection of your apps.

How to pick the right set of OWASP tests

Since we’ve decided to use the OWASP Testing Guide as a baseline for testing a web application, we have around 200 tests to choose from. So it’s quite complicated to define which tests should be performed and which can be skipped.

In order to choose the right tests for your product, you need to do the following:

1. Define the scope of testing.
2. Define business-critical features.
3. Select the necessary tests.
4. Prioritize the tests.

Let’s go through the OWASP tests to determine which you should include in your minimal SDLC audit checklist and why.

Gathering information

The first thing you should do before looking for web app vulnerabilities is to collect information about your application under test.

Unfortunately, QA specialists often don’t have access to the code, so they can’t be sure they’ll map all possible paths through it.

The solution is to create a site map by:

• using automatic tools like crawlers that follow all links on the website
• opening pages manually
• using brute force to find a way to directories not linked to on the site

In this way, you can define the important aspects of the application as well as its potentially weak spots. All these will help you prioritize tests and plan your work accordingly.

Start with highly problematic areas of the application. Detect areas where users are allowed to add, delete, or modify content, and make sure they’re secured with encoding and user verification.

During this stage, make sure to check such critical things as:

• the number of entry points, to estimate the time needed for testing
• metafiles on the web server (a robots.txt file may contain the web application’s directory or folder paths)
• other web applications hosted on the web server, as some of them may have known vulnerabilities and attack strategies that can be exploited for gaining remote control or exploiting sensitive data
• webpage comments and metadata left by developers, as they may include information that should be hidden from third parties

Although conducting search engine discovery and reconnaissance for information leakage is an important test for your web application, you can skip it if the application is under development and isn’t accessible via the internet.

Blind penetration testing isn’t an option for mapping the application architecture. Make sure to request the required information from developers in writing or through interviews, and use DNS research tools such as DNSdumpster and SecurityTrails.

Configuration and deployment management

A complex web server can contain hundreds of web applications, and sometimes one vulnerability can undermine the security of the entire infrastructure. This is why the deployed configuration of the server that hosts your web application plays a significant role in ensuring your application’s security.

There are a wide range of application platforms, but you should know the configuration errors of some key platforms and take them into account when conducting penetration testing. Platform vulnerabilities can compromise the application in the same way an unsecured application can compromise the server.

Penetration testing services help to ensure that misconfigurations or mistakes made during deployment don’t allow attackers to compromise infrastructure or applications.

Let’s explore some examples:

1. By conducting proper testing of the configuration of the network and the application platform, you can detect existing vulnerabilities and prevent potential attackers from disclosing application source code and internal information.
2. Testing file extensions not only helps you validate files to be uploaded but also provides a better explanation of the underlying technologies so you can come up with relevant attack scenarios. Take ImageMagick, for example: one of its vulnerabilities can lead to remote code execution if you process user-submitted images.
3. Testing for backup and unreferenced files is necessary, as it helps you find unreferenced files, forgotten files, or automatically generated files that may store sensitive information, such as infrastructure and credential data.

When testing HTTP methods, use nmap script: nmap --script http-methods <target>, to see the list of HTTP methods used.

The content discovery function of Burp Suite can help you enumerate infrastructure and application admin interfaces. Use it to arrange brute force attacks for files and directories.

You can skip the HTTP strict transport security test if you’re using a non-production environment, while HSTS is usually disabled for applications deployed to the test environment.

You can also skip testing the RIA cross-domain policy if your web application doesn’t use it.

Identity management

Identity management covers account creation and resumption processes. The main goal of identity management pen testing is to make sure that the web application allows users to access only specific information that’s aligned with their roles.

Major issues you should be worried about here are weak password policies and a weak user registration process.

When testing for account enumeration and guessable user accounts, focus on login forms, recovery password forms, and fuzzed user IDs in case there’s a possibility to find a particular user by their ID. On Facebook, each user can create a unique link for their profile, for instance https://www.facebook.com/zuck.

You should also test role definitions and account registration processes. Try to circumvent the application to find out whether identity requirements for user registration align with business and security requirements.

You can skip the account suspension/resumption process test if the application has no mechanism for temporarily suspending accounts (for instance, for a week). If this option does exist, check whether a user has any chance to log in using the deactivated account. Also, test the account’s performance after resumption to make sure it works properly.

There’s no need to test for weak username policies if all usernames are provided by users. And naturally, there’s no need to test for permissions of guest accounts if there are no such accounts in the application.

Authentication

Authentication requires proper security testing to ensure that malicious attackers have no chance to gain access to the application. For instance, if you don’t test for default or auto-generated credentials, you may miss a vulnerability due to passwords and logins (for example, logins of the format firstname.lastname) that can be guessed by tools used by attackers and lead to unauthorized access.

Another critical test is concerned with bypassing authentication schemas, which can be tricked into thinking that a user is already authenticated.

A comprehensive understanding of the way the authentication process works is extremely important for performing tests that try to circumvent it.

For instance, a tester should know all aspects of lockout mechanisms to make sure they function correctly. Lockout mechanisms have to balance guarding accounts from hackers and protecting users from being denied authorized access. If the application has no protection mechanisms from brute force attacks, you can skip checking lockout mechanism performance but should mark this test as failed.

Other important tests that have to be performed when checking the authentication mechanism include:

• checking that the browser is correctly instructed by the application not to remember sensitive data
• testing the remember password functionality and checking that passwords and hashes aren’t stored in cookies
• checking whether the answers to security questions are easily guessable, brute forcible, or discoverable
• making sure that the password change mechanism is secure against guessing and bypassing
• testing the CAPTCHA for its resistance to brute force attacks
• verifying that user data and credentials are transferred via an encrypted channel

When testing a web application for weak password change or reset functionalities, check that:

• a logged-in user can’t change the password without typing in the existing password
• a user doesn’t receive a new password by email in plain text format after resetting the password
• tokens for resetting passwords are unique and can’t be guessed
• the previous password doesn’t work after a token for resetting the password is sent to the user

A test for resilience to password guessing is used to check how the web application is secured from brute force attacks. There are two major ways to ensure security: using a CAPTCHA and locking the account after a certain number of invalid passwords.

Authorization

Authorization usually comes right after successful authentication, and the tester will verify this after he holds valid credentials associated with a well-defined set of roles and privileges.

The main goal here is to get a comprehensive understanding of how the authorization mechanism works and to see whether it’s possible to trick it.

Pay attention to path traversal vulnerabilities with well-known dot-dot-slash attacks. Testing for path traversal vulnerabilities can take a lot of time and produce a lot of requests to the target server. It’s better to start testing for path traversal vulnerabilities in the evening and check the results in the morning.

Test the bypass authorization schema, by calling an internal page and skipping the login page or making the application think the user is already authorized. Don’t forget to check whether it’s possible to access administrative functions while being logged in as a user with standard privileges.

Try to find any ways to change the roles or privileges assigned to a user in order to achieve privilege escalation.

Find a way to leave insecure direct object references. When conducting this test, the first thing to do is to map out all locations within the application that allow user inputs to be used to directly reference objects.

Consider using the Autorize plugin for the last three tests to save time.

Session management

Session management is one of the core components of any web application, as it covers everything from the moment users authenticate until they log out.

Making sure that a session management schema can’t be bypassed is the first step of proper security testing for session management. Your goal is to trick an application to give you access to a user account without providing the correct credentials. This can be possible because of the various mechanisms the application uses to store and validate credentials for a better user experience.To define major application security flaws and prevent session hijacking, you also have to include the following OWASP tests into your web application testing checklist:

• Checking cookie attributes, which are often the first attack vector
• Searching for a session fixation vulnerability that can help attackers get access to a user’s account through an active session
• Identifying exposed session variables that can allow an attacker to impersonate an authorized user
• Testing for cross-site request forgery to see whether it’s possible to compromise end user data and operations
• Checking the logout functionality to define whether the lifetime of session tokens is short enough to prevent a session hijacking attack
• Testing session timeout to ensure that the application logs out automatically, leaving no chance for attackers to use the same session

When conducting security testing for session management, understanding the ground rules of application performance is key to choosing the right tests and interpreting their results correctly.

Browsers, websites, and web applications use HTTP for communication. Since HTTP is a stateless protocol, each command runs independently without knowing previous commands. This means that web servers respond to client requests without linking them to each other.

Meanwhile, even simple application logic requires multiple requests to be associated across a session. To ensure this happens, it’s necessary to implement third-party solutions, such as off-the-shelf middleware and web server solutions or bespoke development implementations.

Commonly used web application environments like ASP and PHP provide developers with built-in session handling routines. These routines allow for issuing identification tokens to identify a user that has logged in. These tokens are assigned to a specific user for the duration of a session and are referred to as a Session ID or cookie.

Applications can interact with users in different ways depending on the nature of the site, and the security, and the availability requirements of the application.

Using accepted best practices for application development (for example, the OWASP Developer Guide 2014) is essential for engineering secure software. However, it’s also crucial for application security to be considered within the context of the client’s requirements and expectations.

Data validation

Failure to properly validate input data that comes from the client or from the environment before using this data is a common weakness of web applications. It’s associated with many major vulnerabilities including SQL injections, file system attacks, and interpreter injections.

Using these vulnerabilities, cybercriminals can tamper with data from an external entity or client. This is why data from clients and form the environment should never be trusted.

It’s easy to say but hard to do. The point is that complex applications tend to have numerous entry points, which makes it hard for developers to follow the rule.

When testing the design and architecture, pay attention to the location of data validation within the application. If it’s performed at multiple places, advise your team to consider a central validation framework. This way, you can fix input validation in just one place instead of hundreds of places. Not to mention it’s far less expensive.

When checking input data validation, the tester’s main objective is to test all possible forms of input to verify that the application sufficiently validates input data before handling it.

A good way to optimize the process of testing for reflected and stored cross-site scripting is to use XSS polyglots, special payloads that can pass lots of filters simultaneously. You can find examples of XSS polyglots on GitHub.

Another critical test is checking for format string vulnerabilities. These manifest mainly in web servers, application servers, and web applications running C/C++ code or CGI scripts written in C.

Error handling

Error handling is crucial for proper web application performance and security. It can allow information disclosure thanks to informative error messages and stack traces.

Improper error handling may provide cyber criminals with enough information to launch an attack. Flaws can provide attackers with clues on how the application operates, how to exploit it, how to expose sensitive data, and more.

Error messages can unveil the inner structure of a web application, so you have to analyze their content. Your two must-have security tests for error handling are analysis of error codes and analysis of stack traces. Pay attention: it’s common for a web server to be enabled in debug mode in a development or test environment.

Cryptography

Cryptography is vital for web applications. The absence of encryption when transferring data between a client and a server makes it possible for hackers to arrange man-in-the-middle (MITM) attacks.

The OWASP Testing Guide v4 highlights three major issues for security testing that definitely should be added to the every checklist for web application penetration testing:

• Testing for weak SSL/TLS ciphers and insufficient transport layer protection
  Consider using the SSL Labs tool, which performs deep analysis of the configuration of any SSL web server on the internet.
• Testing for padding oracle
  You may want to request access to the code in order to perform this test accurately.
• Testing for sensitive information sent via unencrypted channels
  Make sure to test for common security risks: check if information transmitted by the application in clear text is transmitted over HTTP instead of HTTPS and whether weak ciphers are used.

Just implementing data encryption into a data transmission channel isn’t enough. The development team should pay attention when choosing a cryptographic protocol, as some protocols are outdated and have critical vulnerabilities. The most well-known examples of such vulnerabilities are POODLE and Heartbleed.

Business logic

Even though business logic attacks aren’t new, they’re often underestimated, so it’s best to add business logic to your penetration testing checklist.

Business logic vulnerabilities are unique to each application, potentially damaging, and can only be tested manually. You have to start thinking out of the box in order to find business logic weaknesses in multi-functional web applications.

Let’s take an online shop, for example. The usual flow for a user is to add items to the cart, fill out a form, submit an order, make a payment, and wait till the goods arrive. An experienced tester should understand where to look for vulnerabilities in such an application. What if some of the steps can be bypassed and a user can receive the goods without paying for them? Or what if the customer’s personal and financial details can be exploited by attackers?

For testing the business logic of a web application, security professionals have to change the way they think, come up with ways to abuse and misuse the system, and borrow many testing techniques and practices from functional testers that focus on logical or finite state testing.

Business logic flaws cannot be discovered via scanning tools, as no vulnerability scanner can replicate the skills of QA specialists and their knowledge of the complete business process, its rules, and the special characteristics of the particular web application.

Make sure to add all of the tests mentioned in the Business Logic Testing section of the OWASP Testing Guide v4 to your checklist. Feel free to skip testing for unexpected file types and malicious files uploads if your application provides no place for a user to upload data.

When performing the other tests, open all the requests that are submitted to the server and detect the variables that are used. Our goal is to define whether it’s possible to change some of the variables by triggering incorrect processing.

Client-side testing

This testing aims to check for correct code execution on the client side, which is distinct from the server side, as the client side returns the subsequent content. Client-side testing is typically performed natively within a web browser or browser plugin.

Client-side attacks can quickly compromise your critical assets and sensitive data. This is why it’s essential to test the network’s ability to recognize these attacks and respond accordingly.

JavaScript, HTML, CSS, ActiveX, Adobe, media players, and Java plugins are the most common entry points for client-side attacks. Don’t forget to add testing for such injections into your pen testing plan.

You can skip testing for CSS injection if there are no features to provide your own CSS styles in the web application. If the site doesn’t use Adobe Flash or WebSockets, you don’t have to test for those vulnerabilities either.

To save time, consider using automated testing tools to cover the following tests:

• DOM-based cross-site scripting
• JavaScript execution
• HTML injection
• Client-side URL redirection

When you’re trying to detect any clickjacking vulnerabilities, use Burp, a tool that will quickly craft an attack and save you time on manual testing. Other helpful tools are Firebug and OWASP ZAP. You can use them when testing local storage.

The test-cors.org website is also helpful. Use it to test CORS requests when testing for cross-origin resource sharing.

Additional guides for web application pen testing

The OWASP Testing Guide isn’t the only well-known industry guide for web application penetration testing.

SANS SWAT Checklist

Created by the SANS Institute, the Securing Web Application Technologies (SWAT) Checklist appeals to developers and QA engineers to raise their awareness of web application security. The list combines best practices of web application pen testing and brief descriptions.

Concise and easy to understand, this checklist helps you identify and neutralize vulnerabilities in web applications. Some of the test descriptions include links to informational pages and real-life examples of security breaches.

However, the SWAT Checklist has its weak point: it mostly features best practices instead of detailed steps on how to perform penetration tests. The lack of details and examples makes it not practical enough for flawless web application pen testing.

MITRE ATT&CK and Cyber Kill Chain

The MITRE ATT&CK and Cyber Kill Chain frameworks provide a comprehensive approach to better detect and mitigate adversarial behavior, and describe the main steps from initial access to command execution.

Being a globally accessible knowledge base of adversarial tactics and techniques based on real-world scenarios, MITRE ATT&CK helps QA specialists and developers better understand the actions of hackers and find proper ways to detect and prevent threats.

The Cyber Kill Chain framework was designed to identify and prevent cyber intrusions. It identifies what adversaries must do in order to achieve their objectives.

However, the information you can learn from both frameworks provides you with no detailed steps on how to perform tests. And lots of recommendations are not valid for web applications.

All the same, it’s important to understand the actions of hackers, as it helps you find the best-case scenario to detect and stop them. Read the article below to learn more.

Conclusion

In this article, we’ve discussed essential components of proper penetration testing for web applications and explored ways to choose a web application security methodology.

Using the OWASP Testing Guide as a basis, we’ve provided tips for each stage of web application testing and pointed out the most important tests to include in a minimum checklist tailored to your application and the current stage of the software development life cycle.

Apriorit has a team of QA specialists capable of finding hidden security vulnerabilities and mitigating them at the earliest stages of development. Don’t hesitate to contact us with any questions about penetration testing, security issues, and challenging cyber security projects.