Curious about how to stay up to speed with Penetration Testing? We’ve gathered 10 useful…
The Difference Between a Pentest and a Vulnerability Assessment
Companies and people are often misinformed or misguided as to what the differences are between a penetration test and a vulnerability assessment. In many cases, there are upper-level executives which ask for a penetration test but really want a vulnerability assessment, and vice-versa. In these scenarios, the assessment is sometimes improperly labeled which can be very misleading.
If someone asks for a pentest but really wants a vulnerability assessment, the scope of the engagement is limited to scanning and enumeration without exploitation – which in my opinion is an unfortunate and misguided situation for testers.
Related post: Vulnerability Scan vs Vulnerability Assessment
The idea here is to finally set apart the main differences between the two so that you and your organization will know what makes most sense for your needs and requirements.
What is a Penetration Test?
Related post: What to Know Before Conducting a Pentest
A penetration test, or pentest, is the manual process where an ethical hacker conducts an assessment on a target to uncover vulnerabilities by exploiting them. The goal is to gain unauthorized access through exploitation which can be used to emulate the intent of a malicious hacker.
A pentest is often broken down into the following phases:
- Scanning and enumeration
- Exploitation (gaining access)
- Post-exploitation (maintaining access)
- Covering tracks
What is a Vulnerability Assessment?
Related post: Benefits of a Vulnerability Assessment
A vulnerability assessment, or VA, is the process of identifying threats and vulnerabilities on a target by using automated vulnerability scanners. This sometimes includes a range of manual testing with additional tools to further evaluate the security of applications or networks and to verify vulnerabilities discovered by the scanning applications.
Timing and Objectives of a Pentest and VA
A pentest is often initiated by various scenarios which can include (but are not limited to) application launches, major network/application changes or updates, compliance regulations or a breach/leak from a targeted attack.
Due to the varied reasons for conducting a pentest, the objectives can often differ greatly as well. More information on each scenario can be found in the article “Pentesting 101 – What to Know Before Conducting a Pentest“.
Generally, the end goal of an ethical hacker is to gain unauthorized access to a target by means of exploiting uncovered vulnerabilities from the scanning and enumeration phase. Your organization, however, may have an alternative end goal in mind due to the requirements for conducting the pentest in the first place.
Some of the objectives and occasions for conducting a pentest are as follows:
- Application launches: A pentest may be conducted as part of the software development life cycle (SDLC) process to uncover existing vulnerabilities which should be resolved before the launch. The main objective is to help save time and money by discovering and fixing vulnerabilities before an application is deployed into production and open to end users or potentially malicious hackers.
- Major network/application change or update: Pentests are often scheduled on an annual, bi-annual or quarterly basis to maintain best security practices and stay on top of any major changes which could potentially uncover new vulnerabilities. A pentest may be initialized on this cycle or when a major change in a network or application occurs.
- Vulnerability management program: The landscape of attacks is evolving at a rapid pace which usually eclipses the awareness and knowledge that organizations maintain with regards to their security posture. In order to attempt to stay on top of this, it is imperative to continuously assess the applications and infrastructure on a regular or at least semi-regular basis. This can fall into the practice of a vulnerability management program which will execute pentests on various cycles.
There is a famous quote in the infosec community by John Chambers (former CEO of Cisco) which explains the need for this continuous maintenance: “There are only two types of companies: Those that have been hacked, and those that don’t know they have been hacked.”
- Compliance regulations: A pentest can be conducted with the objective of meeting certain compliance standards which have requirements to perform penetration tests at certain periods. Depending on the type of data organizations process or store, they may be required to abide by different compliance regulations (e.g. PCI DSS, HIPAA, Sarbanes-Oxley). Some of these regulations require a pentest to address the risks and potential security exposure an organization may have to aid in the protection of this regulated data.
- After a breach or leak: This is quite possibly the worst reason to conduct a pentest but is also very common unfortunately. After having already been breached and having confidential data being exposed to the public, an organization may panic and immediately hire a vendor to conduct a pentest to prevent a similar leak from happening again in the future. The objective here is to uncover any additionally existing vulnerabilities and holes an organization may have since they are already well aware that flaws exist in the first place. This is a reactive approach used to prevent similar breaches in the future.
The timing and objectives of a vulnerability assessment can be somewhat different compared to that of a penetration test.
Where a pentest can sometimes be more responsive or mandatory for various reasons, a vulnerability assessment can be more cyclical to be proactive at discovering vulnerabilities and to perform patching as part of an ongoing vulnerability management program or when new vulnerabilities are released. Pentests are also included as part of a vulnerability management program, however, these will be much less frequent than vulnerability assessments within the actual program.
In this case, VAs should be a frequent and ongoing process to continuously monitor and identify weaknesses in an organization and reduce the attack surface.
There are also many cases where a VA is performed after a leak occurs, when a new prominent vulnerability comes to light, or if a change in a network or application takes place.
The timing and objectives of these are as follows:
- New vulnerability released: When a new headline vulnerability hits the market, many companies and executives panic immediately. Calls are made to their VA teams or vendors to conduct an assessment ASAP to hunt for this new vulnerability in their organization. Recent examples of this include EternalBlue, KRACK, Meltdown/Spectre and Heartbleed.
The assessments take place shortly after word gets out that another one of these vulnerabilities have come to light, and the objective here is to determine whether there is any presence of such a vulnerability in an organization.
- Network/application change: This falls somewhat into a vulnerability management program but will remain separate since not all organizations have or maintain such a program. Anytime a major change, update or migration takes place (have you moved buildings recently?), this should be an immediate trigger to re-scan and assess the environment to find weaknesses which may have been created due to these changes. Maybe something was missed while setting up the network and an extra access point or server was left behind open to the external network. It’s things like this which happen frequently and are often forgotten.
- Vulnerability management program: Application security and patch management are a continuous process within a good vulnerability management program. This will include vulnerability scanning of applications and networks to identify weaknesses and patches that should be applied. The entire program should be cyclical which will require vulnerability assessments on a monthly, quarterly or annually basis, depending on the targets, to stay on top of new vulnerabilities and exposures in an organization.
Related post: Vulnerability Management is the Key to Stopping Attacks
- After a breach or leak: This fits in the same category as execution of a pentest after a breach or leak. A vulnerability assessment should be initiated to uncover potential flaws which still exist within the organization to prevent another attack from occurring. If you’ve made the headlines due to the attack, maybe your internal sensitive information has been passed around in some dark hacker forums. This information, or just the fact that there are/were open vulnerabilities, can be enough to trigger additional attacks by hackers. Staying on top of all attack surfaces is paramount in keeping this exposure to a minimum and minimizing the chances occurrence in the near future.
Reporting and Deliverables of a Pentest and VA
Penetration Test Reports
Once all is said and done and your ethical hackers returns the final report of their benign break-in attempts, you will receive a penetration test report. Hopefully you don’t have to read it while holding your breath, desperately hoping that nothing too critical or impossible to fix was uncovered. Nevertheless, you need to move forward!
Each pentest report will typically follow a similar outline in the sense that it will have a beginning, middle and an end.
Here’s the breakdown of what that usually looks like in order:
(Note that not all vendors are going to follow this verbatim – they may add or remove certain sections, or follow a different structure.)
- Executive summary: Almost all reports open with this. It is a summary of the report contents in a non-technical format so that everyone can understand. On a high level, the process, findings and recommendations will be covered to give a general review of the assessment outcome.
- Summary findings and recommendations: This may or may not be included in a report. Executives and slightly more technical people will find it useful to have a better understanding of some of the specific findings in the report written in a paragraph format. This can be broken down into two sub-sections – an overview of the findings and a review of the recommendations. The findings may include some of the higher risk findings (if any) as well as some worthy mentions of positive security controls discovered. The recommendations generally include the higher risk fixes or anything that was consistently seen which needs to be addressed (e.g. too many outdated SSL/TLS configurations!).
- Detailed findings in order (i.e. by risk rating): This is the bulk of the report. Each finding will be written with a summary, technical details, reproduction steps (including images if any), and recommendations which can also include links for reference.
- Methodology: Some vendors will include a methodology section somewhere in the report to outline the process taken by the ethical hackers and their testing approach. This is generally a template created by the vendor to include in all reports. It can help your organization have a better understanding of how technical and deep the testing methodology was.
- References: This section can include a list of used tools and applications for the assessment and any other relevant material.
- Appendices: The last section is reserved for any material which required additional information relevant to the report but isn’t essential to the main body of the report.
Vulnerability Assessment Reports
Since the process of a VA can drastically change from company to company, or just from assessment to assessment depending on the requirements and scope, we will just look at what a general report looks like when executing a VA on a network, web-based application or source-code.
The most important thing you can expect is a detailed report (usually in PDF or Excel format) of findings found on the entire scope of the assessment. This is usually a direct or slightly modified output from a scanner application which includes full details of the hosts, vulnerabilities, CVE numbers, risk ratings, impact, recommendations and more. Depending on the requirements or reasons for the assessment, this report and process can change greatly.
For example, you may just require an OWASP Top 10 scan on your web application, or you may need a full internal assessment of your network to meet compliance requirements for an auditor. Regardless, this report will meet those needs with full details of any findings discovered.
In addition to the report and depending on the scope of the engagement, a report may come with a verification of findings which includes any false-positives. This is an extra step in some assessments which occurs after the actual scanning to manually check each finding one by one to verify that they are a true positive or not. Some organizations find this very beneficial since they can be certain with the findings provided to them and won’t have to waste their time trying to patch something which doesn’t exist. However, the main catch is that this additional step requires extra hours and money to complete, which is why not all companies want this included as part of the engagement.
Who Usually Performs a Pentest/VA?
Hackers! White hat ones, of course (we hope). A pentest team can have an extremely varied background as far as education and experience goes. But no matter what, they all have one important thing in common – a passion for security and great curiosity to find and break things. It’s this passion that unites all pentesters and makes them great at their jobs.
Can Part of the Job Be Automated?
The short answer is: yes and no. It depends on the type of assessment and the granularity of the technical testing.
The main phase of a vulnerability assessment is completed by some sort of automated scanning application which will perform vulnerability checks on a network, application or code by using baseline signatures from a database to discover and report findings. The entire execution of this phase is automated – meaning that while it’s running, the hacker has time to fill up on some much-needed coffee. The setup of this step and the subsequent reporting are all manual efforts. Some people are very familiar with scanning applications and can very easily setup and sift through reports after years of experience with a certain tool.
On the other hand, a penetration test requires much more manual work that’s not easily automated. The same vulnerability scans may take place, however the next step is to manually poke at a network or application to probe for further vulnerabilities and to attempt exploits. This is often the process from a network pentest since it’s impossible for a small team of testers to manually check every single host, port and service running to identify vulnerabilities. The manual part after can be very time consuming depending on the size of the network and report and takes a lot of knowledge and precise testing to be done properly. There are several automated testing frameworks built around pentesting which are very slowly improving, but none have proven to be successful compared to the skill of a highly-trained and curious ethical hacker.
The bulk of an application pentest will be a manual effort by testers while using a proxy tool to review requests and responses made to the application which will then be used to probe inputs to uncover any issues or vulnerabilities. An application pentest is almost a fully manual effort as long as your vendor is not just directing their dynamic analysis scanning tool at your URL and firing it off (hint: they should be fired if this is their “pentest”). Manually testing a large application can take some time and resources and requires a lot of previous knowledge of web app architecture and testing frameworks.
Before starting the process of hiring a vendor for a security assessment, it is crucial to know the differences between a pentest and a vulnerability assessment to be sure that your organization is getting what it wants and needs based your business requirements. A pentest can have a drastically higher price tag than a VA, but if you only require a small VA, then it isn’t worth it to pay out all that cash for a pentest.
Once you have a solid understanding of the main differences, you can make the right decision for your organization and better determine the scope of the engagement. In the end, you will be left with the type of test and report you were looking for and the required patching that lies ahead.