On average, only 68.8% of organizations across the globe maintained compliance with PCI DSS Requirement 11.3, which states that organizations must perform penetration testing annually. 

Penetration testing is a simulated cyber attack engagement used to identify and exploit vulnerabilities that could give cyber criminals unauthorized access to information.

The exercise, also referred to as a pen test, could reveal whether your cardholder data environment is susceptible to a wide variety of vulnerabilities that could leave your cardholder data vulnerable. It could also reveal what improvements are necessary to keep this information safe.

In this article, we explain what exactly PCI penetration testing is, how it’s different from regular pen testing, and the test components and processes you need to know. We also provide a penetration test report checklist for help evaluating the quality of a report.

What is PCI DSS penetration testing?

A penetration test is an exercise where a hired third party launches a simulated attack on your systems to discover areas your organization can improve its information security.

A PCI penetration test specifically evaluates the security of your cardholder data environment, any networks or systems that connect to your cardholder data environment, and even isolated systems or networks  from your internal infrastructure and applications to external systems connected to public networks. 

PCI penetration testing is a requirement of maintaining PCI DSS compliance, and noncompliance can result in legal penalties or loss of payment card processing privileges.

How is a penetration test different from a vulnerability scan?

Although PCI penetration testing may include vulnerability scanning, they’re not the same thing.

A vulnerability scan focuses on finding, prioritizing, and reporting vulnerabilities that exist in a system by sole use of an automated tool. A PCI penetration test could leverage a vulnerability scan, but include an actual white hat hacker whose main goal is to exploit those vulnerabilities and potentially pivot through your environment to discover deeper threats. 

Refer to the table below to find a breakdown of the major differences between a vulnerability scan and penetration test.

What is the difference between a PCI pen test and a standard pen test?

Compared to a standard pen test, PCI pen tests have more specific guidance regarding the scope and frequency of the test. The testing methodology also specifically requires testing against the application-layer and any critical systems or cardholder data environment connected-to systems.

PCI Penetration Testing Requirements

PCI DSS Requirement 11 contains controls related to the establishment of a vulnerability management process. The controls include performing quarterly internal and external vulnerability scans and an annual penetration test.

PCI DSS Requirement 11.3 addresses penetration testing and specifies requirements for the following.

• Who performs penetration testing: A qualified internal resource or third party.
• Scope: Critical systems and any networks or systems connected to the cardholder data environment.
• Frequency: Should be performed at least once a year or after any significant changes. Service providers need segmentation testing performed semi-annually.
• Methodology: A methodology needs to be defined including scoping, documentation, and rules of engagement. Actual testing methodology should be based on industry standards and PCI defined testing guidance.
• Components: Scoping, segmentation testing, network and application layer testing.
• Reporting and documentation: Penetration test methodology must be documented and penetration test report must include vulnerabilities assigned with an associated score and description.
• Remediation: All high and medium externally facing vulnerabilities must be remediated as well as all high internal vulnerabilities. Any networks that were defined as segmented but were found connected would be brought into PCI DSS scope unless otherwise remediated.

Who typically performs a PCI DSS penetration test? 

Secureframe recommends hiring a third-party penetration testing firm to perform the penetration test. 

PCI DSS does allow an internal resource to perform the penetration test. This individual must be organizationally independent — meaning, they cannot be responsible for the management, support or maintenance of the target systems or environment. This individual must also be qualified, which entails having past experience as a penetration tester or holding a penetration testing certification, such as an Offensive Security Certified Professional(OSCP), Certified Ethical Hacker (CEH), or Global Information Assurance Certification (GIAC).

PCI penetration testing methodology

There are several methodologies that can be used for penetration testing. These refer to activities and processes aside from testing itself that can help make a penetration test successful.

The methodologies used depend on the company offering penetration test services as well as the threats and vulnerabilities of the cardholder data environment and complexity and size of the organization being tested.

There are also different methodologies that can be used before, during, and after a pen test. For example, a pen tester may use social engineering techniques during testing to identify and gain access to servers, network components, and other targets in the CDE. Others may not incorporate these techniques into their penetrating testing methodology.

When considering which activities and processes to incorporate, pen testers may look to several industry-accepted methodologies, including:

• Open Source Security Testing Methodology Manual (“OSSTMM”)
• The National Institute of Standards and Technology (“NIST”) Special Publication 800-115
• OWASP Testing Guide
• Penetration Testing Execution Standard (PTES)
• Penetration Testing Framework

PCI penetration test components

Below find more information on scoping, segmentation testing, network and application layer testing.

Scope

When performing a PCI pen test, critical cardholder data environment (CDE) systems, the external perimeter, and any cardholder data environment connected-to systems are included in the scope of the test. It is up to the organization performing the penetration test to define the scope and to ensure it includes all of the following components:

• Critical systems: Any systems, networks or devices that store, process or transmit cardholder data, or are considered incredibly impactful to your service.
• Connected-to systems: Any networks, systems, or devices that connect to your cardholder data environment.
• External CDE perimeter: Any public-facing services, remote access points, and critical system components that are connected to public network infrastructure (e.g., web applications).

Application-layer and network-layer testing

An application security and network-layer penetration test identifies any security flaws that arise from insecure application design and coding. Pen testers also check for cybersecurity defects that stem from insecure configuration, implementation, usage, or maintenance of organization-specific software.

The application and network-layer testing begins once the scope is defined, rules of engagement are established, and any access is granted.

The penetration testing team will likely define the dates in which the penetration test will take place including the time in which testing will be performed.

Pen testers will then perform the actual assessment against the application and network, discovering vulnerabilities that may exist within your environment.

Segmentation testing

Segmentation testing is required annually for merchants and semi-annually for service providers.

If you are utilizing segmentation controls such as firewalls or VLANs to segment networks out-of-scope from your cardholder data environment, segmentation testing must take place.

A segmentation check tests that these controls are operational and effectively isolating the CDE from other systems. This check is important because it prevents hackers from being able to pivot into the CDE if they gain control of a different isolated network. 

Social engineering

Social engineering such as phishing is the most popular and effective way hackers can breach your environment. Social engineering tests aim to identify risks that may arise from users’ inability to decipher a malicious request from a true request.

PCI DSS doesn’t require a social engineering test but does require security awareness training, which should include phishing modules. 

Methods of penetration tests

Three methods of penetration tests exist: black-box, white-box, and grey-box. Typically, PCI DSS penetration tests include all three of these methods of testing.

• Black-box assessments are performed without any prior authentication or even specific scoping information given by the organization to the pen tester. This could be providing the penetration tester only the IP range of the scope.
• White-box assessments are when the organization provides access and complete and detailed information of the network and all web applications prior to testing. This could include multiple roles of access, all functions performed by an application, and each IP address of every device in scope.
• Grey-box assessments are the middle ground between black-box and white-box. The organization could provide limited access to systems and applications and require the penetration tester discover additional services themselves.

The PCI penetration test process

A successful PCI pen test consists of three steps: pre-engagement, engagement, and post-engagement. Pre-engagement consists of scoping and information gathering, engagement encompasses the evaluation steps, and post-engagement is made up of reporting and retesting. You can find an overview of each step below.

1. Pre-engagement

Prior to testing, the pen tester and organization identify the test’s scope based on PCI DSS requirements and the networks, applications, systems and users in-scope. The PCI DSS specifies that the scope of a PCI pen test should include any critical systems, connected-to systems or networks, externally facing or publicly facing systems, and any isolated environments for segmentation testing.

The pen tester will then gain authorization for the test, stating the specific dates and times testing will occur potentially including the IP addresses the penetration test will originate from.

Any credentials and authentication will then be tested prior to the penetration test to ensure access is granted properly. 

2. Engagement

In this stage, the penetration tester will begin the test by gathering information about the target systems such as open ports, services, and the network topology. 

Next, the penetration tester will perform scanning to try and discover vulnerabilities and attack vectors against the in-scope systems. Once vulnerabilities are discovered, the attacker will then begin to try and exploit the vulnerabilities to try and gain access to the target systems or networks or exfiltrate data such as cardholder data. The tester will then try and escalate privileges to gain deeper administrative access by exploiting additional vulnerabilities within the systems.

During a segmentation test, the tester will be planted within the isolated systems and environment and try to gain access to the cardholder data environment. This can be done utilizing scanning techniques and then trying to utilize the services found to gain access, or by trying to exploit vulnerabilities found or utilizing credentials discovered to gain access. 

3. Post-engagement

Once the engagement is completed, the tester will document in a report the scope of the test, the methodology utilized, and the vulnerabilities found. The report should include all vulnerabilities found, the score associated regarding the severity of the vulnerability, description, and a proof of concept. The penetration test report should also include the segmentation testing details, including the originating networks, the cardholder data environment and if access was discovered.

The organization will now have an opportunity to remediate found vulnerabilities and request the penetration tester perform a retest of the vulnerabilities to ensure they have been remediated. Note that usually it is a PCI DSS requirement that critical and high vulnerabilities are remediated on the internal network and critical, high, and medium vulnerabilities are remediated on any externally facing system. 

Penetration Testing Reporting Guidelines

Penetration test reports can look very different between penetration testing companies. However, the PCI SCC does outline common content on an industry standard penetration test. This content is outlined below.

• Executive summary: A high-level overview of the pen test scope and findings
• Statement of scope: A detailed definition of all in-scope systems and networks
• Statement of methodology: Details on methodologies used for testing
• Statement of limitations: Documentation of any restrictions on testing, like designated testing hours
• Testing narrative: Details on how testing progressed and any issues encountered
• Segmentation test results: A summary of the testing performed to validate segmentation controls
• Findings: A description of found vulnerabilities, the severity based on an industry recognized scoring system, and what targets would be affected
• Tools used: Details on which tools were used during testing
• Cleanup instructions: Directions on how to clean up the targeted environment and verify security controls have been restored after testing
• Remediation evidence: The results showing which vulnerabilities were remediated, if remediation testing was performed

In addition to this content outline, you can use the checklist below to verify whether the necessary content is included in your penetration test report.

Penetration test report evaluation tool

According to Verizon's 2022 Payment Security Report,  only 68.8% of organizations across the globe maintained compliance with the PCI DSS requirement for annual pen testing. When exploring what contributed to the poor performance of this requirement, they discovered that organizations were in fact performing a penetration test but then failing to mitigate the findings. In some cases, this was because the organizations didn’t understand or were unsure where to start mitigating. 

That’s why it’s important to have some guidance on how to interpret a penetration test report and its quality. Use the penetration test report evaluation checklist below to help evaluate the completeness and depth of your report.

How Secureframe can help

Maintaining PCI DSS compliance is crucial if you impact the security of cardholder data. Fulfilling annual and semi-annual PCI DSS penetration testing requirements is an essential part of continuous compliance, but you don’t need to carry the load of achieving and maintaining PCI DSS certification alone. 

Secureframe streamlines the PCI DSS compliance process by integrating your tech stack and automating technical controls. We also partner with external penetration testing providers which have been vetted by our internal team of security experts. Looking to accelerate your PCI DSS compliance? Schedule a personalized PCI DSS demo today.