PCI DSS Level 1 Assessments: Mastering the Art of Crafting Effective Responses to your Assessment Findings.

BACK TO BLOGS

Introduction

From 31 March 2024, PCI DSS Assessors will need to understand and apply the new rules for crafting suitable responses and let’s face it, despite this being the opportunity to shake off some bad habits, many Assessors will not have the time and will need to learn these new rules ‘on the hoof’ (e.g., While carrying out an assessment).

Many of the seasoned Assessors will be extremely familiar with the v3.2.1 ways of writing up an assessment and many of these will be the naysayers, who will preach or argue that this has not changed or that it has only slightly changed. However, in this article I will present my insights and some recommendations so that you can judge this for yourselves and decide whether my recommendations will be beneficial to a QSA, or not.

Background

The current Report On Compliance (ROC) template requires the application of a very much an evidence-based approach, whereas, with the v.4.0 iteration this adds a layer of evidence-focused practices. Consequently, crafting suitable responses to the assessment findings requires a different approach.

As a gentle refresher, in v.3.2.1 the Assessors have the choice of one of the following five responses:

  1. In Place.
    • The expected testing has been performed, and all elements of the requirement have been met as stated.
  2. In Place with CCW (Compensating Control Worksheet).
    • The expected testing has been performed, and the requirement has been met with the assistance of a compensating control.
    • All responses in this column require completion of a Compensating Control Worksheet (CCW)
    • Information on the use of compensating controls and guidance on how to complete the worksheet is provided in the PCI DSS.
  3. Not In Place.
    • Some or all elements of the requirement have not been met, or are in the process of being implemented, or require further testing before it will be known if they are in place.
  4. N/A (Not Applicable).
    • The requirement does not apply to the organization’s environment.
    • All “not applicable” responses require reporting on testing performed to confirm the “not applicable” status.
    • Note that a “Not Applicable” response still requires a detailed description explaining how it was determined that the requirement does not apply. In scenarios where the Reporting Instruction states, “If ‘no/yes’, mark as Not Applicable,” assessors may simply enter “Not Applicable” or “N/A” and are not required to report on the testing performed to confirm the “Not Applicable” status.
    • Certain requirements are always applicable (3.2.1-3.2.3, for example), and that will be designated by a grey box under “Not Applicable.”
  5. Not Tested.
    • The requirement (or any single aspect of the requirement) was not included for consideration in the assessment and was not tested in any way.

The Reporting Instructions (Ris) provide the explanation of the intent of the required response and there is no need to repeat the testing procedure or the reporting instruction within each assessor response. Each response should be specific and relevant to the assessed entity, with the details providing a concise quality of detail, rather than lengthy, repeated verbiage and the parroting of the testing procedure (without additional detail or generic template language) should be avoided.

Figure 1: ROC Template, v3.2.1 – PCI DSS Requirement 9.2.

The Assessor responses generally fall into the following categories:

  • A single word response (yes/no).
  • A document name or interviewee job title/reference (Sections 4.9, “Documentation Reviewed,” and 4.10, “Individuals Interviewed”).
  • Sample description.
    • For sampling, the QSA must use the table at “Sample sets for reporting” in the Details about Reviewed Environment section of this document to fully report the sampling, but it is the QSA’s choice to use the Sample set reference number (“Sample Set-5”) or list out the items from the sample again at the individual reporting instruction response.
    • If sampling is not used, then the types of components that were tested must still be identified in Section 6 Findings and Observations. This may be accomplished by either using Sample Set Reference numbers or by listing the tested items individually in the response.
  • Brief description/short answer.
  • Short and to the point but providing sufficient detail and individual content which DOES NOT simply an echo the testing procedure or reporting instruction nor a template answer used from report-to-report, but instead relevant and specific to the assessed entity.
  • It is important that these responses include unique details, such as the specific system configurations reviewed (which should include what the assessor observed in those configurations), and the specific processes observed (which must include a summary of what was observed and how it verified the testing criteria).
  • Responses must go beyond simply stating that the requirement was verified. They must also include details on how it was met.

Now, for a level 1 assessment, under v4.0, what goes into crafting a suitable Assessor’s Assessment Findings response is a much more involved process.

Understanding the Intricacies of the Rules for PCI DSS v4.0 ROC Assessment Findings Responses

In PCI DSS v4.0, the number of Assessment Findings have been reduced from five to four and each has an assessor’s response narrative and specific testing and evidence requirements:

  • In Place.
    • The expected testing has been performed, and all elements of the requirement have been met.
      • A combination of how the testing and evidence demonstrates the requirement is In Place.
    • Not Applicable.
      • The requirement does not apply to the organization’s environment.
      • Not Applicable responses require reporting on testing performed to confirm the Not Applicable status including a detailed description explaining how it was determined that the requirement does not apply.
      • Note that reporting instructions that start with “If Yes” or “If No” do not require additional testing to confirm the Not Applicable status. For example, if the Reporting Instruction was “If Yes, complete the following” and the response was “No” then the assessor would simply mark that section as Not Applicable, or N/A and no further testing is required.
        • What testing was performed and how does the results of this testing confirm that the requirement is Not Applicable?
      • Not Tested.
        • The requirement (or any single aspect of the requirement) was not included for consideration in the assessment and was not tested in any way.
        • Note: Where Not Tested is used, the assessment is considered a Partial
      • Not in Place.
        • Some or all elements of the requirement have not been met, are in the process of being implemented, or require further testing before it will be known if they are In Place.
        • This response is also used if a requirement cannot be met due to a legal restriction, meaning that meeting the requirement would contravene a local or regional law or regulation. The assessor must confirm that a statutory law or regulation exists that prohibits the requirement from being met.
        • Note: Contractual obligations or legal advice are not legal restrictions.
          • How does the testing and evidence confirm that the requirement is Not in Place?
          • If the requirement is Not in Place due to a legal restriction, how does the statutory law or regulation prohibit the requirement from being met?

As highlighted in Figure 2 & 3, each of the Assessment Findings can be grouped into eight categories and with some of these groups being able to be divided into sub-groups:

  • Group 8 (Eight evidential requirements).
    • 9.3.2. Procedures are implemented for authorizing and managing visitor access to the CDE.
      • 2 x Documentation evidence requirements.
      • 2 x Interview evidence requirements.
      • 4 x Observation evidence requirements.

Figure 1: Group 8

  • Group 7 (Seven evidential requirements and three sub-groups).
    • 2.2. All changes to network connections and to configurations of NSCs are approved and managed in accordance with the change control process defined at Requirement 6.5.1.
      • 3 x Documentation evidence requirements.
      • 2 x Interview evidence requirements.
      • 2 x Systems examinations.
    • 3.1.1. Physical access to sensitive areas within the CDE for personnel is controlled.
      • 3 x Interview evidence requirements.
      • 2 x Observation evidence requirements.
      • 2 x Systems examinations.
    • 6.1. A change- and tamper-detection mechanism is deployed.
      • 3 x Documentation evidence requirements.
      • 1 x Interview evidence requirement.
      • 3 x Systems examinations.

Figure 1: Group 7.

As detailed in Figure 4, another feature of the new enhanced evidence-focused practices is the inclusion of the additional Customized Approach reporting option, instead of the Defined Approach, which provides the assessor with an increased scope for applying a more flexible of pragmatic approach.

Figure 4: Defined Approach Versus Customized Approach

Applying the Rules to Create Suitable Assessor Findings Responses.

As you can see from Figure 5, the Ris fall into four distinct categories:

Figure 5: RI categories.

Okay, so to help you create a suitable assessor’s Assessment Finding response for an ‘In Place’ finding, it is important to ensure that the narrative incorporates the testing procedures that the assessor carried out, along with the supporting evidence.

One approach that you might consider is applying the following formula:

“Assessor [reviewed/sampled/observed/etc.] [whatever process/action/state/sample set/etc.], compared [it/them] to the corresponding documentation and responses obtained during interviews and verified that [paraphrased control verbiage from the Requirement] is [in-place/not in place].”

However, I would urge caution here as highlighted in the ROC Instructions from the v3.2.1, the use of the ‘paraphrased control verbiage from the Requirement’ may not be deemed to be suitable. Alternatively, you might find the following example (as shown (using the 27k1 RMS) in Figure 6), against 9.3.2 (Group 8), to be a more comprehensive and suitable assessor’s Assessment Finding response:

Figure 1: Example Assessment Finding Response

Whilst excluding the need to paraphrase the control verbiage, this example narrative includes a specific cross-reference to the expected testing and the supporting evidence which is needed to validate the PCI DSS Requirement as being assessed as being ‘In Place’.

Conclusion

Whilst both approaches might be deemed to be suitable, it is very important to ensure that your chosen approach align with the PCI SSC’s Reporting Expectation Do’s and Don’ts list, as shown in figure 7.

Figure 1: Reporting Expectations – Do’s & Don’ts

By breaking down the RIs and evidence expectations for each of the 260 Assessment Findings and grouping them, you might find that you will be better prepared to create suitable assessor responses, which meet all the expected criteria:

  • Specific and concise quality of detail, rather than lengthy, repeated verbiage.
  • Avoiding templated language.
  • Specifically detailed relevant to the assessed entity.