Compliance testing strategies in an Agile world

Publish date:

Compliance testing refers to the set of activities needed to ensure that a product, process, or system meets a defined set of regulations.

“Control leads to compliance; autonomy leads to engagement”—Daniel H. Pink

The software industry has been leveraging agile frameworks for years now. They have enabled the delivery of high-quality software to the market at optimum quality faster than traditional waterfall methods. This is why heavily regulated industries such as financial institutions and medical devices have considered adopting Agile or Hybrid Agile frameworks.

One area that continues to be a challenge and that is exponentially increasing is compliance validation of regulations such as SOX, GDPR, MiFID, PCI, etc. in an agile environment. To explain it a little better, we must understand compliance testing. Compliance testing refers to the set of activities needed to ensure that a product, process, or system meets a defined set of regulations. It is a painfully slow and stage gate-oriented process, where organizations must first interpret the regulations and provide objective evidence that the system meets the compliance requirements. It primarily comprises static testing activities in the form of audits and inspections and dynamic testing activities in the form of executable test cases.

Agile and compliance may appear diametrically opposite and one may think that there are very few synergies between them. However, there are three key fundamental differences in agile compliance validation versus waterfall compliance validation that speed the compliance validation process:

  1. Compliance testing requirements are interpreted as features are built. The regulatory controls required are built into the features, hence reducing ambiguity in requirement interpretation.
  2. Compliance evidence to meet the compliance controls is designed and agreed upon by the entire agile team, leading to greater autonomy and engagement.
  3. Given the short duration (two or four-week sprints) to deliver working products, these requirements must be continuously tested. This can be done only by developing as many automated tests as possible to enable quick feedback and course correction.

Below are key strategies to address compliance regulations in an agile environment:

  1. In Sprint 0 or release planning, determine prioritized testable requirements, organized by risk, for both regulatory compliance needs (such as SOX, FDA, etc.) and organizational compliance needs (such as ISO 900 x, etc.). This is ideally done by the product owner engaging with a regulator.
  2. Determine objective evidence to meet the compliance requirements as part of the definition of “done.” This could be in the form of review evidence or dynamic testing logs. Regulations typically provide criteria that your process needs to meet—that is the “what” but not the “how.” It is up to the entire team to determine and sign off on the activities and the objective evidence they produce.
  3. Map compliance testing requirements to the most granular level possible; for example, by release, product, feature and story. Use an Agile ALM tool to maintain traceability between requirements, test assets, and evidence.
  4. Automate dynamic compliance testing to the extent possible at a sprint, system, and release level. For example, accessibility testing a part of ADA compliance and security test controls can be easily automated. In-sprint automation may not be possible for the functional compliance tests, but they could be subsequently automated and form a part of the regression test bed executed across multiple sprints. Leverage behavior driven development for feature automation. The compliance acceptance criteria can be easily written in the gherkin format and automated with a tool such as a cucumber.
  5. Adopt a lightweight governance strategy. This should include lightweight milestones, metrics, and stated deliverables as expected by many regulations.
  6. Integrate dynamic compliance testing as a part of the DevOps pipeline to the extent possible with in-built quality gates. For example, if all feature level accessibility tests as a part of ADA compliance do not pass  , then release level compliance tests should not be executed.

In summary, agile and compliance can be a great combination of just-right rigor, control, and autonomy. Adopting the right strategies with the right degree of automation will speed up the process without comprising on quality.