Agile Testing is different – Here is what you need to know

I started working in an Agile environment about ~14 years ago, having previously worked in “Waterfall” and “V-Model” methodologies. Like any other tester from a traditional working environment, I didn’t know what to expect at first.

Since then, I have worked with many organizations and built many Agile teams, allowing me to gain experience and knowledge about Agile implementation and, more importantly, a true perspective of testing in this fast-changing environment.

Moving from a single release to multiple releases that occur every few weeks was a big shock at the start because it demands a completely different state of mind and a different way of thinking about how the testing process is conducted.

How can I ensure that the product is tested if I do not see the full requirements that make up the big picture of how the product should work in the end? And what about the new team structure that suddenly mixes testers and programmers that must work together on all levels of the product? I remember one question raised by almost all testers in the organization: “how can we possibly test the product so quickly?” Where do we have the time to plan the testing effort and build our test plans?

After a few months, I was able to see the true meaning of being part of an Agile team and why testing is a team activity and not just owned by me. Another thing to learn is that the customer is the focal point of the whole process, which means that you will receive focused requirements due to the close relationship between the business and the customer.

As a team member, I was no longer just a tester that has to wait for work; I could find different ways to help the team by taking part in the coding activities, setting up test environments, and reviewing tests done at the lower level of the application (unit, component, and integration)

Traditional Testing at a Glance

Before we review the differences between the two environments, I want to set the baseline for this topic by reviewing the waterfall model. The waterfall model was probably the most common approach for software development projects before Agile frameworks. Let’s examine its different phases before go-live (i.e., release) and maintenance.

Phase 1 – Requirements analysis

All requirements for the software are captured in a software requirement specification (SRS) document.

Phase 2 – Design

Development teams learn the requirements defined in the SRS document and then create the system design specifications (system architecture, flows, hardware, etc.).

Phase 3 – Coding (AKA: Build)

The development team uses the system design document and develops the software (data layers, services, procedures, etc.). The basic development cycle starts with the units modules integrated into advanced development phases to create the system.

Phase 4 – Testing

The system developed in the coding phase is now handed to QA teams for functional and non-functional tests. During this phase, testers report software bugs that need to be resolved by the development teams.

Phase 5 – Release (AKA: Implementation)

The product is released to the customers once the functional and non-functional testing is complete.

Traditional Testing Vs. Agile Testing

A few core aspects differentiate Agile testing from traditional testing. These differences are driven by the nature of Agile development and based on the test principles of the Agile Manifesto. Below are some high-level differences between Agile testing and Traditional testing.

Serial vs. parallel testing activities

Now that you understand the Waterfall model, it is easy to see that the testing process happens at the end, just before release. In theory, it may seem that there is as much time for the team to test the product as there is in the earlier phases of requirement analysis and coding.

This is the theory. I have been involved in hundreds of software development processes that use this method, and I rarely had the same time for testing as was available in earlier phases. In most cases, the time allocated to the testing phase is reduced, as there is always a delay in the coding phase that takes more time than expected. When this happens, testing teams feel the impact as their time for testing is reduced.

Agile is iterative and incremental. There is no dedicated phase of testing, and by that, I mean that the testing activities will start even on the smallest part of the working code as soon as it’s available for testing. In this way, the team can test a little bit of code each time and ensure it meets the quality standards defined in the DoD.

Focus on the things that matter

Testing is a mandatory part of any project, no matter how big. In Agile, testing can refer to Epics, Features, and User Stories. No matter where you choose to start, the team must ensure a high-level understanding of the object of their attention.

Agile testing allows the team to focus on small PBIs without developing a massive test plan or test strategy as in traditional approaches. The team can focus on creating tests per PBI instead of creating tests based on the SRS document created by Product Management before a single line of code has been written. Working at the PBI level allows the team to design and execute precise and focused tests on the things that matter.

User Acceptance Testing (UAT)

The basic definition of ‘Acceptance Testing,’ as published in the ISTQB guide, is:
“formal testing concerning user needs, requirements, and business processes conducted to determine whether a system satisfies the acceptance criteria and to enable the user, customers or other authorized entity to determine whether or not to accept the system.”
In traditional testing, the team does the customer’s acceptance tests only after the software is released. Do you feel the pressure? The panic? The risk?

In Agile Testing, acceptance tests are done by the team on a sprint basis. The team follows a DoD that ensures that the team meets all the standard quality criteria. Therefore, the team can perform the tests during the sprint and before delivering the product to the customer, which reduces risk and overall pressure.

Documentation is not the main goal.

In traditional testing projects, the test team will create endless test documentation (STP, STD, etc.) containing test scenarios, test cases, and each test’s expected results. When you have weeks and sometimes even months to invest in this exhaustive test documentation and each test’s expected results, it may be reasonable. Still, in Agile testing, this timeframe is narrowed to a few days and sometimes even hours.

If you have ever worked on a traditional testing project, you already know that this approach has some major disadvantages:

• Test teams cannot create effective test plans for weeks and even months of testing while writing test documentation.
• Creating massive test plans is not worth the time spent writing them down. Imagine this time invested in the actual testing process instead.
• Writing comprehensive test documentation does not benefit team morale. I never met a tester who enjoyed writing thousands of tests that might be worthless once actual testing starts.

So, is there test documentation in Agile? Yes, there is. The difference is in the size, complexity, and amount of time invested by the team to create it. The team should focus on the essence of the tests rather than complete writing of their details.

Instead of writing a detailed functional test scenario, team members conduct their tests based on more advanced testing methods such as Exploratory Testing (ET) and Risk-Based Testing (RBT), which allow the team to run more efficient testing.

Collective responsibility for quality

There are so many jokes about the difference between testers and developers regarding software quality. The origin of these jokes is in traditional testing. As you saw, traditional testing has two different phases are independent testing from coding. As a result, each department is responsible for a single aspect of the software (developers create the code, and testers test it).

In Agile, this separation does not exist. This is because the entire team is responsible for the quality. The team’s focus is to produce quality software in a limited timeframe that maximizes its value to the customer. Most often, developers will execute manual tests and automated tests. These developers will work with their testers to guarantee the software delivered is of high quality.

Quick feedback cycles

The most important difference for testers in an Agile project is the fast feedback generated from testing, which is crucial in pushing the project forward. Good testing feedback allows the team to understand the quality of the code and improve it faster than can be done in traditional testing projects, where the testing feedback is generated after the coding is completed.

Simplified and clean code

In traditional testing, the test team reports software bugs. These bugs are fixed only after the coding phase is done. Therefore, more bugs discovered mean more fixes and more risk added to the project. In Agile, bugs are fixed as they are raised in the same Sprint. Handling all bugs in short sprints reduces risk and increases the team’s ability to create clear and resilient code and ultimately better quality deliverables.

Often, the team will fix not all bugs in the same sprint. It is common practice for the Product Owner to review open bugs with the team and decide which bugs are more important. Using this process, the team will handle only some of the bugs that do not affect the DoD and the quality of the developed code.

A new way for headlining test requirements

The foundations of any project are the requirements, no matter what the chosen test methodology is. Below is a comparison between Agile testing and traditional testing related to testing requirements.

A different mindset

As you can see, there are many differences between Agile development and traditional development. For me, the biggest and most important one is the “whole-team” approach that demands a completely different state of mind from every stakeholder involved in the Agile development process. This state of mind helps teamwork and collaboration as one strong unit to achieve a common goal of delivering quality products and increasing business value.

This whole-team approach is related to all team members, not just developers, and testers. The whole team focuses on delivering high-quality products by collaborating to initiate the testing process at the unit level and taking it to end-to-end testing.

Now, this is just a way of thinking. In real projects, some teams will try to adopt it but do not have the skills and expertise to handle the whole-team approach to testing. We add other criteria for an Agile team called “cross-functional.” This means that we must build our teams with all the technical skills needed during the testing process, allowing the team to take responsibility for all kinds of testing activities such as risk-based testing, automated testing, and manual exploratory sessions.