QA Transformation Journey

In the following article, I will talk about a situation that many QA professionals have encountered and what I would recommend the approach should be to move forwards.

The problem statement

Imagine you are in a scenario where there are lots of incidents, delivery is slow and customers are not happy with the quality of your products.

What should you do?

As a Head of QA, and Test Manager in the past, I have had experience in these situations before. I will now take you on a journey of change… grab a nice drink and come on the journey with me.

Step 1: Investigate

The first step is to hold independent interviews with multiple team members to understand the problem at hand.

I would recommend asking open-ended questions like:

• What is the reason in your view for the incidents and customer complaints?
• How would you describe the culture?
• If asking questions outside the QA team:
• How would you describe the QA approach?
• How do you collaborate with the QA team?

Once the inspector gadget phase is over, we must now digest the information. I would recommend independent chats or surveys with QA, product, engineering, delivery, and customers (if possible). These chats should not be biased in any way and it should be shared by the interviewee that chats are confidential, with no names or details being shared outwards.

Step 2: Data analysis

Based on the data gathered, this data should be analyzed to then decide where to focus your attention.

For example, in my experience data can be generally focused on the following:

1. Cultural issues
2. Lack of ‘Assurance’
3. Poor product knowledge
4. Lack of collaboration between Product, Engineering, and QA

Step 3: What to do?

The next aspect is to decide what to do. I will explain some examples of what I did for each scenario, with additional examples.

1. Cultural issues

a. Poor work-life balance

Start to track hours worked and establish why working hours are so long. People MUST have a work-life balance. In the past, this has been due to poor estimation in sprints and also the fact that a change in mindset is needed by senior management.

b. People are not developing

People MUST be developing, a successful company is built around its people. I would recommend ensuring that there is a career development plan in place for each member and also ensuring there are training services available via the learning and development team.

2. Lack of assurance

This generally can be a standard issue across most companies. Yes, we want to work in an agile way and release at pace, but at the same time, we must have some of the basics in place so that we know that we are testing the right things. If the basics are not in place it generally will lead to incidents and customer complaints. Key assurance documents that I would recommend:

1. QA strategy – aligned to your technology strategy, what are you testing and why?
2. Testing approach – what is your particular approach to testing on a particular project/squad?
3. General overview on what you are testing and why? At least then you can then collaborate with engineering, product, and stakeholders to review gaps in your test cases.
3. Poor product knowledge

In this case, without a great relationship with the product team it may mean that we are testing the wrong aspects, also that incidents/customer issues are increasing – which is bad! How can you improve your product knowledge:

1. Get the product team to review and contribute to the test coverage in the test cases.
2. Get the product team to review and contribute to the acceptance criteria on tickets.
3. Meet regularly with the product team and ensure that product knowledge is shared/updated.
4. Poor collaboration with Product and Engineering

This is a complex one, though firstly there should be a walkthrough between QA and Engineering once the code has been developed (shift left). From a product perspective, they should be reviewing test cases and sharing gaps in the test cases.

Step 4 – How do we maintain success?

We should track whether our mitigating measures have made a difference or not. Going back to our problem statement, we wanted to see whether we can reduce customer incidents and also customer complaints.

I would recommend a full post-mortem review as learning when incidents take place and reviewing whether the incident amount is going down. If they are not, do another review independently to establish the root of the issues.

Tracking success can only be done if we start to track certain metrics.

Aspects that I have found useful to track:

1. Escaped defects
2. Customer complaints
3. Test cases that have been added to your regression suite every sprint
4. Unit test coverage
5. Number of times a ticket is re-tested and defects found against it

Conclusion

Quality assurance transformation is a journey piece, and by tracking QA metrics, step-wise improvements can be achieved.

Success cannot be maintained solely by QA, rather it is a collaborative effort with QA, Engineering, product, and the wider team.

Dileep Marway

Founder of Be More Meerkat (https://bemoremeerkat.com/), where I write blogs on leadership and I provide technology consultancy.
I am passionate about the use of technology and innovation to create change, deliver value and solve complex problems whilst improving the lives of users, by using the most effective technology stack.
A commercially focused, results driven executive level technology leader with over 15 years experience: leading strategy development and execution, successful digital transformation and delivering organisational, IT change alongside cost effectiveness (to budget), secure and high availability operational IT.
Extensive experience of outsourcing and managing IT delivery in multi-site environments.
Strongly focused on partner selection and management, actively managing IT and change risk with an emphasis on quality assurance (QA), and full review of technology and engineering effectiveness.
I create, scale and optimise business portfolios that bring customer success and better service. To accomplish this, I focus on delivering key outcomes and building “high performing” teams. I act as a bridge between the C-suite and the technology technical teams.

See author's profile