Who Is Testing? What Tools Will Help Them? A Report on Research in Progress: Isabel Evans [Testμ 2022]

After more than 30 years of being a testing practitioner, working on software industry projects, and quality, Isabel Evans is a part-time postgraduate student at the University of Malta. Her research focuses on human factors in test automation. Her interest in this topic arose from real-life experiences as a test manager, quality manager, and test consultant.

A popular speaker, and a storyteller at software conferences worldwide, Isabel has chaired many significant events such as Eurostar and Hudson.

In this session from the Testμ Conference 2022, Isabel shared the findings from her research between 2017 and the summer of 2022. She has been researching the experiences of testers with their test tools.

If you missed this insightful session, here is the video link.

Isabel started the session by introducing testers’ experiences with the toolset they have been using and whether their tools help and support them.

She divided her session into the following parts:

• Report back on early results.
• The latest developments in her research and where it seems to be headed.
• What are the next steps of the experiment going forward?

Starting with how Isabel started learning about research and her journey in 2017, it led to conversations and interviews with a wide range of people in the industry and collating those findings. It also led her to come up with a new hypothesis which made her realize that her original thought was somewhat going in the wrong direction. Therefore, she focused on conducting recent surveys and refocusing her research during 2021-22. She concluded from the data that the testing community is quite diverse and that not everyone can use the tools offered.

This made Isabel develop a Design Framework, which differs from an automation framework because it works around developing tools based on a tester/project’s requirements.

To begin with, Isabel highlighted that the earliest results stated that people were experiencing a lot of challenges than expected. They also thought certain things were quite important such as usability came up a lot in comments from individuals, both as a characteristic to look for in a tool and as a barrier to successful tool use, along with quality and the use case of the tools. However, additional technical aspects of the tools, such as the maintainability of the tests over time, and difficulties in accessing them by security, were all identified as obstacles.

There were 232 issues raised, some of which were positive. Isabel noticed something that wasn’t expected. Surprisingly, many people not only related to the objective instead, but many of them also displayed strong emotions.

She said –

“When we’re talking about usability, what are we talking about?”

Isabel explained what usability means and how it connects with people’s experiences. Looking at the slide above, on the left, there’s the UI design which is supported by interaction design and contributes to usability. It’s not the whole of usability because usability also means factoring in the goals and the context. It is one of the contributions towards quality, including effects, flexibility, and freedom for risk.

She summarized that user experience is a contributor to people’s lived experience, so their emotional response to what’s going on and what they’re experiencing as human beings has been presented in the data: the tools and the experiences of attempting to implement tools were affecting people’s lives quite profoundly. Therefore, getting a good UI isn’t enough, and multiple other aspects must be considered to optimize the user experience.

What are the next steps of the experiment going forward?

Isabel begins to explore if there are other ways to understand “who is testing?”. This starts by realizing the context within which the tool will be used and then moving to whether the person who makes the tools is also a user.

Hence, understanding that straying further away from these contexts could be the reason for the misunderstanding between testers and their tools.

This is where Isabel introduces the idea of making the tools more people-centric.

Creating a framework for tools designers, that’s people driven and based on evidence that highlights more about the characteristics of testers and their approaches as a consequence can help the designers to use the framework as a guide to help them understand their users better.

Dividing the model into four aspects:

• A1 – The Tester

In terms of the tester, beginning with what information can be collected about their capabilities and preferences. And collectively, look at communication styles, team shapes, and problem-solving styles.

• A2 – Their Organizational Chart

Focusing on the organizational culture and technical maturity level, for example: How does the organization define quality, leadership styles, and technical maturity within the company?

• A3 – The tester’s goals

This aspect highlights whether the tools used by a tester align with their goals or not.

• A4 – The Tester’s activities

A brief look into the daily activities of particular testers or a specific system they intend to follow while conducting tests. These usually tend to differ from person to person and therefore need to be included as a major part of the model.

• A5 – The Tools

What are the tools that the testers have been using in the past to help their tests, and what blockers are they experiencing while working with them?

According to Isabel:

“Bug management activity is done by different people for different reasons. It may be very niche and focused but is quite broad in its impact.”

Lastly, Isabel summarizes that moving forward, her research will collect data through meta-studies, that is, working on information that other people have gathered across the industry and re-analyzing it. She will also conduct case studies focusing on bug and defect management as an activity and the tools to support it. She explained her reason behind it being that it is central to everything done in testing, and it’s not the only reason testing is done; testers are constantly raising defects and managing them. This affects a wide range of people who are not only testers but people in other roles, and it is a method of discovering the risk and keeping track of progress broadly.

The session ended with a few questions asked by the attendees to Isabel. Here are some:

• What does everyone think about pushing automation left and starting that in the early phases? Which Test Automation should we start in the initial phase of Development?
• What factors should be prioritized when selecting a test automation tool?
• Can you share some good reads on usability testing and related heuristics?
• How important is QA to participate in development planning sessions?

After successful Testμ Conference 2022, where thousands of testers, QA professionals, developers worldwide join together to discuss on future of testing.

Join the testing revolution at LambdaTest Testμ Conference 2023. Register now!” – The testing revolution is happening, and you don’t want to be left behind. Join us at LambdaTest Testμ Conference 2023 and learn how to stay ahead of the curve. Register now and be a part of the revolution.