Balancing the Test Pyramid the AWS way!

Maintaining a balance between extensive test coverage and operational efficiency has been an extreme challenge in software testing. AWS has taken up the stand to confront this dilemma, presenting the strategy of combining hybrid UI and API testing with an innovative synthetic canary testing approach. Being aware of the complexity, the AWS team embarked on developing a methodology that not only addresses this complication of the balance but also upholds the most successful approach to quality assurance.

The speaker has discussed exploring the AWS team’s hybrid testing approach, combining UI and API testing to achieve test coverage and efficiency and maintain a balance between them. During the session, the speaker also discussed how the AWS design and implementation of test coverage with a hybrid mode simultaneously validate user interface interactions and back-end functionality. She also explored techniques to manage and maintain synthetic canary to enhance your organization’s testing efforts and ensure your systems’ availability and reliability. Along with all this theory, the speaker has also guided via practical implementation, giving an insight into the tools and frameworks utilized by AWS teams to implement the hybrid UI and API testing strategy and apply these practical techniques to enhance your testing strategies.

About the Speaker

Min Xu has 15+ years of industry experience and is currently serving as the Manager of the engineering teams at AWS. She worked as an integral part of Amazon for the past five years. Before her tenure at Amazon, she spent five years at ECM as a Senior Software Engineer, where she shaped her technical skills and gained expertise in software development and quality processes. Before joining ECM, she worked for Honeywell for five years as a software specialist, where she played a key role in ensuring the delivery of high-quality products and developed a solid foundation in quality management methodologies.

She is passionate about leveraging her expertise to help AWS maintain and deliver high-quality standard services.

If you couldn’t catch all the sessions live, don’t worry! You can access the recordings at your convenience by visiting the LambdaTest YouTube Channel.

Agenda

The outlined agenda set the flow for the session, laying the foundation for what was to come.

• Introduction
• Understanding UI Testing
• Grasping API Testing
• Combining UI and API Testing
• Real-world Scenarios: “UI & API Testing in Action”
• Synthetic Canary: Testing in Production
• Key Takeaways

Introduction

Min Xu started with a walk-through of the agenda and an introduction, talking about what issues and at what percent the bugs can be identified and captured before production release.

Some of the challenges of comprehensive test coverage, as discussed by Min Xu, are as follows:

• Diverse Test Scenarios: Testing across various situations is difficult; each test must ensure the software works correctly.
• Evolving Codebase: Frequent changes in the code make it hard to maintain, and new code might lack proper testing.
• Integration Complexity: Testing how different parts work together is complex, especially during integration.
• Time and Resource Constraints: Limited resources and time can lead to incomplete tests and bugs.
• Manual vs. Automation: Choosing between manual and automated testing is the key to intuition and efficiency.

UI Testing

UI testing involves checking if the software interface works as expected. The testers are responsible for UI testing; these testers click around and use the program like a user to record or capture bugs or errors.

Why UI Testing?

UI testing is important to ensure that software looks and behaves as the user expects. It checks that buttons, menus, and all visual elements function properly. This helps find problems like buttons not responding or text appearing wrong, ensuring users have a smooth and frustration-free experience. By catching these issues early, UI testing helps create software people find easy to use and enjoy.

Types of UI Testing

Min Xu thoroughly explained each UI Testing type so her audience could understand.

• Exploratory Testing: This type of testing is like an adventure where testers explore the software freely, trying different things to uncover hidden bugs and issues that might not be found using planned tests. It’s like being a detective, finding clues that other tests might miss.
• Usability Testing: Consider this as asking regular people to use the software and watching how they react. Testers observe if users find the software easy to use, where they might get stuck, and what frustrates them. It’s like making sure a new game is fun and clear.
• Navigation Testing: Imagine you’re in a new place and want to see if you can easily move around. Navigation testing checks if users can smoothly move from one part of the software to another, like going from a homepage to a settings page. It’s like ensuring road signs guide you well in a new city.
• Compatibility Testing: This is like seeing if your favorite app works on different phones or computers. Compatibility testing ensures the software looks and functions correctly on various devices and web browsers. It’s like ensuring a recipe works regardless of your kitchen.

API Testing

API testing is like checking if different computer programs can talk to each other correctly. It’s like making sure the parts of a machine fit and work well together. This type of testing ensures that data and commands flow smoothly between software components.

Why API Testing?

API testing ensures that different pieces of computer code can talk to each other without problems. By testing how software components interact, API testing helps catch errors early and guarantees that the final product works as expected.

Types of API Testing

Min Xu provided a comprehensive explanation of each type of API Testing, ensuring her audience could grasp the concepts effectively.

• Function API Testing: Testers ensure that individual software functions, like sending a message or saving a file, do what they’re supposed to without any mistakes.
• Load API Testing: Testers check if the software can handle many users or tasks without slowing down or crashing.
• Security API Testing: Testers try to find ways to break into the software and fix any weak spots that bad actors could use.
• Integration API Testing: Testers ensure that different software parts, like login and payment systems, cooperate smoothly.

Combining UI & API Testing

When testing an online shopping website, UI testing ensures buttons and links work for users, while API testing checks if the shopping cart and payment systems communicate correctly behind the scenes. This way, we ensure the website looks good and functions well for customers.

Min Xu highlighted a real-time case study to make her audience understand how UI & API Testing can work together.

Further, Min Xu discussed how they have divided the UI and API functions to make them work together and perform integration testing when each module with a combination of UI & API is developed.

Synthetic Canary: Testing in Production

Min Xu started by explaining what a canary is in a very basic manner, and then she discussed more in detail how synthetic canary testing in production helps.

Canary: A small group of users or a simulated user helps test if things are working fine. Synthetic Canary: Instead of real users, the simulated user follows certain steps like a real user would. It’s like having a robot user.

Key Takeaways

Min Xu ended her session by recalling her entire session as a brief.

Software applications are becoming complex with each passing day! This makes it imperative to focus on observability and testing. Read the key takeaways from Min’s insightful session#TestMuConf 👇 pic.twitter.com/bzuPP5U5r1

— LambdaTest (@lambdatesting) August 23, 2023

The next slide in the session showed the Multi-Factor authentication (MFA) process with AWS for user login.

Min Xu deep dives into how to automate user login with MFA (Multi-factor Authentication) using AWS.#TestMuConf pic.twitter.com/t6CvEeXuib

— LambdaTest (@lambdatesting) August 23, 2023

Min Xu conducted a fantastic session, wrapping it up by addressing questions from the participants during the Q&A session.

Q & A Session

1. Why is API less important than when the UI won’t work without it? The backend can work without the UI in a microservice architecture.

   Min Xu: Depending on the testing scenario needed, you can use the AWS cloud for optimization.

2. What strategies or techniques does AWS use to ensure an optimal balance between integration and unit tests?

   Min Xu: There can be two strategies for balancing integration and unit tests; one is Smart Building Blocks, where you create small yet very strong “blocks” of code that work well on their own. Second, you need to take care of early mix checks that ensure the block fits early on and match them before building.

3. How does AWS handle performance testing and optimization for applications hosted on its cloud infrastructure?

   Min Xu: AWS uses scalable resources to match real-world conditions and employs various tools to fine-tune the application’s performance, ensuring it delivers a smooth experience to users.

Feel free to add questions or clear your doubts on our LambdaTest Community.