Test your AI Agents with the all-new Agent to Agent Testing Platform.Learn More
Randomized testing: Gotta Catch βEm All | David Pilato | TestMu 2025
Playlist
- Opening Note by Joe Colantonio | TestMu 2025
- Keynote: Air Fryers, Automation, and AI | TestMu 2025
- Intent Over Scripts: Modernizing Software Testing with AI | TestMu 2025
- CI and the Great Flakiness Adventure | TestMu 2025
- AI for Accessibility: Empowering Inclusive Digital Experiences | TestMu 2025
- Panel Discussion: AI and Community: Shifting Roles, Rising Impact | TestMu 2025
- Ask Me Anything: Future-Proof Your Career: AI, Testing & Path Ahead | TestMu 2025
- 2025: The Agentic Shift β Are We Reasoning, Or Just Retrieving Smarter? | TestMu 2025
- Accelerating Success: How to Optimize Value Delivery with DORA | TestMu 2025
- Rapid Threat-to-Test for Agents | TestMu 2025
- How to Build Enterprise-Grade AI Agents Using Robust Evaluation | TestMu 2025
- Ship Code. Without Writing It. | TestMu 2025
- Exploratory Testing with AI | TestMu 2025
- AI-Powered Debugging & Browser Automation with Playwright MCP | TestMu 2025
- Network Control for End-to-End Web Testing | TestMu 2025
- Keynote: Zero-UI Engineering: Architecting Systems for Agent Experience (AX) | TestMu 2025
- So you think a new tool will help? Hereβs an idea-t to think aboutβ¦ | TestMu 2025
- AI, Automation & DevEx: Fueling High-Velocity Engineering | TestMu 2025
- Fast Doesnβt Mean Fragile: Delivering AI-Powered Software at Scale | TestMu 2025
- How to Test LLM Agents | TestMu 2025
- The Great Reckoning: How AI is Exposing the Existential Crisis of Software Testing | TestMu 2025
- Your Test Suite Canβt Catch a Hallucination: Real Talk on AI in Production | TestMu 2025
- Event Driven Architecture: Love Triangle in Integration Testing | TestMu 2025
- When Life Gives You Lemons⦠Are You Counting Them or Making Lemonade? | TestMu 2025
- AI-Driven Quality Engineering Practices | TestMu 2025
- Transforming Retail with Quality Engineering for Seamless Digital Experiences | TestMu 2025
- Role of Quality Engineering in Shaping the Future of Financial Services | TestMu 2025
- Opening Note Day 2 | TestMu 2025
- Build Your Testing Sidekick: Custom Tools with Model Context Protocol | TestMu 2025
- Reactive Browser Testing with WebDriver BiDi | TestMu 2025
- How Software Testing can Increase Agent Autonomy | TestMu 2025
- What can go wrong with AI in testing? | TestMu 2025
- Code It Forward: Making Your Mark in Open Source | TestMu 2025
- Testing the Untestable: Agent to Agent Testing | TestMu 2025
- Testing Early, Testing Right - Balancing Early Testing with Real-World Reliability | TestMu 2025
- The Enterprise AI Playbook: Strategies for Scaling AI in Quality Engineering | TestMu 2025
- Advanced Playwright with AI | TestMu 2025
- Generative to Agentic to Quantum - The Evolution of AI | TestMu 2025
- Test Data Key to Effective Test Coverage | TestMu 2025
- QE Strategic Shift: What's Changing with AI, Automation, and Speed? | TestMu 2025
- Building AI Fluency: Leading Teams Through the Learning Curve | TestMu 2025
- The Practical Automation Playbook | TestMu 2025
- Building a Handwriting Recognition System for the New York Times Crossword | TestMu 2025
- Agentic Cloud: Using Agents to Build Tomorrowβs Cloud | TestMu 2025
- QA to QE: Scaling Quality with Ownership, Code, and Culture | TestMu 2025
- Automated Test Data Portal for Financial Services | TestMu 2025
- QA in the Age of AI: Enhancing Agent Reliability Through Evaluation-Driven Development | TestMu 2025
- Ensuring quality testing in an AI-driven world | TestMu 2025
- AI-Driven Strategies for Scalable & Resilient Performance Engineering | TestMu 2025
- Day 3 Opening Note | TestMu 2025
- Mastering Appium 3: Architecture, Gestures & Beyond | TestMu 2025
- From Zero to MCP: Automating Test Environments for DevOps & QA | TestMu 2025
- AI & GenAI in Quality Engineering: Crawl, Walk, Run | TestMu 2025
- Embracing Agentic AI: From Autonomous Goals to Enterprise Guarantees | TestMu 2025
- Oops, AI Did It Again: How to Get AI to Stop Being Weird and Actually Be Useful | TestMu 2025
- Should We Let AI Take Over Test Automation Completely? | TestMu 2025
- From Hours to Minutes: Run Thousands of CI Tests in Just Minute | TestMu 2025
- Evaluating RAG Applications: From Retrieval to Response Quality | TestMu 2025
- Stop Breaking Your Teams: Seeing the Whole Instead of Pieces | TestMu 2025
- Surviving and Thriving with AI in QA | TestMu 2025
- The Quality Leadership Shift: From Compulsiveness to Cautiousness | TestMu 2025
- Full Court Quality: Lacing Up for Speed, Stability & Style | TestMu 2025
- Navigating Mobile App Testing and App Store Rejection: From Review to Release | TestMu 2025
- Randomized testing: Gotta Catch βEm All | TestMu 2025
- Balancing release & sprint delivery speed with thorough testing | TestMu 2025
- Building for AI at Scale: Infrastructure, Integrity, and Innovation | TestMu 2025
- Trusting the Machine: Building Confidence in AI-Driven Testing Decisions | TestMu 2025
- Observability - Holistic Quality across Software Systems | TestMu 2025
- From SDLC to ADLC: The Enterprise Agent Development Lifecycle | TestMu 2025
- Agentic Testing: Your Skilled Human Tester | TestMu 2025
- Evolution of Quality Engineering in Financial Services | TestMu 2025
- Closing Note Day 3 | TestMu 2025
About the talk
In this TestMu session, πππ―π’π ππ’π₯πππ¨, Developer Evangelist at Elastic, demonstrates how introducing randomness into your test strategy can uncover hidden bugs youβd never think to write by hand. From obscure edge cases to unpredictable failures, randomized testing helps teams discover problems before they hit production.
David shares real-world examples from Elasticsearch and open-source projects, showing how randomness in locales, time zones, cluster sizes, and execution order can surface subtle issues. He also dives into the mechanics of randomized testing frameworks, the role of seeds in reproducibility, and how to balance randomness with consistency in CI pipelines.
Whether youβre a developer or tester, youβll leave with practical insights on how to make βluckβ your ally in testing.
πππ¬π¬π’π¨π§ πππ² πππ€πππ°ππ²π¬:
Why randomness matters: uncovering hidden bugs that traditional test cases miss.
How seed-based randomness ensures reproducibility in failing tests.
Practical use cases: locales, time zones, shards, replicas, and execution order.
Strategies to integrate randomized testing into CI/CD pipelines.
Lessons from real-world failures and open-source projects.
TestΞΌ
TestΞΌ (TestMu) is more than just a virtual conference. It is an immersive experience designed by the community, for the community! A 3-day gathering that unites testers, developers, community leaders, industry experts, and ecosystem partners in the testing and QA field, all converging under a single virtual roof. π
