Playwright Visual Regression Testing Complete Guide

This tutorial focuses on Playwright visual regression testing, providing a reliable way to test web applications and ensure that they look and behave correctly across different browsers and platforms


Total Chapters (22)

Chapter 14 : Playwright Visual Regression Testing Complete Guide

Testμ Conference


Have you ever heard ‘How could you miss that?’ or ‘That bug is so obvious, anyone could see it’ - You would have heard it many times, and retrospectively you can agree, you’ve missed some clear-as-day bugs, which although may not be functional issues, are formatting or layout related.

As delivery is constantly accelerating and the time spent manually checking and exploring the website decreases, we need a tool that can take some weight off our shoulders. This is where Visual Regression comes in.

This additional quality gate can analyze and verify that our web page's content, layout, and design are as expected - as our product evolves through iterative releases. Images and designs can have a significant impact on the success or failure of a product. By looking at how they have evolved, we can better understand what aspects have worked and what haven't.

In this Playwright testing tutorial, we will focus on visual regression testing with Playwright, a popular tool for visual regression testing, because it provides a reliable way to test web applications and ensure that they look and behave correctly across different browsers and platforms.

So, let’s get started with this Playwright visual regression testing tutorial!

What is visual regression testing?

Visual regression testing focuses on image and pixel comparison - it allows us to define the view, area, or component of a website where we would like to perform visual analysis. We start with our baseline image, our ‘master/main’ branch equivalent, and what we expect the site to look like. We then compare it with our current screenshot, which may have been taken in a different environment or branch.

Here is an example. We have taken two screenshots from the Playwright homepage. We will label them A and B.

  • A is what the website looks like right now, and
  • B is what it looks like after we’ve made some changes on the local machine.

Can you tell the visual or styling issues in less than 10 seconds?

Playwright Visual Regression Testing A and B

Did you find them? Three styling issues have been raised in our visual comparison below - the header navigation, hero buttons, and browser logos.

This comparison was executed with Playwright. It was run once to create a baseline and then edited for the purposes of this example. Finally, it was run the second time to compare our baseline to how the Playwright homepage looks.

Playwright Visual Regression Testing A and B

Maybe some of you found 1 or 2, and maybe some of you found all 3 - but it should start to become clear that, as humans, we aren’t designed to pick up on these small visual changes in small spaces of time. The reason to find them in less than 10 seconds is how fast our visual tools can find them.


Benefits of visual regression testing

So why should we introduce visual testing? What does it bring to the team or product you are working on, and how can you show its value?

  • Economical and scalable - in terms of time invested in setting up and getting to the point where you can execute tests, you’ll have something up and running in no time. Visual testing is built-in with the Playwright framework, so no additional installations or plugins are needed
  • Super-fast execution - you’ll be able to implement it in your build and release pipelines with minimal impact on runtime, with tests executing in seconds, not minutes.
  • Simplicity - as a testing type, you don’t need to debug it - if a test fails, you look at the visual output, and all issues are highlighted in red. It means the barrier to entry for all team members is low, and everyone can collaborate on it
  • Code confidence - it’s another gate or safety net that you can add to your project and a way to raise the confidence in the quality of your code changes further.

What will visual regression testing not catch?

So far, visual regression sounds great, and you can’t wait to add it to your project, but keep in mind it won't replace your existing test suites. Here is a quick list of bugs or issues it won’t pick up on:

  • Functional issues - clicking on links, buttons, and general interactions with your website
  • API or Network issues - requests for content or data in the background
  • Performance or security issues - response times or code vulnerabilities

Those are just a few, but you get the idea - visual testing focuses on the appearance and layout of the website and doesn’t dig deeper than that.

Implementing visual testing for a single web page

In this Playwright visual regression testing section, we will be using Playwright for our visual testing, so ensure you have the relevant frameworks and dependencies installed before we get started.

Once set up, delete unnecessary folders, so your structure looks like the one below. The example.spec.js contains a basic code snippet to get us started with Playwright visual regression testing:

File: example.spec.js

// example.spec.js
const { test, expect } = require('@playwright/test');

test('example test', async ({ page }) => {
  await page.goto('');
  await expect(page).toHaveScreenshot();
Playwright Visual Regression Testing example.spec.js

To run our basic visual test, we execute the following (these tests run headless, so you won’t be able to see the test running):

npx playwright test

You should notice a few things - first, all our tests failed, which is fine. And that we have some new folders created. The first execution has no baselines to run against, so the tests fail, and the baselines are created for us.

The folder tests/example.spec.js is where our baseline .png files are stored. And the test-results folder is where we can see our Playwright visual regression testing results. You will also notice we have run our visual tests across three different browsers - Chromium, Firefox, and Webkit.

Playwright Visual Regression Testing tests/example.spec.js

Let’s run the tests a second time! This time we will notice all of our tests have passed, which is to be expected - the baseline images we took are being compared to the website in its current state, which is the same. You will also notice that our test-results sub-folders are empty this time (highlighted blue). This is because there were no failures to report on.

Playwright Visual Regression Testing small manual changes

Next, let’s check that our Playwright visual regression testing is working correctly by making some small manual changes to one of our snapshot baselines - the changes we’ve made are meant to simulate a real-world situation where elements are ordered incorrectly, or missing entirely, due to a code change.

breaking change to one image Playwright visual regression

Although we have only made a breaking change to one image in this Playwright visual regression testing tutorial, two of them have failed - let’s look into both failures further and see what happened. There are two places where we can check the visual results - directly in the test results folder and by opening the files that end with diff.png or the generated HTML report.

This is the file view within Visual Studio Code; as you can see by the red highlights, we have locators out of place or missing entirely. This is good for a quick check, and we can switch between the actual, expected, and diff files to see what has changed.

nicer way of viewing it within the Playwright report

There is also a nicer way of viewing it within the Playwright report. When your test execution finishes, another window should appear which contains the HTML report:

Serving HTML report at http://localhost:9323. Press Ctrl+C to quit.

If we open that page, we will see more useful information - the first is the error logs. It tells us that 744 pixels are different than what was expected, and the ratio comparison of all other pixels in total. This information is useful, as we can manage threshold ratios for our comparisons, which I will get into in more detail later.

 - 744 pixels (ratio 0.01 of all image pixels) are different.

If we scroll further down the page, we will see similar ‘actual’, ‘expected,’ and ‘diff’ images - and a nice additional feature, which lets us swipe between views:

gif Playwright Visual Regression

That test was the one we intentionally changed. Let's look at the comparison for the file we didn’t change. As you can see, there is a small red highlighted area near the bottom of the web page. This could be due to the layout or elements on the page loading differently from the baseline, as it's a small banner image that has come into view.

This can happen occasionally and is another reason why it's good to tweak the thresholds at which the tests break and narrow the focus of our image comparison.

layout or elements on the page loading differently

Implementing visual testing for a single element

Leading from our flaky visual test, let’s instead hone down our Playwright visual regression testing to a single element rather than the whole web view. We’ve updated our test to focus on the header logo:

// example.spec.js
const { test, expect } = require('@playwright/test');

test('example test', async ({ page }) => {
await page.goto('');

const headerLogo = page.locator('#entry_217821 > figure > a > img')
await expect(headerLogo).toHaveScreenshot()

Before we perform Playwright visual regression testing, we will need to update our baseline images. We can do that by running this simple command:

npx playwright test --update-snapshots

If we check our snapshots folder, you will see the nice little icon for the LambdaTest Playground website:

LambdaTest Playground website PLaywright visual regression testing

Now, if we run our tests with the usual command, a visual comparison will be made just for the header logo locator snapshot:

npx playwright test

The test results should be three passing tests and an empty test results folder, as no discrepancies between the header logo visual comparison were found. We could break the whole website into smaller components for visual regression and minimize flakiness.

It would also bring the same benefits of breaking down UI tests into smaller chunks, we get more focused test results and outcomes, and finding the area for failure becomes easier.

breaking down UI tests into smaller chunks

Working with Thresholds

With Playwright visual regression testing, as with any type of automation testing, there will always be a margin for error. This is where thresholds come in useful. We can adjust and manage various threshold types depending on our application and use case.

Here are three examples below:

  • maxDiffPixelRatio - this value can be between 0 and 1 and is defined as the acceptable amount of pixels that can differ from the total amount of pixels.
  • maxDiffPixels - this can be any value and is just a count of how many pixels can be different - it's worth experimenting with your test execution and seeing what an acceptable difference is.
  • Threshold - this value can be between 0 (strict) and 1 (lax) and is the acceptable perceived color difference between the same pixel in the compared images - again, this is worth experimenting with and seeing how strict you want it to be.
await expect(headerLogo).toHaveScreenshot({ maxDiffPixelRatio: 0.1 })
await expect(headerLogo).toHaveScreenshot({ maxDiffPixels: 100 })
await expect(headerLogo).toHaveScreenshot({ threshold: 0.1 })

Defining these values in every test can be repetitive if they are generally the same - we have the option to define them in the Playwright config file in this instance. This can be applied globally to all tests or just to a specific project.

module.exports = {
expect: {
toHaveScreenshot: { maxDiffPixels: 100 },

Ignoring sections of the webpage during comparison

Most websites have some form of dynamic content, like a hero carousel, gallery feed, or list of headings - in most cases this could change on a daily or weekly basis, and constantly updating your baseline images isn’t feasible.

The nice solution to this is to use an option called ‘mask,’ which allows us to ignore a locator or group of locators from our comparison. For this Playwright visual regression testing tutorial, let's continue to use the eCommerce website - the main area that stands out to me is the image carousel, which constantly rotates between different products.

Ignoring sections of the webpage during comparison

If we update our test to the following and run:

npx playwright test --update-snapshots
// example.spec.js
const { test, expect } = require('@playwright/test');

test('example test', async ({ page }) => {
await page.goto('');
await expect(page).toHaveScreenshot({mask: [page.locator('.carousel-inner')]})

We will have new baseline images generated which look like this:

new baseline images generated Playwright Visual Regression

The bright pink area is the locator we masked, which will be ignored from all test runs. So let’s perform Playwright visual regression testing with our new baseline.

npx playwright test

And we should have three passing tests - nice one! Now we can perform Playwright visual regression testing against the elements of our website, which we know are static, and any issues raised will be genuine and not the result of content changes.

carousel image page

Implementing full page visual comparisons

Another visual comparison we can perform is a full page - this will capture the full height of the webpage. This can be useful if your website has a lot of scrollable components or content you need to verify.

Remember that as we capture a larger area, the chance of failure may be higher. We can reduce this risk by using a few additional parameters.

  • animations: “disabled” - this will stop any CSS animations or transitions on your webpage.
  • maxDiffPixelRatio: 0.2 - which we covered earlier, will allow some room for minor differences.

This is what the test looks like with the additional parameters added.

// example.spec.js
const { test, expect } = require('@playwright/test');

test('example test', async ({ page }) => {
await page.goto('');
await expect(page).toHaveScreenshot
({ fullPage: true, animations: "disabled", maxDiffPixelRatio: 0.2 });

And if we execute the update screenshots command, we can see how the baseline will look. As you can see, some content and animations have been disabled, but the core components of the website have been rendered. So our visual test will cover quite a lot while remaining stable.

bionic rocket

The test execution will be the same as before, covering Chromium, Firefox, and Webkit for the full page view. If we perform Playwright visual regression testing, we should see a successful output for all 3.

playwright visual regression testing

Running your Playwright suite on cloud with SmartUI

You can power up your Playwright visual regression testing suite by executing your tests on LambdaTest SmartUI cloud, where you can access 3000+ browser versions and various OS combinations with a single test execution from your Continuous Integration (CI) seamlessly.

LambdaTest is a cloud-based testing platform that enables you to perform Playwright automated testing at scale, significantly reducing the time required to run your Playwright tests.

By using LambdaTest, you can execute your Playwright tests on a comprehensive online browser farm consisting of more than 50 different browser versions, including Chrome, Chromium, Microsoft Edge, Mozilla Firefox, and WebKit. This allows you to test your applications on various environments, ensuring maximum coverage and compatibility.

You can also subscribe to the LambdaTest YouTube Channel and stay updated with the latest tutorial around Selenium testing, Cypress E2E testing, Appium, and more.

Drive your scale for your application with continuous integration with LambdaTest SmartUI for your Playwright suite. The following are the feature differences between the local testing and LambdaTest SmartUI.

Local ExecutionLambdaTest SmartUI
Need to maintain multiple browsers in local memory.Test on the latest and legacy browsers and OS versions without the hassle of maintaining them.
Generate results locally and share them manually with the team.Generate results on the cloud and provide quick access to your team.
Hard to maintain the baseline screenshots in local folders.Easy to maintain and update baseline screenshots with just a click on the UI or LT capability.
See results in only the Horizontal view.Explore the compared results with Parallel and Horizontal view with a magnifier experience to view pixel blocks.
Hard to maintain the status of the screenshot.Easy to maintain the status of the screenshots for approving or rejecting the compared results with the Baseline.
No native integration with third-party project management tools.Native integration to create tasks with third-party project management tools like Jira, Slack, etc.
No advanced settings to auto-approve if the mismatch is less than expected.Provides advanced project settings, which allow the user to set custom mismatch accepted % where if the mismatch % is less than the custom mismatch set, SmartUI will auto-approve the screenshot status.
It is hard to maintain the mapping with the browser and resolution combination, making it hard for verification.Verify the mapped browser and resolutions with ease in LambdaTest SmartUI.

To summarize, Playwright visual regression testing can be helpful for your initial POCs or during the development period of an application, website, etc. If you are a team of QA engineers looking to collaborate to build a CI/CD pipeline with Playwright visual regression testing, it is highly recommended to host your tests with LamdbaTest SmartUI for your test suites.

Getting started with Smart UI testing using Playwright

This section of this Playwright visual regression testing tutorial will act as your step-by-step guide to perform Playwright tests with SmartUI.

  • Basic understanding of the Playwright is required.
  • Go to SmartUI and log in along with your credentials.
  • Copy LT_USERNAME and LT_ACCESS_KEY credentials from the Access Key button on the top right of the dashboard.

The following steps will guide you in performing your first Playwright visual regression testing on the LambdaTest platform -

Step 1: Create a SmartUI Project.

The first step is to create a project with the application in which we will combine all your builds to run on the project. To create a SmartUI Project, follow these steps:

  • Go to the Projects page.
  • Click on the New Project button.
  • Add the project's name, approvers for the changes found, and tags for any filter or easy navigation.
  • Click on the Submit button.

Step 2: Configure your test with Playwright Desired Capabilities.

Once you have created a SmartUI Project, you can generate screenshots by running automation scripts. Follow the below steps to generate screenshots successfully.

  • Install the node modules using the command
  • npm i
  • Set up the LambdaTest credentials by following the instructions mentioned in the file.
  • Edit the required capabilities in your test file playwright-smartui.js.
  • Add the following code snippet to run SmartUI with Playwright.
  • const { chromium } = require('playwright')
    const { expect } = require('@playwright/test');
    (async () => {
      const capabilities = {
        'browserName': 'Chrome', // Browsers allowed: 'Chrome', 'MicrosoftEdge', 'pw-chromium', 'pw-firefox' and 'pw-webkit'
        'browserVersion': 'latest',
        'LT:Options': {
          'platform': 'Windows 10',
          'build': 'Playwright SmartUI Build',
          'name': 'Playwright SmartUI Test',
          'user': process.env.LT_USERNAME,
          'accessKey': process.env.LT_ACCESS_KEY,
          'network': true,
          'video': true,
          'console': true,
          'smartUIProjectName': '<projectName>' //Add the required Smart UI Project name
      const browser = await chromium.connect({
        wsEndpoint: wss://'
      const page = await browser.newPage()
      await page.goto('')
      // Add the following command in order to take screenshot in SmartUI
      await page.evaluate((_) => {},
        'lambdatest_action: {"action":"smartui.takeScreenshot","arguments":{"fullPage":true,"screenshotName":"<Your Screenshot Name>"}}') // Add a relevant screenshot name here
      const element = await page.$('[id="sb_form_q"]')
      await element.type('LambdaTest')
      await page.waitForTimeout(1000)
      await page.waitForSelector('[class=" b_active"]')
      const title = await page.title()
      try {
        expect(title).toEqual('LambdaTest - Search')
        // Mark the test as completed or failed
        await page.evaluate(_ => {}, 'lambdatest_action: {"action":"setTestStatus","arguments":{"status":"passed","remark":"Title matched"}}')
      } catch {
        await page.evaluate(_ => {}, 'lambdatest_action: {"action":"setTestStatus","arguments":{"status":"failed","remark":"Title not matched"}}')
      await browser.close()
  • Execute the test using the following command.
  • node playwright-smartui.js

You can check the executed builds over at LambdaTest SmartUI.

If you are a developer seeking to showcase your expertise as a Playwright automation tester, LambdaTest's Playwright 101 certification program is an excellent opportunity. This program is designed for individuals who wish to demonstrate proficiency in performing Playwright end-to-end testing on contemporary web applications. It offers a chance to validate your skills and highlight your expertise in this field.

Playwright certi CTA


Playwright visual regression testing is a powerful layer to add to your quality gates, which can provide extensive test coverage with minimal effort. It can also reduce the number of UI test cases you currently have, which are just checking the element or component displayed. Add visual testing to your next project and see how quickly your confidence grows in the quality of changes being made.


Frequently asked questions

  • General ...
  • LambdaTest related FAQs
What is visual regression testing?
Visual regression testing is a type of software testing that involves comparing screenshots or images of an application's user interface (UI) before and after a change has been made to the code. The purpose of visual regression testing is to detect any unintended changes or regressions in the UI that may have occurred due to the code change.
How do you compare two images in Playwright?
Playwright is a JavaScript library for automating web browsers. To compare two images in Playwright, you can use the built-in screenshot function to take screenshots of two different pages or elements and then use an image comparison library like pixelmatch to compare the two images.

Did you find this page helpful?



Try LambdaTest Now !!

Get 100 minutes of automation test minutes FREE!!

Next-Gen App & Browser Testing Cloud