Run Playwright Automation Tests on HyperExecute Using PyTest
Playwright is a Node.js library that uses a single API to automate Chromium, Firefox, and WebKit. It is designed to enable powerful, reliable, and efficient automated browser testing. Playwright can also automate Microsoft Edge since it is built on the open-source Chromium web framework.
LambdaTest allows you to run Playwright tests across 40+ real browsers and operating system combinations. This guide will cover the basics of getting started with Playwright testing using PyTest on the Lambdatest HyperExecute grid.
HyperExecute has several state of the art features to help you optimize your testing process. Go through the features page to take a look at all the tools that HyperExecute offers.
HyperExecute is compliant with leading security standards - SOC2, GDPR, and CCPA. Refer to HyperExecute Getting Started Guide for more information about features offered by HyperExecute.
All the code samples in this documentation can be found in the Playwright-PyTest HyperExecute GitHub repository. You can either download or clone the repository to run tests on the HyperExecute Grid.
Gitpod
Follow the below steps to run Gitpod button:
- Click the Open in Gitpod button (You will be redirected to Login/Signup page).
- Login with Lambdatest credentials. You will be redirected to HyperExecute dashboard with pop-up confirming to 'Proceed' to Gitpod editor in the new tab and current tab will show hyperexecute dashboard.
Prerequisites for running PyTest Playwright tests on the HyperExecute Grid
Download the HyperExecute CLI:
The HyperExecute CLI is used for interacting with, and running the tests on the HyperExecute Grid. It provides a host of other useful features that accelerate test execution. You need to download the HyperExecute CLI binary corresponding to the platform (or OS) from where the tests are triggered. Shown below is the HyperExecute CLI download location for different platforms:
| Platform | HyperExecute CLI download location |
|---|---|
| Windows | https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe |
| macOS | https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute |
| Linux | https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute |
For detailed information about HyperExecute CLI, please refer to HyperExecute CLI section in the HyperExecute getting started guide.
Configure Environment Variables:
Before the tests are run, please set the environment variables LT_USERNAME & LT_ACCESS_KEY from the terminal. The account details are available on your LambdaTest Profile page. For macOS:
export LT_USERNAME=LT_USERNAME
export LT_ACCESS_KEY=LT_ACCESS_KEY
For Linux:
export LT_USERNAME=LT_USERNAME
export LT_ACCESS_KEY=LT_ACCESS_KEY
For Windows:
set LT_USERNAME=LT_USERNAME
set LT_ACCESS_KEY=LT_ACCESS_KEY
Core
Auto-split YAML file (yaml/win/.hyperexecute_autosplits.yaml) in the repo contains the following configuration:
globalTimeout: 90
testSuiteTimeout: 90
testSuiteStep: 90
Global timeout, testSuite timeout, and testSuite timeout are set to 90 minutes.
The runson key determines the platform (or operating system) on which the tests are executed. Here we have set the target OS as Windows.
runson: win
Auto-split is set to true in the YAML file.
autosplit: true
retryOnFailure is set to true, instructing HyperExecute to retry failed command(s). The retry operation is carried out till the number of retries mentioned in maxRetries are exhausted or the command execution results in a Pass. In addition, the concurrency (i.e. number of parallel sessions) is set to 2.
retryOnFailure: true
maxRetries: 5
concurrency: 2
Pre Steps and Dependency Caching
To leverage the advantage offered by Dependency Caching in HyperExecute, the integrity of requirements.txt is checked using the checksum functionality.
cacheKey: '{{ checksum "requirement.txt" }}'
By default, pip in Python saves the downloaded packages in the cache so that next time, the package download request can be serviced from the cache (rather than re-downloading it again).
The caching advantage offered by pip can be leveraged in HyperExecute, whereby the downloaded packages can be stored (or cached) in a secure server for future executions. The packages available in the cache will only be used if the checksum stage results in a Pass.
The cacheDirectories directive is used for specifying the directory where the packages have to be cached. The mentioned directory will override the default directory where Python packages are usually cached; further information about caching in pip is available here. The packages downloaded using pip will be cached in the directory (or location) mentioned under the cacheDirectories directive.
In our case, the downloaded packages are cached in the CacheDir folder in the project's root directory. The folder is automatically created when the packages mentioned in requirements.txt are downloaded.
cacheDirectories:
- CacheDir
Content under the pre directive is the precondition that will run before the tests are executed on the HyperExecute grid. The --cache-dir option in pip3 is used for specifying the cache directory. It is important to note that downloaded cached packages are securely uploaded to a secure cloud before the execution environment is auto-purged after build completion. Please modify requirements.txt as per the project requirements.
pip3 install -r requirements.txt --cache-dir CacheDir
Post Steps
The post directive contains a list of commands that run as a part of post-test execution. Here, the contents of yaml/win/.hyperexecute_autosplits.yaml are read using the cat command as a part of the post step.
post:
- cat yaml/win/*.*hyperexecute_autosplits.yaml
The testDiscovery directive contains the command that gives details of the mode of execution, along with detailing the command that is used for test execution. Here, we are fetching the list of Python files that would be further executed using the value passed in the testRunnerCommand
testDiscovery:
type: raw
mode: dynamic
command: grep -lr 'def' *.py
testRunnerCommand: pytest $test
Running the above command on the terminal will give a list of Python files that are located in the Project folder:
- testsample.py
- test_app.py
The testRunnerCommand contains the command that is used for triggering the test. The output fetched from the testDiscoverer command acts as an input to the testRunner command.
testRunnerCommand: pytest $test
Test Execution
The CLI option --config is used for providing the custom HyperExecute YAML file (i.e. yaml/win/.hyperexecute_autosplits.yaml for Windows, yaml/linux/.hyperexecute_autosplits.yaml for Linux and yaml/linux/.hyperexecute_autosplits.yaml for Max).
Execute PyTest tests using Autosplit mechanism on Windows platform
Run the following command on the terminal to trigger the tests in Python files with HyperExecute platform set to Windows. The --download-artifacts option is used to inform HyperExecute to download the artifacts for the job.
./hyperexecute --download-artifacts --config --verbose yaml/win/.hyperexecute_autosplits.yaml
Execute PyTest tests using Autosplit mechanism on Linux platform
Run the following command on the terminal to trigger the tests in Python files with HyperExecute platform set to Linux. The -ifacts option is used to inform HyperExecute to download the artifacts for the job.
./hyperexecute --download-artifacts --config --verbose yaml/linux/.hyperexecute_autosplits.yaml
Execute PyTest tests using Autosplit mechanism on Mac platform
Run the following command on the terminal to trigger the tests in Python files with HyperExecute platform set to MAc. The -ifacts option is used to inform HyperExecute to download the artifacts for the job.
./hyperexecute --download-artifacts --config --verbose yaml/mac/.hyperexecute_autosplits.yaml
Matrix Execution with PyTest
Matrix-based test execution is used for running the same tests across different test (or input) combinations. The Matrix directive in HyperExecute YAML file is a key:value pair where value is an array of strings.
Also, the key:value pairs are opaque strings for HyperExecute. For more information about matrix multiplexing, check out the Matrix Getting Started Guide
Core
In the current example, matrix YAML file (yaml/win/.hyperexecute_matrix.yaml) in the repo contains the following configuration:
globalTimeout: 90
testSuiteTimeout: 90
testSuiteStep: 90
Global timeout, testSuite timeout, and testSuite timeout are set to 90 minutes.
The target platform is set to Windows. Please set the [runson] key to [mac] if the tests have to be executed on the macOS platform.
runson: win
Python files in the 'tests' folder contain the test suites run on the HyperExecute grid. In the example, the tests in the files tests/lt_sample_todo.py and tests/lt_selenium_playground.py run in parallel using the specified input combinations.
files: ["test_app.py", "test_sample_.py"]
The testSuites object contains a list of commands (that can be presented in an array). In the current YAML file, commands for executing the tests are put in an array (with a '-' preceding each item). The Python command is used to run tests in .py files. The files are mentioned as an array to the files key that is a part of the matrix.
testSuites:
- pytest $files
Pre Steps and Dependency Caching
Dependency caching is enabled in the YAML file to ensure that the package dependencies are not downloaded in subsequent runs. The first step is to set the Key used to cache directories.
cacheKey: '{{ checksum "requirement.txt" }}'
Set the array of files & directories to be cached. In the example, all the packages will be cached in the CacheDir directory.
cacheDirectories:
- CacheDir
Steps (or commands) that must run before the test execution are listed in the pre run step. In the example, the packages listed in requirements.txt are installed using the pip3 command.
The --cache-dir option is used for specifying the location of the directory used for caching the packages (i.e. CacheDir). It is important to note that downloaded cached packages are securely uploaded to a secure cloud before the execution environment is auto-purged after build completion. Please modify requirements.txt as per the project requirements.
pre:
- pip3 install -r requirements.txt --cache-dir CacheDir
Post Steps
Steps (or commands) that need to run after the test execution are listed in the post step. In the example, we cat the contents of yaml/win/.hyperexecute_matrix.yaml
post:
- cat yaml/win/.hyperexecute_matrix.yaml
Test Execution
The CLI option --config is used for providing the custom HyperExecute YAML file (i.e. yaml/win/.hyperexecute_matrix.yaml or yaml/linux/.hyperexecute_matrix.yaml or yaml/mac/.hyperexecute_matrix.yaml).
Execute PyTest tests using Matrix mechanism on Windows platform
Run the following command on the terminal to trigger the tests in Python files with HyperExecute platform set to Windows. The --download-artifacts option is used to inform HyperExecute to download the artifacts for the job.
./hyperexecute --download-artifacts --config --verbose yaml/win/.hyperexecute_matrix.yaml
Execute PyTest tests using Matrix mechanism on Linux platform
Run the following command on the terminal to trigger the tests in Python files with HyperExecute platform set to Linux. The --download-artifacts option is used to inform HyperExecute to download the artifacts for the job.
./hyperexecute --download-artifacts --config --verbose yaml/linux/.hyperexecute_matrix.yaml
Execute PyTest tests using Matrix mechanism on Mac platform
Run the following command on the terminal to trigger the tests in Python files with HyperExecute platform set to MAC. The --download-artifacts option is used to inform HyperExecute to download the artifacts for the job.
./hyperexecute --download-artifacts --config --verbose yaml/mac/.hyperexecute_matrix.yaml
Secrets Management
In case you want to use any secret keys in the YAML file, you can set it set by clicking on the Secrets button on the dashboard. Now create a secret key that you can use in the HyperExecute YAML file.
All you need to do is create an environment variable that uses the secret key:
env:
PAT: ${{ .secrets.testKey }}
For any query or doubt, please feel free to contact us via 24×7 chat support or you can also drop a mail to support@lambdatest.com.
Happy testing!