Skip to main content

Run automation tests on HyperExecute using Robot


HyperExecute lets you run E2E (end-to-end) Selenium tests at the fastest possible speed. Unlike normal Selenium grid that involves multiple hops during the process of test execution, speed of execution on HyperExecute Grid is super fast as the code is executed on a secure cloud.

End-to-end encryption of the data (including the source code) ensures that the data is secure whether it is rest or in transit. YAML-based workflow helps in realizing the benefits of optimal test execution and orchestration. Along with the unique features offered by HyperExecute, you also get access to a host of LambdaTest cloud features like detailed logs, Smart CI features, network insights, video recording, access to a range of browsers & platforms on the cloud, amongst others.

HyperExecute has several state of the art features to help you optimize your testing process. Go through the features page to take a look at all the tools that HyperExecute offers.

HyperExecute is compliant with leading security standards - SOC2, GDPR, and CCPA. Refer to HyperExecute Getting Started Guide for more information about features offered by HyperExecute.

All the code samples in this documentation can be found in the Robot HyperExecute GitHub repository. You can either download or clone the repository to run tests on the HyperExecute Grid.

Gitpod


Follow the below steps to run Gitpod button:

  1. Click 'Open in Gitpod' button (You will be redirected to Login/Signup page).
Gitpod popup
  1. Login with Lambdatest credentials. You will be redirected to HyperExecute dashboard with pop-up confirming to 'Proceed' to Gitpod editor in the new tab and current tab will show hyperexecute dashboard.

Run in Gitpod

Getting started with Robot tests on HyperExecute Grid


The HyperExecute CLI is used for triggering tests on HyperExecute Grid. The CLI provides a host of other useful features that accelerate test execution. You should download the HyperExecute CLI binary on the host system for running tests on HyperExecute. Shown below is the HyperExecute CLI download location for different platforms:

PlatformHyperExecute CLI download location
Windowshttps://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe
macOShttps://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute
Linuxhttps://downloads.lambdatest.com/hyperexecute/linux/hyperexecute

For detailed information about HyperExecute CLI, please refer to HyperExecute CLI section in the HyperExecute getting started guide.

The fundamental difference between running Selenium tests on a cloud Selenium Grid and HyperExecute Grid is that you need not have any configurations on the local machine (i.e. the machine from where Concierge is triggered). This is because the source code from the local machine will be zipped and securely uploaded to the cloud where the execution will be performed on the remote Virtual Machine (VM).

Prerequisites for running Robot tests on HyperExecute Grid


Before using HyperExecute, you have to download Concierge CLI corresponding to the host OS. You also need to export the environment variables LT_USERNAME and LT_ACCESS_KEY that are available in the LambdaTest Profile page. Follow the below mentioned steps to set the environment variables LT_USERNAME & LT_ACCESS_KEY from the terminal.

For macOS:

export LT_USERNAME=LT_USERNAME
export LT_ACCESS_KEY=LT_ACCESS_KEY

For Linux:

export LT_USERNAME=LT_USERNAME
export LT_ACCESS_KEY=LT_ACCESS_KEY

For Windows:

set LT_USERNAME=LT_USERNAME
set LT_ACCESS_KEY=LT_ACCESS_KEY

The project requirements (or prerequisites) must be installed as a part of the pre step in HyperExecute. This step remains the same irrespective of whether you are using global or virtual installation of Python packages. Here are the pre steps as mentioned in the sample Matrix and Auto-Split YAML files.

In the sample project, dependency management and packaging is done using the poetry tool. The dependencies available in pyproject.toml can be installed by triggering poetry install command in the pre step of the YAML file. As mentioned in the official robot documentation, Selenium2Library has been renamed to SeleniumLibrary since version 3.0. Hence, robotframework-seleniumlibrary is used instead of robotframework-selenium2library.

pre:
- pip3 install -r requirements.txt --cache-dir pip_cache
- poetry config virtualenvs.path poetry_cache
- poetry install

To reduce the upload size, it is recommended to include files to be added in the ignore list in .hyperexecuteignore file which is located at the root of the project. You can modify the content of .hyperexecuteignore as per your project requirements:

txt
hyperexecute
hyperexecute.exe
*.zip
artifacts
logs

Here is the overall directory structure of the sample code available in the sample Robot HyperExecute repository

robot-selenium-hyperexecute-sample
|
|--- Resources
|
|---- Common.robot (consists of helper functions for opening browser, closing browser, etc.)
|---- LambdaTestStatus.py (consists of helper functions for reading status of test execution)
|--- Tests
|---- lt_todo_app.robot (Test Scenario - 1)
|---- lt_selenium_playground.robot (Test Scenario - 2)
|--- yaml
|---- win/robot_hyperexecute_matrix_sample.yaml
|--- win/robot_hyperexecute_autosplit_sample.yaml
|--- Makefile (consists of Robot build commands that helps run tests across different browser & OS combinations)
|--- pyproject.toml
|--- requirements.txt

Running Robot Tests on HyperExecute Grid using Matrix Execution


Shown below is the HyperExecute YAML file for matrix execution:

---
version: 0.1
globalTimeout: 90
testSuiteTimeout: 90
testSuiteStep: 90

matrix:
files: ["Tests/lt_todo_app.robot", "Tests/lt_selenium_playground.robot"]

runson: win

env:
# PAT: ${{ .secrets.testKey }}
TARGET_OS: Windows 11

cacheKey: '{{ checksum "requirements.txt" }}'
cacheDirectories:
- pip_cache
- poetry_cache
pre:
# Robot Framework and Robot Selenium Library need to be installed globally
# Rest of the packages can be installed in venv
- mkdir pip_cache
- mkdir poetry_cache
- pip3 install -r requirements.txt --cache-dir pip_cache
- poetry config virtualenvs.path poetry_cache
- poetry install
post:
- cat yaml/win/robot_hyperexecute_matrix_sample.yaml

mergeArtifacts: true

uploadArtefacts:
- name: HTML_Reports
path:
- /*.html
- name: XML_Reports
path:
- /*.xml

testSuites:
- make test_windows_10_edge_latest
- make test_windows_10_chrome_latest

Here are the major pointers that you should know for executing Robot tests using matrix execution:

  • Global timeout, test suite timeout, and test suite step timeout : 90 minutes
globalTimeout: 90
testSuiteTimeout: 90
testSuiteStep: 90
  • The target platform is set to win using the runson key
runson: win
  • A matrix of [files] is created for running the Robot tests on the target platform (i.e. win).
matrix:
files: ["Tests/lt_todo_app.robot", "Tests/lt_selenium_playground.robot"]
  • Test dependencies and required packages are installed as a part of the pre step. Global dependencies like Robot Framework, Robot SeleniumLibrary, and Poetry are installed using the pip3 command. Rest of the dependencies are installed using the poetry install command.
pre:
- pip3 install -r requirements.txt --cache-dir pip_cache
- poetry config virtualenvs.path poetry_cache
- poetry install
  • Commands to be run after the test execution are listed in the post step. In the example, we cat the contents of yaml/robot_hyperexecute_matrix_sample.yaml
post:
- cat yaml/win/robot_hyperexecute_matrix_sample.yaml
  • The command(s) mentioned in testSuites key are represented as an array of strings. For demonstration, the matrix of OS and files are run using the robot command.
testSuites:
- make test_windows_10_edge_latest
- make test_windows_10_chrome_latest

Test Execution using Matrix Multiplexing

The CLI option --config is used for providing the custom HyperExecute YAML file (i.e. yaml/win/robot_hyperexecute_matrix_sample.yaml). Run the following command on the terminal to trigger the tests in Python files on the HyperExecute Grid. The --download-artifacts option is used to inform HyperExecute to download the artifacts for the job.

Shown below is HyperExecute CLI command that has to be triggered from the root folder of the project:

./hyperexecute --config yaml/win/robot_hyperexecute_matrix_sample.yaml --download-artifacts --force-clean-artifacts

Visit HyperExecute Automation Dashboard to check the status of execution:

automation-dashboard

Shown below is the execution screenshot when the YAML file is triggered from the terminal:

Robot HyperExecute Terminal LogsRobot HyperExecute Terminal Logs

Running Robot Tests on HyperExecute Grid using Auto-split Execution


Shown below is the HyperExecute YAML file for auto-split execution:

---
version: 0.1
globalTimeout: 90
testSuiteTimeout: 90
testSuiteStep: 90

runson: win

autosplit: true
retryOnFailure: true

maxRetries: 5
concurrency: 2

env:
# PAT: ${{ .secrets.testKey }}
TARGET_OS: Windows 11

cacheKey: '{{ checksum "requirements.txt" }}'
cacheDirectories:
- pip_cache
- poetry_cache
pre:
# Robot Framework and Robot Selenium Library need to be installed globally
# Rest of the packages can be installed in venv
- mkdir pip_cache
- mkdir poetry_cache
- pip3 install -r requirements.txt --cache-dir pip_cache
- poetry config virtualenvs.path poetry_cache
- poetry install
post:
- cat yaml/win/robot_hyperexecute_autosplit_sample.yaml

mergeArtifacts: true

uploadArtefacts:
- name: HTML_Reports
path:
- /*.html
- name: XML_Reports
path:
- /*.xml

testDiscovery:
type: raw
mode: dynamic
command: grep 'test_windows' Makefile | sed 's/\(.*\):/\1 /'

testRunnerCommand: make $test

Here are the major pointers that you should know for executing Robot tests using auto-split execution:

  • The target platform is set to win using the runson key
runson: win
  • Auto-split is set to true in the YAML file
autosplit: true
  • retryOnFailure is set to true to instruct HyperExecute to retry the failed commands. The retry operation is carried out till the number of retries mentioned in maxRetries are exhausted or the command execution results in a Pass.
retryOnFailure: true
maxRetries: 5
  • The concurrency (i.e. number of parallel sessions) is set to 2.
concurrency: 2
  • Test dependencies and required packages are installed as a part of the pre step. Global dependencies like Robot Framework, Robot SeleniumLibrary, and Poetry are installed using the pip3 command. Rest of the dependencies are installed using the poetry install command.
pre:
- pip3 install -r requirements.txt --cache-dir pip_cache
- poetry config virtualenvs.path poetry_cache
- poetry install
  • Commands to be run after the test execution are listed in the post step. In the example, we cat the contents of yaml/robot_hyperexecute_autosplit_sample.yaml
post:
- cat yaml/robot_hyperexecute_autosplit_sample.yaml

The testDiscovery directive contains the command that provides the details of test execution.

testDiscovery:
type: raw
mode: dynamic
command: grep 'test_windows' makefile | sed 's/\(.*\):/\1 /'

testRunnerCommand: make $test

Running the below command on the terminal will give a list of Python files that are located in the Project folder:

grep 'test_windows' makefile | sed 's/\(.*\):/\1 /'
test_windows_10_edge_latest
test_windows_10_chrome_latest

Command to trigger Autosplit execution

Shown below is HyperExecute CLI command that has to be triggered from the root folder of the project:

./hyperexecute --config yaml/win/robot_hyperexecute_autosplit_sample.yaml --download-artifacts --force-clean-artifacts

Visit HyperExecute Automation Dashboard to check the status of execution:

automation-dashboard

Shown below is the execution screenshot when the YAML file is triggered from the terminal:

Robot HyperExecute Terminal LogsRobot HyperExecute Terminal Logs

Pre Steps, Post Steps, Dependency Caching, and Artifacts Management


Here are some of the advanced features (i.e. dependency caching, retries, post, and artifacts management) of HyperExecute that must be leveraged for accelerated package installation and verifying the results of the test execution. All the advanced features are applicable to Matrix and Auto-split modes.

Pre Steps and Dependency Caching

To leverage the advantage offered by Dependency Caching in HyperExecute, the integrity of requirements.txt is checked using checksum functionality

cacheKey: '{{ checksum "requirements.txt" }}'

By default, pip in Python saves the downloaded packages in the cache so that next time, the package download request can be serviced from the cache (rather than re-downloading it again). The caching advantage offered by pip can be leveraged in HyperExecute whereby the downloaded packages can be stored (or cached) in a secure server for future executions. The packages available in the cache will only be used if the checksum stage results in a Pass.

The cacheDirectories directive is used for specifying the directory where the packages have to be cached. The mentioned directory will override the default directory where Python packages are normally cached, further information about Caching in pip is available on pip documentation.

Set the array of files & directories to be cached. The packages installed using pi3 are cached in pip_cache directory and packages installed using poetry install are cached in the poetry_cache directory.

cacheDirectories:
- pip_cache
- poetry_cache

Content under the pre directive is the precondition that will be run before the tests are executed on the HyperExecute grid. In the example, we have used Poetry for handling dependency & packaging of the Python packages required for running the tests.

Poetry, Robot framework (robotframework), and Robot Selenium library (robotframework-seleniumlibrary) are installed by triggering the pip command. All the required packages are also installed in this step using pip3 install. Packages mentioned in pyprojet.toml are installed by triggering poetry install as a part of the pre directive.

pre:
- pip3 install -r requirements.txt --cache-dir pip_cache
- poetry config virtualenvs.path poetry_cache
- poetry install

Post steps

The post directive contains a list of commands that run as a part of post-test execution. In the sample shown below, the contents of yaml/win/robot_hyperexecute_matrix_sample.yaml are read using cat as a part of the post step.

post:
- cat yaml/win/robot_hyperexecute_matrix_sample.yaml

Retries

The retryOnFailure directive when set to true instructs HyperExecute to retry failed command(s). The retry operation is carried out till the number of retries mentioned in maxRetries are exhausted or the command execution results in a pass. Here is the combination of retryOnFailure and maxRetries that can be used for retrying command (or test) execution:

  retryOnFailure: true
maxRetries: 5

Artifacts Management

The mergeArtifacts directive (which is by default false) is set to true for merging the artifacts and combing artifacts generated under each task.

The uploadArtefacts directive informs HyperExecute to upload artifacts [files, reports, etc.] generated after task completion. In the example, path consists of a regex for parsing the directory/file (i.e. report that contains the test reports).

mergeArtifacts: true

uploadArtefacts:
- name: HTML_Reports
path:
- /*.html
- name: XML_Reports
path:
- /*.xml

HyperExecute also facilitates the provision to download the artifacts on your local machine. To download the artifacts, click on Artifacts button corresponding to the associated TestID.

automation-dashboard

You can download the artifacts by clicking on the Download button as shown below:

automation-dashboard

Secrets Management

In case you want to use any secret keys in the YAML file, the same can be set by clicking on the Secrets button the dashboard.

secret management

Now create secrets that you can use in the HyperExecute YAML file.

secret management

Every test run on the HyperExecute Grid has a unique jobId associated with it. Each jobId can in turn constitute single (or multiple) groupId(s). You can visit HyperExecute automation dashboard for checking the status of the test execution.

HyperExecute lets you seamlessly navigate between jobId's and taskId's. The same can be done by navigating to Automation -> HyperExecute logs -> Corresponding jobId on the HyperExecute automation dashboard.

automation-dashboard

The snapshot below shows how to navigate to the respective testID for viewing the Selenium logs:

automation-dashboard

For any query or doubt, please feel free to contact us via 24×7 chat support or you can also drop a mail to support@lambdatest.com.
Happy testing!