Skip to main content

Sharding for Espresso

Running Espresso tests sequentially can be laborious and time-intensive. This guide offers an efficient solution by introducing the concept of sharding. By breaking down tests into shards, they can be executed in parallel, significantly trimming down the total testing duration.

Furthermore, this document provides insights on executing Espresso Tests on real devices using the innovative HyperExecute platform. Designed with precision, HyperExecute stands out as a test orchestration solution, tailored to execute end-to-end tests at breakneck speed. To streamline its setup and configuration, a user-friendly YAML file is employed. Dive into this guide to make your testing process more agile and efficient.

Note

You can learn more about the HyperExecute portal and UI by going through our Guided Walkthrough page. It contains all the relevant information that you need to optimize your testing process with HyperExecute.

Objectives​

By the end of this topic, you will be able to:

  1. Speed up your Espresso tests with sharding
  2. Learn more about HyperExecute
  3. Explore advanced features of LambdaTest

Pre-requisites​

Before you start performing your App automation testing with Espresso, please make sure:

  • You have access to LambdaTest username and accessKey. If you have not registered yet, you can do the same by visiting our website. You will be able to access the credentials at the LambdaTest Profile
  • Make sure you have App Automation Cloud plans on your account.

Uploading your App & Test suite​

Step 1: Upload Your Application​

Upload your android application (.apk file) to the LambdaTest servers using our REST API. You need to provide your Username and AccessKey in the format Username:AccessKey in the cURL command for authentication. Make sure to add the path of the appFile in the cURL request.

Here is an example cURL request to upload your app using our REST API:

Using App File:

curl -u "undefined:undefined" \
--location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' \
--form 'appFile=@"/Users/macuser/Downloads/proverbial.apk"' \
--form 'type="espresso-android"'

Response of above cURL will be a JSON object containing the App URL of the format - lt://APP123456789123456789 and will be used in the last step as appId.

Step 2: Uploading Test Suite​

Upload your test suite (.apk file) to the LambdaTest servers using our REST API. You need to provide your Username and AccessKey in the format Username:AccessKey in the cURL command for authentication. Make sure to add the path of the appFile in the cURL request. Here is an example cURL request to upload your app using our REST API:

Using App File:

curl -u "undefined:undefined" \
--location --request POST 'https://manual-api.lambdatest.com/app/uploadFramework' --form 'appFile=@"/Users/macuser/Downloads/ProverbialEspressoTest.apk"' --form 'type="espresso-android"'

Response of above cURL will be a JSON object containing the App URL of the format - lt://APP123456789123456789 and will be used in the next step as testSuiteAppId

How to use Sharding​

  1. Firstly, create a folder on your local.
  2. Download the HyperExecute CLI file and put it under this folder based on your platform.

Download HyperExecute CLI​

PlatformHyperExecute CLI download location
Windowshttps://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe
macOShttps://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute
Linuxhttps://downloads.lambdatest.com/hyperexecute/linux/hyperexecute

Types of Sharding​

Sharding can be categorized into two types:

Manual Sharding: This feature enables users to manually specify shards along with their definitions, offering multiple strategies such as class, package, and more. By doing so, users can have greater flexibility and control over the sharding process.

Refer to the sample .yaml file here

SampleYamlFile.yaml
version: "0.2"
concurrency: 2
runson: android
autosplit: false
globalTimeout: 180  #MAXQUEUETIMEOUT

framework:
  name: "android/espresso"
  args:
    buildName: "Espresso"
    video: true
    deviceLog: true
    
    # You can use either the appId (lt://APP1234567) or provide the path of the application using appPath. Both examples are given below.

    appPath: Proverbial.apk 

    testSuitePath: ProverbialExpressoTest.apk
    # We have used the appPath and testSuitePath here. 

    
    appId: lt://APP1010461471690377432133206
    testSuiteAppId: lt://APP10104592261690377454846669
    # We have used the appId and testSuiteAppID here.

    deviceSelectionStrategy: all
    devices: ["Galaxy.*", "Pixel.*"]

    shards:
      mappings:
     - name: shard1
        strategy: "class"
        values: ["com.lambdatest.proverbial.BrowserTest"]
    # The strategy for this shard is based on "class".
    # This shard will run all tests from the class com.lambdatest.proverbial.BrowserTest.
     - name: shard2
       strategy: "package"
       values: ["com.lambdatest.proverbial"]
    # The strategy for this shard is based on "package". 
    # This shard will run all tests that belong to the package com.lambdatest.proverbial.l.
     - name: shard3
       strategy: "skipClass"
       values: ["com.lambdatest.proverbial.BrowserTest"]
    # The strategy for this shard is to skip a specific class. 
    # This shard will avoid running tests from the class com.lambdatest.proverbial.BrowserTest.
When shards are added

If you are using the deviceSelectorStrategy: all,then in that case all the specified shards will be executed on every device available.
For example: If there are 3 shards, 2 devices in list (d1, d2) and test suites contains total 10 test cases. Then, d1 has 3 shards, i.e., total 3 devices of d1 configuration will be used. 10 test cases distributed among these 3 shards and similarly d2 also has 3 shards, i.e., total 3 devices of d2 configuration will be used. 10 test cases will be distributed among these 3 shards.

If you are using the deviceSelectorStrategy: any, then in that case all the mentioned shards will be executed on just one device from the provided list.
For example: If there are 2 shards mentioned in .yamland 3 devices mentioned, the system will create 2 shards. These shards might use any 2 devices from the given 3 configurations. It's also possible that the same device configuration could be used for both shards. In this setup, test cases will be distributed between these shards.

Auto Sharding : The system intelligently determines the distribution of tests across devices, employing specific criteria to optimize the testing process.

Refer to the sample .yaml file here

SampleYamlFile.yaml
version: "0.2"
concurrency: 2
runson: android
autosplit: true
globalTimeout: 180  #MAXQUEUETIMEOUT

framework:
  name: "android/espresso"
  args:
    buildName: "Espresso"
    video: true
    deviceLog: true
    
    # You can use either the appId (lt://APP1234567) or provide the path of the application using appPath. Both examples are given below.

    appPath: Proverbial.apk 

    testSuitePath: ProverbialExpressoTest.apk
    # We have used the appPath and testSuitePath here. 

    
    appId: lt://APP1010461471690377432133206
    testSuiteAppId: lt://APP10104592261690377454846669
    # We have used the appId and testSuiteAppID here.

    deviceSelectionStrategy: all
    devices: ["Galaxy.*", "Pixel.*"]

When shards aren't added

If you are using the deviceSelectorStrategy: all, then in that case the tests will be executed on all mentioned devices in .yaml based on the concurrency.
For example: If the concurrency is 3, 2 devices in list (d1, d2) and test suites contains total 10 test cases. Then, d1 has 3 shards, i.e., total 3 devices of d1 configuration will be used. 10 test cases distributed among these 3 shards and similarly d2 also has 3 shards, i.e., total 3 devices of d2 configuration will be used. 10 test cases will be distributed among these 3 shards.

If you are using the deviceSelectorStrategy: any, then in that case all the specified tests will be executed on each device from the provided list, considering the concurrency setting.
For example: If the concurrency is set to 2 and 3 devices mentioned, the system will create 2 shards. These shards might use any 2 devices from the given 3 configurations. It's also possible that the same device configuration could be used for both shards. In this setup, test cases will be distributed between these shards.

  1. Go to the location of the folder on the terminal and run the below commands. After the second command, you may need to give permission on System Settings/Privacy & Security. 
chmod u+x <cliFileNAme>
./<cliFileNAme> --user <userName> --key <accessKey> --verbose -i <yamlFileName>.yaml

You can refer to this example and screenshot below:

./hyperexecute --user my_user_name --key xyx123abc --verbose -i hyperexecute.yaml
cmd
  1. After the test is started you can follow the test on the below links.
note

globalTimeout ​ The globalTimeout value determines the maximum duration (in minutes) of a Job . It can be set between 5 and 180 minutes, and has a default value of 10 minutes.

Filters in Sharding​

You can filter the Classes / Tests that you'd like to execute using filters. Here's an example of the same.This will not work

filters:
      attributes:
      - type: class
        values: ["com.lambdatest.proverbial.BrowserTest","com.lambdatest.proverbial.MainActivityTest"]
      - type: package
        values: ["com.lambdatest.proverbial"]

This example will fetch all the test cases from 2 classes and 1 package as defined above and divide them with respect to concurrency given in the yaml file.

Last updated on