Karate Automation on HyperExecute

Karate is a popular open-source test automation framework that combines API test-automation, mocks, performance testing, and even UI testing into a single framework. It allows writing expressive tests in a simple Gherkin syntax, while leveraging Java for extensibility.

Why Choose Karate for API Testing?

Karate is a unified test automation framework that combines API test automation, mocks, performance testing, and UI automation into a single tool. It’s designed for simplicity and power:

Key Benefits of Karate:

• Readable BDD syntax: Write feature files in plain English.
• Built-in HTTP client: No need for external tools.
• Supports parallel execution: Speed up test cycles.
• All-in-one testing: Covers API, UI, mocks, and performance.
• CI-friendly: Works well with Jenkins, GitHub Actions, GitLab, and more.

The Challenge: Scaling Karate Tests in CI/CD

As your project grows, so do your tests. And with scale come challenges:

• Long test execution times (e.g., 30+ minutes for 100+ feature files)
• Inefficient retries, often requiring full suite re-runs
• Hard-to-debug flakiness and poor visibility into trends
• Scattered logs and reports across environments

Why HyperExecute Is Built for Karate Teams

Challenge	Karate Alone	Karate + HyperExecute
Slow suite execution	30+ mins	< 5 mins with autosplitting
Flaky test tracking	Manual logs	Built-in flakiness insights
Parallel test scaling	Custom threads	Seamless, node-based scaling
Debug logs & reports	Scattered	Unified dashboard & rich logs
Retry support	Full suite reruns	Scenario-level retry logic

Example Use Case: Scaling Karate API Tests with HyperExecute

A QA team executes 100+ Karate feature files nightly. Over time, they face:

• 30+ minute run times
• Flaky test behavior hard to trace
• Manual re-runs of the entire test suite on failure

HyperExecute Solution:

1. Blazing Fast Execution via Autosplit

autosplit: true
concurrency: 10

HyperExecute distributes feature files intelligently across defined parallel nodes, bringing execution time down to minutes from hours.

2. Retry Only What Fails

Automatically re-run failed scenarios without restarting the entire suite.

3. Real-Time Logs & Reports**

Debug faster with per-test logs, reports, and console outputs—available instantly in the HyperExecute dashboard.

4. Flakiness & Stability Insights**

Track unstable tests using built-in analytics that detect patterns of failure across builds.

Getting Started with Karate on HyperExecute

Prerequisites

To run the Tests on HyperExecute from your Local System, you are required:

• Your LambdaTest Username and Access key
• HyperExecute YAML file which contains all the necessary instructions.
• HyperExecute CLI in order to initiate a test execution Job .
• Setup the Environmental Variable

Step 1: Download Project

You can use your own project to configure and test it. For demo purposes, we are using the sample repository.

Sample repo

Download or Clone the code sample for the Karate from the LambdaTest GitHub repository to run the tests on the HyperExecute.

View on GitHub

Step 2: Download the CLI in your Project

Download the HyperExecute CLI and copy it into the root folder of the downloaded sample repository.

Platform	HyperExecute CLI
Linux	https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute
Windows	https://downloads.lambdatest.com/hyperexecute/windows/hyperexecute.exe
macOS	https://downloads.lambdatest.com/hyperexecute/darwin/hyperexecute

Step 3: Configyure karate-config.js file

This file defines runtime behaviors and integrates Karate with LambdaTest status reporting.

karate-config.js

function fn() {
  var lambdaHooks = function() {
    if (karate.info.errorMessage) {
      script('lambda-status=failed');
    } else {
      script('lambda-status=passed');
    }
  }

  var env = karate.env || 'dev';
  karate.log('karate.env:', env);

  var config = {
    env: env,
    hub: karate.properties['hub']
  };

  karate.configure('afterScenario', lambdaHooks);

  return config;
}

note

• lambdaHooks sets test status based on execution outcome.
• hub is dynamically picked to support Selenium Grid for UI tests.
• Supports environment-based configuration using karate.env.

Step 4: Create your hyperexecute.yml file

The core of HyperExecute configuration lies in the hyperexecute.yaml file. Let’s break down the structure and understand each section:

1. Environment & Runtime Setup

This section specifies the OS, runtime language, concurrency, and autosplit features:

version: 0.1
runson: linux   # OS to run the tests (e.g., linux, win)
autosplit: true
concurrency: 2   # Defines the number of test sessions to run concurrently

runtime:
  language: java
  version: 11

2. Dependency Resolution with Maven

Before running the actual performance test, ensure all project dependencies are resolved locally for a reproducible build. This step pulls all required Maven dependencies to a local directory (.m2), ensuring environment consistency.

pre:
  - mvn -Dmaven.repo.local=./.m2 dependency:resolve

3. Configure the Test Execution Command

The testRunnerCommand defines how each test is executed on the HyperExecute infrastructure. With autosplit: true, HyperExecute dynamically distributes each test to a separate runner, enabling parallel execution.

testRunnerCommand: mvn test -Dtest=MyApiRunner -DFeaturePath="$test" -Dhub=https://LT_USERNAME:LT_ACCESS_KEY@hub.lambdatest.com/wd/hub -Dmaven.repo.local=./.m2

• -Dtest=MyApiRunner: Specifies the Java class that acts as the Karate runner. It must extend KarateRunner.
• -DFeaturePath="$test": $test is dynamically provided by the discovery command. Each value here is a specific .feature file path.
• -Dhub=...: Optional. Used when your Karate test needs to connect to Selenium Grid for UI tests.
• -Dmaven.repo.local=./.m2: Uses a local Maven repo to avoid repeated dependency downloads on each VM.

info

This command runs one feature file per runner, as decided by autosplit. However, you can change the granularity and run:

• All scenarios tagged with @smoke
• A specific step definition class
• A subset of feature files based on folder/module

mvn test -Dkarate.options="--tags @smoke"

4. Test Discovery Configuration

Test discovery determines which test files or test cases to run, and provides those to the test runner. In HyperExecute, the testDiscovery block parses and lists all .feature files, which are then split and executed.

testDiscovery:
  type: raw
  mode: static
  command: snooper --targetOs=win --featureFilePaths=src/test/java/app --frameWork=java | sed 's/:.*//' | uniq

• snooper: A built-in utility provided by LambdaTest to list relevant test files.
• --targetOs=win: Targets Windows OS runners.
• --featureFilePaths=src/test/java/app: Points to where Karate .feature files are located.
• --frameWork=java: Indicates framework type for parsing.
• sed 's/:.*//' | uniq: Cleans up duplicate or unnecessary output from the snooper tool.

How It Works

Discovery command runs first and lists paths to all .feature files. These paths are saved as individual test cases. HyperExecute passes one path at a time to each test runner via the $test placeholder.

The runner command executes the test against that feature. Each of these will be executed in parallel across the nodes defined under concurrency.

src/test/java/app/login.feature
src/test/java/app/signup.feature
src/test/java/app/payments.feature

Here is a complete working YAML configuration that runs Gatling performance tests on linux runners via HyperExecute:

hyperexecute.yaml

loading...

View on GitHub

📘 For a deeper understanding and project-specific configurations, check out the YAML documentation.

Step 5: Execute your Test Suite

From the project root directory, execute the below CLI command in your terminal:

./hyperexecute --user undefined --key undefined --config hyperexecute.yaml

NOTE : In case of macOS, if you get a permission denied warning while executing CLI, simply run chmod u+x ./hyperexecute to allow permission. In case you get a security popup, allow it from your System Preferences → Security & Privacy → General tab.

Advanced Parameters

Optimize your test pipeline using the following advanced features:

• ✅ Smart Test Splitting – Automatically distribute tests across parallel runners
• ⚡ Fail Fast – Stop test runs on the first failure
• 📊 Detailed Reports – Real-time terminal logs & rich test reports