Skip to main content

GitLab PR Checks with SmartUI Exec (SDK)

This guide shows you how to integrate SmartUI visual regression testing with GitLab merge requests using the SmartUI SDK/Exec method. This works for both web testing (Selenium, Playwright, Cypress, Puppeteer) and mobile app testing (Appium, WebdriverIO) across all supported languages.

SmartUI SDK/Exec vs Hooks

This guide covers the SmartUI SDK/Exec approach, where you use npx smartui exec -- <command> to run your tests. This is different from the Hooks approach:

SmartUI SDK/Exec (This Guide):

  • ✅ Uses npx smartui exec -- <your-test-command>
  • ✅ Works with Java SDK and CLI-based projects
  • ✅ Requires SmartUI CLI installation
  • ✅ Suitable for Selenium/Playwright/Cypress/Puppeteer/Appium
  • ✅ Works with web testing (Selenium, Playwright, Cypress, Puppeteer)
  • ✅ Works with mobile app testing (Appium, iOS/Android)

SmartUI Hooks:

  • No npx smartui exec command needed
  • Tests run normally (e.g., npm test, mvn test, pytest)
  • SmartUI integration happens automatically through capabilities
  • See GitLab PR Checks with SmartUI Hooks for Hooks approach

Prerequisites​

Before you begin, ensure you have:

  • LambdaTest account with active subscription
  • GitLab repository with CI/CD enabled
  • SmartUI project created in LambdaTest SmartUI Dashboard
  • Test suite configured (Selenium/Playwright/Cypress/Puppeteer/Appium/WebdriverIO)
  • Test framework configured in your preferred language (TypeScript/JavaScript/Java/Python/Ruby/C#)
  • LambdaTest credentials (LT_USERNAME and LT_ACCESS_KEY)
  • SmartUI CLI installed (or use npx @lambdatest/smartui-cli)

Step 1: Integrate GitLab with LambdaTest​

  1. Go to LambdaTest Integrations page
  2. Search for GitLab and select the integration
  3. Click on OAuth as your preferred authentication method
  4. Click Install and authorize the integration
  5. After successful authentication, refresh the Integrations page to verify GitLab is installed
GitLab integration setup in LambdaTest
Integration Status

You can verify your GitLab integration is active by checking the Integrations page. The GitLab integration should show as "Installed" or "Active".

Step 2: Configure SmartUI Project​

Create or update your SmartUI configuration file (.smartui.json):

.smartui.json
{
  "projectName": "your-smartui-project-name",
  "buildName": "smartui-build-${CI_PIPELINE_ID}",
  "baseline": false
}
Configuration File

The SmartUI configuration file is used to specify project settings. You can also use environment variables or command-line options to override these settings.

Step 3: Configure GitLab CI/CD Pipeline​

Create or update your .gitlab-ci.yml file. The key difference with the Exec method is that you use npx smartui exec --gitURL to run your tests.

Complete GitLab CI/CD Configuration​

.gitlab-ci.yml - TypeScript/JavaScript Example
stages:
  - test

variables:
  NODE_VERSION: "18"
  LT_USERNAME: $LT_USERNAME
  LT_ACCESS_KEY: $LT_ACCESS_KEY
  PROJECT_TOKEN: $PROJECT_TOKEN

visual_regression_tests:
  stage: test
  image: node:${NODE_VERSION}
  
  before_script:
    - npm ci
    - npm install -g @lambdatest/smartui-cli
  
  script:
    # Get GitLab project ID and commit SHA
    - |
      PROJECT_ID=${CI_PROJECT_ID}
      COMMIT_SHA=${CI_COMMIT_SHA}
      
      # For merge requests, use the merge request commit SHA
      if [ -n "$CI_MERGE_REQUEST_IID" ]; then
        COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}}
      fi
      
      # Construct GitLab API URL for status updates
      GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}"
      
      echo "GitLab Project ID: ${PROJECT_ID}"
      echo "Commit SHA: ${COMMIT_SHA}"
      echo "GitLab Status URL: ${GIT_URL}"
      
      # Run tests with SmartUI Exec and GitLab integration
      npx smartui exec --gitURL "${GIT_URL}" -- npm test
      # Or: npx smartui exec --gitURL "${GIT_URL}" -- npx wdio run wdio.conf.ts
      # Or: npx smartui exec --gitURL "${GIT_URL}" -- npm run test:mobile
      
  only:
    - merge_requests
    - main
    - develop
  
  environment:
    name: visual-regression/$CI_COMMIT_REF_NAME

Key Configuration Points​

  1. SmartUI CLI Installation: Install SmartUI CLI globally or use npx @lambdatest/smartui-cli
  2. GitLab Project ID: Automatically available as CI_PROJECT_ID in GitLab CI/CD
  3. Commit SHA: Use CI_COMMIT_SHA for regular commits, or CI_MERGE_REQUEST_SHA for merge requests
  4. GitLab API URL: Construct as https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}
  5. Exec Command: Use npx smartui exec --gitURL "${GIT_URL}" -- <your-test-command>
Understanding the SmartUI Exec Command

The npx smartui exec command wraps your test execution and provides SmartUI integration:

npx smartui exec --gitURL "<gitlab-url>" -- <your-test-command>
  • --gitURL: GitLab API URL for status updates (legacy name, works with GitLab)
  • --: Separator before your test command
  • <your-test-command>: Your normal test command (e.g., npm test, mvn test, pytest)

Step 4: Set Up GitLab CI/CD Variables​

Configure the following variables in your GitLab project:

  1. Go to your GitLab project → Settings → CI/CD → Variables
  2. Add the following variables:
GitLab CI/CD Variables configuration
Variable NameDescription
LT_USERNAMEYour LambdaTest username
LT_ACCESS_KEYYour LambdaTest access key
PROJECT_TOKENYour SmartUI project token (found in SmartUI project settings)
Project Token

The PROJECT_TOKEN is different from LT_USERNAME and LT_ACCESS_KEY. You can find it in your SmartUI project settings in the SmartUI Dashboard.

Step 5: View Pipeline Results in GitLab​

After your pipeline runs, you can view the results in the GitLab Pipelines page:

GitLab Pipelines page showing SmartUI test results

The pipeline will show:

  • Pipeline status (Success/Failed)
  • Job status for SmartUI tests
  • Screenshot statistics (Total, Approved, Changes Found) in the job tooltip

Step 6: View PR Check Results in GitLab Merge Request​

After your pipeline runs, you'll see SmartUI status checks in your GitLab merge request:

GitLab merge request showing SmartUI PR check status

Successful Status​

When all visual tests pass:

  • ✅ Status: Success
  • Details: Click "Details" to view the SmartUI build in the dashboard
  • Screenshot: All screenshots match baseline or are approved

Failed Status​

When visual differences are detected:

  • ❌ Status: Failed
  • Details: Click "Details" to review differences in SmartUI dashboard
  • Action Required: Review and approve/reject changes in SmartUI dashboard

Complete Working Examples​

.gitlab-ci.yml - Complete Web Testing Example
stages:
  - test

variables:
  NODE_VERSION: "18"
  LT_USERNAME: $LT_USERNAME
  LT_ACCESS_KEY: $LT_ACCESS_KEY
  PROJECT_TOKEN: $PROJECT_TOKEN

visual_regression_tests:
  stage: test
  image: node:${NODE_VERSION}
  
  before_script:
    - npm ci
    - npm install -g @lambdatest/smartui-cli
  
  script:
    - |
      PROJECT_ID=${CI_PROJECT_ID}
      COMMIT_SHA=${CI_COMMIT_SHA}
      
      if [ -n "$CI_MERGE_REQUEST_IID" ]; then
        COMMIT_SHA=${CI_MERGE_REQUEST_SHA:-${CI_COMMIT_SHA}}
      fi
      
      GIT_URL="https://gitlab.com/api/v4/projects/${PROJECT_ID}/statuses/${COMMIT_SHA}"
      
      echo "GitLab Status URL: ${GIT_URL}"
      
      # Run web tests with SmartUI Exec
      npx smartui exec --gitURL "${GIT_URL}" -- npm test
      
  only:
    - merge_requests
    - main

Troubleshooting​

Issue: PR Check Not Appearing in GitLab

Symptoms: Pipeline runs but no SmartUI status check appears in merge request.

Solutions:

  1. Verify GitLab integration is active in LambdaTest Integrations
  2. Check that --gitURL parameter is correctly set in the exec command
  3. Verify GitLab API URL format: https://gitlab.com/api/v4/projects/{projectId}/statuses/{commitId}
  4. Ensure CI_PROJECT_ID and CI_COMMIT_SHA are correctly set
  5. For merge requests, use CI_MERGE_REQUEST_SHA instead of CI_COMMIT_SHA
  6. Check pipeline logs to ensure tests completed successfully
  7. Verify SmartUI CLI is installed and accessible

Key Differences: Exec vs Hooks​

AspectSmartUI Exec (This Guide)SmartUI Hooks
CommandUse npx smartui exec --gitURL <url> -- <command>Run tests normally (npm test, mvn test, pytest)
IntegrationRequires CLI wrapperAutomatic via capabilities
SetupInstall SmartUI CLI, configure .smartui.jsonAdd capabilities to test config
GitLab IntegrationUse --gitURL parameter with execAdd github.url capability
LanguagesJava SDK, CLI projects, all frameworksTypeScript/JS/Java/Python/Ruby/C#/WebdriverIO/Appium
Project TokenRequired (PROJECT_TOKEN)Not required (uses LT_USERNAME/LT_ACCESS_KEY)
Server AddressMay need SMARTUI_SERVER_ADDRESS for non-SeleniumNot required

Next Steps​

Last updated on

Test across 3000+ combinations of browsers, real devices & OS.

Book Demo

Help and Support

Related Articles