How To Integrate Cucumber With Jenkins?

Posted by Piyusha | February 11, 2021
CI/CD β€’

21434 Views | 15 Min Read

Cucumber is an open-source BDD (Behavior-Driven Development) testing tool for automation testing of web applications. It remains to be one of the best frameworks for functional and acceptance testing. This is because of its widespread adoption in the open-source community, enhanced natural language support, and the ability to support new programming languages quickly. Almost all organizations that use Selenium prefer to integrate Selenium with Cucumber since Cucumber makes it easy to read and understand the application flow. To schedule test case executions remotely, we can also go for Cucumber with Jenkins integration and utilize the immense benefits of Jenkins.

Getting Started with Cucumber Jenkins Setup

Cucumber is well-known for agile development of web applications and reporting the results. With an easy-to-understand language, powerful plugins, and a straightforward integration with Selenium, starting Cucumber with Jenkins integration has never been easier.

Before we begin, let’s understand why you need Cucumber with Jenkins integration in the first place. Here’s why- Cucumber acts as a bridge between:

  1. Manual Tester and Developers.
  2. Software Engineer and Business Analyst.
  3. Manual Tester and Automation Tester.

And as you already know, Jenkins is an open-source, accessible, easy to use CI/CD tool. With Jenkins, you can schedule a run on scheduled time or trigger builds after an event, and it also comes with a plethora of plugins. You can refer to our comprehensive Jenkins tutorial to learn more about this tool.

Now, let us see how to integrate Cucumber with Jenkins.

Prerequisites for Cucumber with Jenkins Integration

We need the installation of the below plugins to start Cucumber with Jenkins integration. Let us see an overview and utilities of the below plugins. This section will see how to make use of some Jenkins plugins that work together with Cucumber formatting outputs.

  1. Cucumber Test Results plugin
  2. Cucumber Reports
  3. Cucumber perf plugin

A. Cucumber Test Results plugin

This plugin takes Cucumber JSON output as input and publishes the results in polychrome. The results will appear in GREEN when they have passed and RED when they fail.

Here’s how to install the Cucumber Test Results plugin. You can also refer to our blog on generating TestNG reports in Jenkins for more information.

  1. Login to your Jenkins server as an administrator.
  2. Navigate to β€œManage Jenkins.”
  3. Go ahead and click β€œManage Plugins.”
  4. Manage Plugins

  5. Go to the β€œAvailable” tab under the manage plugins page.
  6. Search for β€œcucumber” in the filter at the right top corner.
  7. Go ahead and check the box against the Cucumber reports.
  8. Choose β€œInstall without restart” to start the installation.
  9. Install without restart

    Cucumber Jenkins 3

  10. Click β€œBack to dashboard” in the left menu
  11. Navigate to the job already created. β€˜Cucumber’ in our case.
  12. Click β€œConfigure” against the job.
  13. Cucumber Jenkins

  14. Scroll down to Post-Build Actions. Under β€œAdd post-build action,” select β€œ Cucumber reports.”
  15. Add post-build action

  16. Click on Advanced. Go to Post-build Actions. Add below report file in the text box as shown. You can Ignore the message at the bottom. It appears since we don’t have any files yet, once we run the build, they will be created.
  17. Post-build Actions

  18. The next step is to generate Cucumber JSON reports.
  19. Open the cucumber.yml file in your page-object_framework and add the below-highlighted string.
  20. This is to make sure we have reports.json output in addition to the HTML reports. The cucumber plugin will parse these on Jenkins.
  21. cucumber_yml_json

Execution

To execute, click the β€œBuild Now” button from the left menu for the job. The build should be successful. Instead of watching console output, select “Cucumber Test Result” at the end of the build.

results_output

Results

Next, go to Feature –> Scenario, where you can see all the steps’ status. As shown below, they are all GREEN.

Test Results

B. Cucumber Reports

Cucumber Reports plugin can read the JSON output and produce pie charts for feature or scenario results & bar graphs for tags. This is possible if we had options β€œ-f json -out cucumber.json,” where you output the results to JSON format, i.e., in cucumber.yml. This plugin has advanced options to help us make vital decisions on the test results.

  1. Add a post-build step and select the β€œPublish cucumber results as a report” option, as shown below.
  2. cucumber_reports

  3. Referring to the job, the below menu option will be visible to us as below.
  4. Cucumber Menu Option

  5. Next, run the job and click the reports. The reports will be in the form of SWF charts, as shown below.
  6. cucumber_reports_piecharts_failed

Results

We get results as below.

Test Results

C. Cucumber perf plugin

This plugin helps us demonstrate the history of how Cucumber results were performing as a trend. This is helpful when you keep running the suite multiple times.

There are additional features to it, like β€œworst-performing features” over time. This mainly helps us focus on those bottlenecks and allows us to debug them further.

Integrating Cucumber with Jenkins

Now, you are all set with the prerequisites for Cucumber with Jenkins integration. In this section, we will see how to integrate Cucumber with Jenkins using these plugins. In case you haven’t installed them yet, let us start from the very beginning-

  1. Access Jenkins URL. The default is port 8080.
  2. Go to β€œManage Jenkins” –> β€œManage Plugins.”
  3. Welcome to Jenkins

    Manage Jenkins

  4. Install the following plugins
    • Cucumber Test Results plugin
    • Cucumber Reports
    • Cucumber perf plugin

    Make sure while installing these plugins, all the dependent plugins are also successfully installed. This is to ensure the plugins work as expected without glitches.

  5. SelectPlugin

  6. The next step is to navigate to β€œManage” –> β€œGlobal Tool Configuration.” Under β€œJDK,” set the path for JDK.
  7. JDK Installation

  8. Under β€œMaven,” set the path for Maven.
  9. Maven

  10. Create a β€œNew Item” as a Maven Project as shown below. Enter any arbitrary name.
  11. Create New Job Jenkins

    JobName

  12. Once the job is created, click on the β€œConfigure” link on the left-hand side panel.
  13. Configure Cucumber Project

  14. Scroll down the page. Set the full path of pom.xml under β€œRoot POM” and β€œGoal and options” as β€˜test’ and proceed to save the configuration.
  15. Setting Path in Pom xml

  16. Once all the above steps are completed, click on the β€œBuild Now” button.
  17. Buildnow

  18. The build will be executed, and the corresponding testing.xml file, which is the pom.xml, will get executed.

Hurray, you have successfully completed Cucumber with Jenkins integration! Before we proceed further, this might be the right time to acquaint yourself with the best Cucumber practices.

Running Cucumber Script with Jenkins

Automation scripts built with the Cucumber framework benefit from being easy to understand, easy to communicate, and good maintenance, so it is popular among test automation teams. An essential aspect of integrating DevOps is incorporating automated testing into a continuous integration (CI) process.

Now that you have completed the Cucumber Jenkins setup, in this section, we will see how to configure Jenkins to run an automated Cucumber test script and show the Jenkins UI report. This can be done using the following methods-

  1. Cucumber’s Command Line Mode
  2. CukeTest’s Command Line Mode
  3. Jenkins Continuous Integration Configuration
  4. Jenkins Test Report Plugin Configuration

Cucumber’s Command Line Mode

To illustrate the Cucumber Command Line, we use Cucumber.js. Many who already know this tool know that it means running the “Cucumber-js” command directly in the directory of the automation script. Or if you don’t have Cucumber installed globally, run “node modules/.bin/cucumber-js” (or, if you’re on Windows, the .cmd extension), and the Cucumber installed locally will be used. After execution, the output of the text on the command line is written. You can refer to our Cucumber.js tutorial for more information.

The command line is identical if you are running it as a job under Jenkins. Just add the “format” parameter, indicating the output file in the “JSON” format, to make Jenkins render a visual report. The following is the command line:

A report is output to report.json, and then this file is used by the Jenkins job to view the report.

CukeTest’s Command Line Mode

CukeTest is Cucumber.js’s development tool. These editions support command line mode and are available on Windows Desktop, Windows Store, MacOS, etc. It is available for free at CukeTest’s official website. CukeTest is used to build scripts for Cucumber.js and can also be used to execute scripts for Cucumber.js from the command line.

The benefit of using CukeTest is that with each project, you don’t have to install Cucumber.js. CukeTest already has Cucumber.js built-in, which saves your storage space and time. Another advantage is that it can run Windows automation with the library built-in, capture video, etc.

The Windows Store version of CukeTest has a different command line process, so the command line varies slightly from the desktop version. Luckily, you don’t need to recall all the specifics of the command line.

  1. In CukeTest, you can open a run profile when your project is opened to get the command line by clicking the drop-down arrow next to the β€œRun Project” button and then clicking β€œEdit Profiles.”
  2. CukeTest

  3. In the β€œRun Profiles” dialogue, the corresponding command line is automatically created for this profile in the command line:
  4. Run Profiles

  5. Click the β€œCopy to clipboard” option to copy the commands to the clipboard.
  6. Then click the β€œOpen console window” option, and paste the command line to this newly opened window.

Below are the command lines for various platforms-

  1. Command line for Windows Store version
  2. CukeTest’s command line output is encoded in UFT8. So using the command “chcp 65001” helps change the command line encoding from UFT8 to UTF8.

  3. CukeTest desktop version

    CukeTest’s command line output is encoded in UFT8. So using the command “chcp 65001” helps change the command line encoding from UFT8 to UTF8.

Jenkins Continuous Integration Configuration

Once we are aware of the steps to run the project in the command line, we can configure Jenkins to run it. Below are the steps to configure the Jenkins job.

For the demo, the source code is kept in https://github.com/CnodejsTest. This project is used to automate test cnodejs.org community site. Alternatively, we can use our Cucumber project to configure and run the job.

  1. Create a new job in Jenkins. Enter any arbitrary name.
  2. Create new job in Jenkins

  3. In the item bar, select “Windows Batch Command.” Here we have to copy & paste the corresponding command line from CukeTest to the Jenkins batch command box. In case you have a CukeTest desktop version installed natively, below is your command line.

    The β€˜npm install’ command installs npm dependency packages that are consumed by the automation script.

  4. Perform the below job configuration UI with the commands as entered below.
  5. npm start

  6. Save this project configuration by clicking β€˜Save.’ Now click the β€œBuild Now” button, and the project will then start executing.

    You can observe the execution progress upon build now of the job in the build history.

    We can also find the running job log under the Build History. At the exact moment, the report file is also generated in the Jenkins Workspace directory.

  7. The next step is to configure the Jenkins job to show the Jenkins interface test report. This can be done quickly using the Plugin we just saw in the previous section to format and display the test report.

Jenkins Test Report Plugin Configuration

In Jenkins, we can use the Cucumber reports plugin to show the report. We have already seen the introduction to this plugin in the Cucumber Jenkins setup above. Additionally, you can also read the information about this Plugin available at the official Jenkins website:

The following are the steps to configure Jenkins to generate reports.

  1. Install the Jenkins reports plugin
    Open Jenkins –> Manage Plugin –> Filter “Cucumber reports” in Optional Plugin.
  2. Install the Jenkins reports plugin

    We need to restart Jenkins after the installation of this plugin.

  3. Reconfigure the Jenkins task

    We need JSON log data files to generate reports because the Cucumber reports Plugin generates reports by parsing the JSON report file created by Cucumber or CukeTest. Therefore, it is necessary that we first configure a running project which will create JSON log data files.

    In the run configuration file, we have to select “JSON” in the report format.

  4. JSON

    When we would run this command, the command line execution output is changed to json.

  5. Display the Cucumber reports
    In the β€˜Add post-build operation’ action, select the β€˜Cucumber reports’ option.
  6. Cucumber reports

    Go to the β€˜Advanced’ option under the β€˜Cucumber reports.’ In the β€˜JSON Reports Path,’ enter “reports.”

    JSON report location

    Click Save to finish the project configuration.

  7. Generating test report
    To generate the test report, we have to click the “Build Now” button. After the job is completed, a test report will be generated automatically. Besides, the test report provides us with a link to each scenario, where each link, when clicked, shows the details of the scenarios. The report also indicates the screenshot captured in each scenario step or hooks along with each step’s execution result.

Generating test report

So till now, what we have seen is how to integrate Cucumber with Jenkins and how to run a Cucumber script on Jenkins along with the generation of graphical reports.

It is also possible to leverage cloud-based platforms’ automation capability to make remote testing more seamless than ever. You can configure your Cucumber scripts to run on LambdaTest’s Selenium Grid cloud and perform testing on 2000+ browsers, devices & operating systems. In the next section, we will learn how you can make the most of Cucumber with Jenkins integration & LambdaTest.

Leveraging Cucumber Jenkins Setup with LambdaTest Cloud Selenium Grid

Below we would see how we can get our Cucumber scripts running on the LambdaTest platform, and further, we can use the same steps as defined above to integrate Jenkins to generate the reports.

Few prerequisites before we start running the scripts:

  1. It is assumed that Java is installed on the machine. Preferably JDK 8 and above.
  2. Also, it is assumed we have node.js installed. If not, you can easily install it from https://nodejs.org/en/.
  3. Your LambdaTest credentials should be set as environment variables.
    • If you are a Windows user, set them as
      set LT_USERNAME=”YOUR_USERNAME”
      set LT_ACCESS_KEY=”YOUR ACCESS KEY”
    • If you are a Mac user, set them as
      export LT_USERNAME=”YOUR_USERNAME”
      export LT_ACCESS_KEY=”YOUR ACCESS KEY”
    • In case you are unaware of the access key, you can easily retrieve them from your LambdaTest dashboard.

      LT dashboard

    • We also have to update .conf.js files inside the conf/ directory with these credentials.

After we have the environment setup aligned, perform the below steps to run the Cucumber scripts.
Make sure we have our repository cloned from Git.

  • In case it is the first time you are running a script for a repository hosted on Git, you can clone a sample repository provided by LambdaTest, which is NodeJs-Cucumber-Selenium from GitHub.
  • Navigate to the cloned directory and install the project dependencies using the below command.

    $ npm install

  • Navigate to karma-jasmine-sample and run $ npm run single to run the test in single or $ npm run parallel to run the test in parallel.

Both the commands validate the test cases and execute the test suite across all the defined test groups. We can check the test results by opening the LambdaTest platform and navigating to the Automation dashboard.

The below sample snippet depicts the same.

sample-screenshot

Conclusion

This blog presents how a Cucumber with Jenkins integration can help run automation scripts successfully. Additionally, this Cucumber with Jenkins integration can also be leveraged to run scripts over the LambdaTest cloud platform. The best part is that project stakeholders can all read the test reports after the Cucumber Jenkins setup and understand the quality status. Jenkins also has many other features, such as setting up timed service, sending mail, and other functions. You can configure to further customize report notification according to project needs, such as sending HTML reports to key stakeholders as an email.

Do leave your comments if you found this blog helpful. If you have any questions or doubts, you can always shoot up your query on LambdaTest Community and get it resolved!

Happy testing!

Frequently Asked Questions

How do I run a Maven Cucumber project in Jenkins?

Login to your Jenkins server as an administrator. Make sure that you have installed the following plugins:

  • Cucumber Test Results plugin.
  • Cucumber Reports.
  • Cucumber perf plugin.

Create a new item as a Maven Project. Set the path for JDK. Set the path for Maven. Create a β€œNew Item” as a Maven Project as shown below. Enter any arbitrary name, and you have your own Maven Project!

How do I add a Cucumber report in Jenkins?

This can be done using the Cucumber reports plugin. Run Jenkins and install the plugin by going to β€˜Manage Plugins.’ Now, you will be able to create Cucumber reports using the β€œPublish cucumber results as a report” option.

Which file is used for getting Cucumber reports in Jenkins?

Cucumber Reports plugin can read the JSON output and produce pie charts for feature or scenario results & bar graphs for tags. This is possible if we had options β€œ-f json -out cucumber.json,” where you output the results to JSON format, i.e., in cucumber.yml.

Written by

Related Articles

How To Generate TestNG Reports In Jenkins?

32786 Views | 16 Min Min Read

build-cicd-pipeline-in-azure-devops

How To Build a CI/CD Pipeline In Azure DevOps ?

32880 Views | 16 Min Min Read

Leave a Reply

Your email address will not be published. Required fields are marked *