E2E Headless Browser Testing Using Nightwatch JS

Headless browsers are gaining popularity as a viable option for testing web applications. As we all know, web browsers are an integral part of automation testing using Selenium Webdriver. While performing Selenium automation testing, Selenium launches the corresponding browser defined in the script during the test run and then executes test steps. However, issues like the slow rendering of web pages can be a potential issue that can delay the test execution speed. As a solution, headless browser testing was introduced to speed up test execution time.

Source

In this article, we will talk about headless browser testing along with its advantages and limitations. We will also look into scenarios where headless browser testing should be adopted or avoided. Furthermore, we will also check out to achieve headless testing in one of the most in-demand automation testing frameworks – Nightwatch.js. By the end, you will be able to run your tests successfully in popular browsers in headless mode using the Nightwatch.js framework.

Let’s get started!

What is Headless Browser Testing?

A headless browser is a web browser without a graphical user interface (GUI) in layman’s terms. This program will behave like any other browser, but the only difference is that the GUI will be hidden from the user. The program runs in the backend, and nothing will be presented on the screen. Hence it is called headless – one without head/GUI.

Headless browsers are favored in scenarios where there is no need to render the visuals. For example – they may not be a good choice for surfing but can be a great tool in testing.

Headless Testing, as it implies, means running your browser UI tests against a headless browser. But why would you want to use headless browsers? In the upcoming section, let’s explore the advantages and disadvantages of using headless browser testing.

Advantages and Limitations of Headless Browser Testing

Let us take a detailed look at the major advantages and disadvantages of leveraging headless browsers for your testing needs.

Advantages:

1. Faster automation test execution

   Headless browsers are faster than real browsers. They do not come with the overhead of starting up a browser GUI. all the time taken to load and render HTML, CSS, and JavaScript can be bypassed. This equates to quicker test execution results and up to 15x enhanced performance.

2. Useful in CI pipeline

   This testing methodology is beneficial to run tests on a remote server or CI environment like Jenkins, as there is no one to view the visuals. Also, scaling can be achieved efficiently.

3. Support for multiple browser versions

   The testers would like to test the applications on different browser versions on the same machine. In this case, a headless browser comes handy as most of them support the simulation of different versions of browsers.

4. Useful in Multi-tasking and Parallel testing

   While executing tests in parallel, UI-based browsers consume a lot of memory and resources, which can be drastically reduced using headless browsers. Similarly, they help you to multitask. While the test runs in the background, you can use your machine to do anything else.

5. Beneficial in Data extraction

   If your requirement is just to perform website scraping and data extraction, loading an entire browser instance would be overkill. For such cases, Headless browsers are a perfect choice.

In short, headless testing offers you a more lightweight, less resource-intensive way of automation testing with enhanced performance and quicker execution.

Limitations:

1. Debugging can be a challenge at times

   Due to the faster page loading ability of headless browsers, sometimes it is difficult to debug an issue. Also, it can be challenging in cases where a lot of browser debugging is required.

2. Unable to detect cosmetic bugs

   Cosmetic bugs like the location of web elements, broken images, the color of an element, etc., get missed using headless browser testing.

3. It doesn’t mimic exact real-time user behavior

   Real users don’t use headless browsers. A headless browser is a different browser, and the bugs and errors thrown in them may not always occur in a real browser. For user experience testing on a website with precise feedback, a headless browser is not recommended.

Headless Browser Testing – When to Use and When Not to Use?

Before proceeding further, it is critical to ensure that the test strategy you are adopting is the most suitable solution for your requirement. Let us examine some cases where the headless browser can be adopted and where it should be avoided.

1. When to adopt headless browser testing?
   • Running tests on machines without a display (GUI).
   • Running tests on a headless system (without GUI) like Linux OS.
   • Simulating multiple browsers on a single machine without resource and memory overhead.
   • Performance-oriented test execution.
   • For web scraping and data extraction from websites.
   • Generating PDFs and screenshots of web pages.
   • Layout testing.
   • Monitoring network application performance etc.
2. When to avoid headless browser testing?
   • If there is a need to observe the test execution visually.
   • If you want to perform the test from the end-user experience perspective.
   • If lots of debugging needs to be done.

Read – Top Automation Testing Trends To Look Out In 2021

With this much said, now let’s get into some coding! In the upcoming sections, we will see how to achieve headless testing using the Nightwatch.js framework.

Headless Testing using Nightwatch.js

Nightwatch.js is an open-source automated testing framework that supports end-to-end browser testing for web applications and websites. It is powered by Node.js runtime and uses the W3C WebDriver API (formerly Selenium WebDriver) for interacting with various browsers to perform commands and assertions on DOM elements.

If you are new to Nightwatch.js, check out running your first test with Nightwatch.js. This detailed beginner-friendly article will help you know more about the Nightwatch.js framework which will get you started. It covers the framework installation, setup, and configuration, along with guidance to write your first test script.

In this Nightwatch.js tutorial, we will be creating a test script and making it run in a normal browser.

Step 1: Create a project.

• Create a project folder in your machine.
• Go to the terminal in your IDE, navigate to the project root folder, and initialize the project by giving the command

  npm init --y

• We have successfully generated the package.json file.

Step 2: Install required dependencies.

Step 3: Project setup is done. Now create the folder structure.

• Let us create two folders in our root directory – “tests” for keeping test files and “reports” where reports will get stored after each test run.

Step 4: Time for configurations.

• In the package.json file, replace “scripts – test” with “nightwatch.” This will enable us to run the tests from the project base directory using the command “npm test”.
"scripts": { "test": "nightwatch" },

1 2 3

"scripts": { "test": "nightwatch"   },

• In the root, create the configuration file, “nightwatch.json.” Refer to the following snippet for configuring the settings. We are targeting chrome as the default browser here.
{ "src_folders" : "tests", "output_folder" : "reports", "webdriver" : { "start_process": true, "server_path": "node_modules/chromedriver/lib/chromedriver/chromedriver.exe", "host": "localhost", "port": 4444 }, "test_settings" : { "default" : { "desiredCapabilities" : { "browserName" : "chrome" } } }, "firefox": { "desiredCapabilities": { "browserName": "firefox", "marionette": true, "javascriptEnabled": true, "acceptSslCerts": true } }, "chrome" : { "desiredCapabilities": { "browserName": "chrome", "javascriptEnabled": true, "acceptSslCerts": true } } }

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

{     "src_folders" : "tests",     "output_folder" : "reports",     "webdriver" : {         "start_process": true,         "server_path": "node_modules/chromedriver/lib/chromedriver/chromedriver.exe",         "host": "localhost",         "port": 4444       },       "test_settings" : {         "default" : {             "desiredCapabilities" : {                 "browserName" : "chrome"                                         }                     }       },       "firefox": {       "desiredCapabilities": {         "browserName": "firefox",         "marionette": true,         "javascriptEnabled": true,         "acceptSslCerts": true       }     },     "chrome" : {       "desiredCapabilities": {         "browserName": "chrome",         "javascriptEnabled": true,         "acceptSslCerts": true       }     }      }

Step 5: Let us write one quick test script.

• Inside the “tests” folder, which we created previously, create a test script – testfile1.js (any name of choice).
• The final folder structure will look like this.



• Paste the following code for a simple test scenario.
module.exports = { "Step one: Navigate to google.com and verify URL and title" : function(browser) { browser .url("http://www.google.com") .waitForElementPresent('body', 1000) .assert.urlContains("google") .assert.title("Google") }, "Step two: Search 'Nightwatch' and verify result" : function(browser) { browser .setValue('input[type=text]', ['nightwatch', browser.Keys.ENTER]) .pause(1000) .assert.containsText('#main', 'nightwatch') .end(); } }

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

module.exports = {     "Step one: Navigate to google.com and verify URL and title" : function(browser) {      browser      .url("http://www.google.com")      .waitForElementPresent('body', 1000)      .assert.urlContains("google")      .assert.title("Google")     },          "Step two: Search 'Nightwatch' and verify result" : function(browser) {       browser      .setValue('input[type=text]', ['nightwatch', browser.Keys.ENTER])      .pause(1000)      .assert.containsText('#main', 'nightwatch')      .end();     } }

GITHUB URL
• This is a simple and straightforward test case which:
  • Launches Google.
  • Verifies URL and title.
  • Searches for “nightwatch” and finally.
  • Verifies search result contains “nightwatch.”

Step 6: Execute the code.

npm test

• The test starts running. The chrome browser will be launched, and the script gets executed successfully.



• In the terminal, we can verify that all the assertions are passed.

Our project setup is completed, and we have verified that our test script is running successfully in the normal browser. Now it’s time to go Headless!

Almost all modern browsers provide the capability to run them in headless mode. We’ll see the configurations required to trigger Chrome and Firefox in headless mode in the upcoming sections.

Running Nightwatch E2E Tests with Headless Chrome

Nightwatch.js framework allows you to run your tests in headless Chrome mode using a few quick configuration changes. In this Nightwatch.js tutorial, we will look into running Nightwatch.js tests with headless Chrome.

Step 1: In your nightwatch.json configuration file, add the chromeOptions settings in chrome.

GITHUB URL

Step 2: Now, it is time to run the test and verify the result.

npm test

• Observe that the browser is not getting launched!! The test run status can be verified in the terminal.



• We have successfully run our test cases in Headless Chrome!!

Running Nightwatch E2E Tests with Headless Firefox

In this Nightwatch.js tutorial, we will now look into running Nightwatch.js test scripts with headless Firefox. Let’s examine the configuration changes.

Step 1: In your nightwatch.json configuration file, add the moz:firefoxOptions settings in firefox.

• Also, change the server_path to geckodriver executable path.
"webdriver" : { "start_process": true, "server_path": "node_modules/geckodriver/geckodriver.exe", "host": "localhost", "port": 4444 }

1 2 3 4 5 6

"webdriver" : {         "start_process": true,         "server_path": "node_modules/geckodriver/geckodriver.exe",         "host": "localhost",         "port": 4444       }

GITHUB URL

Step 2: Now, it is time to run the test and verify the result.

npm test

• The script gets executed in headless Firefox mode, and we can verify the run status in the terminal.

Test Execution With Remote Selenium Grid

To stay ahead in a highly competitive market, it is our duty as testers to ensure that our app behaves as expected across all platforms, browsers, and device combinations without crashing. Hence derive the importance of considering test coverage and cross browser testing into our testing approach.

Selenium Grid is always a good call when it comes to distributed test execution. It can run test cases on different versions of browsers and different operating systems simultaneously.

To leverage the test coverage, cloud-based solutions are widely adopted by companies due to various merits like:

• Low investment cost compared to buying the infrastructure.
• Access to hundreds of device and platform combinations.
• Extends supports to old and outdated browser versions and OS devices
• No set-up or installation, or maintenance needs to be done locally.
• Scalability.

Hence, using a combination of Selenium Grid with the right cloud-based solution provider for cross browser compatibility testing is the optimum strategy that you can deploy to assure quality. Let us see how we can perform automated browser testing with headless browsers using a remote Selenium Grid provided by LambdaTest.

LambdaTest is a popular cloud-based platform that provides you access to 2000+ browsers for mobile and desktop to help you gain the maximum test coverage during automated browser testing. Moreover, you can choose from a wide range of Windows and Mac operating systems, along with all legacy and latest browsers.

The good news is that we can seamlessly integrate LambdaTest into our Nightwatch.js framework through a few quick and easy steps. Let’s look into it step by step!

Step 1: Create a free account or log in to your LambdaTest account and fetch your unique username and access key. You can fetch the details from the LambdaTest Profile section.

Step 2: We need to pass browser and environment details to the LambdaTest Selenium Grid via the desired capabilities class. Unlike the local setup, now we have the flexibility to choose the precise settings we want.

LambdaTest provides a helpful tool to generate desired capabilities based on the combination we want. Visit the LambdaTest Desired Capabilities page to generate them.

From the browser-specific capabilities, you can opt for the “Headless” mode.

In this case, the desired capabilities class in nightwatch.json will look like this:

Step 4: Paste the following snippet in your nightwatch.conf.js file. (If not created already, you can create this file in your root).

Here we are declaring the user configuration for an access key, username, host, and port for the test.

Github URL

Step 5: Configure your nightwatch.json file. Since we are using LambdaTest, why don’t we go for cross-browser testing across multiple platforms?

In this code, we are configuring for the script to run across two browsers – Chrome and firefox across two different platforms – Windows and macOS, respectively.

Check out the code below, and let’s look into the required changes and additions to be made on your existing file:

• Pass your Access Token and User details.
• Provide Desired Capabilities for all the browsers.
• Set the Hostname corresponding to LambdaTest.
• Optionally we have also enabled report formats like video recording, console, network logs, etc., to be true.

Do check out the Nightwatch.js official documentation to know more about configuration settings.

Step 6: In package.json, change the “ scripts – test” argument value.

This is important as it is here to specify the browsers where we need to run our test.

Step 7: Execute the test.

The code starts running, and logs can be observed in the terminal.

To observe the live running status, go to your Automation dashboard in LambdaTest.

We can also analyze individual test run reports from Automation Logs.

We have successfully automated and run our test cases using a cross browser testing solution provided by LambdaTest through a hassle-free experience!

Read – Nightwatch.js Tutorial For Test Automation Beginners – With Examples

Conclusion

Source

Headless browsers are gaining popularity as a performance-boosting strategy in Selenium automation testing. Headless browser testing offers you a more lightweight, less resource-intensive automation testing method with enhanced performance and quicker execution. Almost all modern browsers provide the capability to run them in headless mode.

In this article, we did an in-depth exploration of headless browser testing along with its pros and cons. Further, we also checked out how to achieve headless testing in popular browsers like chrome and firefox for Selenium test automation using Nightwatch.js. Additionally, we have leveraged our automation testing quality by adopting test coverage concepts and cross browser testing using a cloud-based remote grid solution provided by LambdaTest.

Hopefully, this Nightwatch.js tutorial will enable you to kickstart your journey in headless browser testing. If you wish to optimize your web application quality by unlocking coverage in a wider variety of platforms or browsers, or versions, feel free to explore the LambdaTest platform.

Happy Testing!