Continuous Test Orchestration And Execution Platform Online

Perform automated and live-interactive testing on 3000+ real desktop and mobile devices online.

How To Automate Using TestNG In Selenium? [TestNG Tutorial]

Vipul Gupta

Posted On: February 28, 2022

view count363963 Views

Read time27 Min Read

Automation testing is a fast-growing industry, and every tester tends to opt for tools and frameworks that are self-sufficient and offer useful features out of the box. Though there are a number of test automation frameworks like Selenium, Cypress, etc; I still prefer using Selenium.

The reason for choosing Selenium is very simple – Selenium supports the Java programming language.

The features offered by Selenium multiply when it is used in conjunction with Java-based test automation frameworks like TestNG, JUnit, etc. An ideal framework in such a case would be robust enough and complement the advantages of Selenium WebDriver & Java.

Of all the candidates available to serve these needs, the one that stands out is TestNG. TestNG is not only easy to implement and learn but also supports reporting of execution and logs. It makes parallel testing such an easy task! Ask any Selenium TestNG user to validate this claim, a majority of them will answer in affirmation. As per the State of open source testing survey, TestNG is one of the best Java testing frameworks preferred by developers. Selenium is the only test automation framework that is ahead of TestNG. Close to 50% of the organizations prefer the TestNG framework.

Testing Frameworks


TestNG, where NG stands for Next Generation, is a testing framework developed by Cedric Beust. In this Selenium TestNG tutorial, I will deep dive into how to install TestNG and cover the essentials on writing your first automation script with TestNG for Selenium automation testing.

What is TestNG?

TestNG is an open-source advanced testing framework built on the capabilities of JUnit and NUnit. The framework is not only robust but also provides awesome features like TestNG annotations, grouping, parameterizations, etc., that help keep tests well-structured and easy to maintain.

At the time of writing this blog, the latest version of TestNG is 7.5. However, the preferred version is 6.14.3. I will be using the said version for demonstrating the features of TestNG Selenium in this Selenium TestNG tutorial.

The TestNG framework is designed to simplify a broad range of testing needs, from unit testing (testing a class in isolation of the others) to integration testing (testing entire systems made of several classes, several packages and even several external frameworks, such as application servers).

Salient features of TestNG:

  • Provides HTML Reports in an efficient easy to read format, thus catering to WebDriver’s limitation to generate such reports
  • Support to Group multiple test cases to be executed together whenever that particular group is invoked
  • Ability to set Priority among test cases which help to decide which case has to be executed first
  • Provides the ability to execute only the failed cases with the help of testng-failed.xml
  • Support for Cross Browser testing, Data Parameterization, and Parallel testing
  • Easy to understand Annotations, which helps control the sequence of execution in automation script without a static main method
  • Handles Uncaught Exceptions inherently, thus preventing sudden test termination.

Read – Parameterization in TestNG

Installation and Setup of TestNG in Selenium

Having understood the features of TestNG, the next would be to know how to set it up on the system before I deep dive into the practical demonstration. In this tutorial on TestNG in Selenium, I will cover the installation process for 2 of the most used IDEs -Eclipse and IntelliJ.

How to install TestNG in Eclipse

To install TestNG in Eclipse, download and install Eclipse IDE on your system.

Step 1: Launch Eclipse, click on Help, and select Eclipse Marketplace.

Selenium & TestNG

Step 2: Inside the Eclipse Marketplace window, go to the Search tab and find TestNG. If it is already installed, you will see the Installed button, and no need to continue with these steps; otherwise, it would say Install. Click Install in such a case.

TestNG with Selenium

Step 3: On the consecutive window, make sure the TestNG checkbox is checked and click Confirm.

Install TestNG

Step 4: It is recommended to restart Eclipse after the installation is completed for changes to take effect correctly.

Step 5: After restarting, verify the installation by right-clicking on any project and checking if the TestNG menu is visible to create a TestNG class or not.

Installing TestNG

How to install TestNG in IntelliJ

In this section of the blog on TestNG in Selenium, I will cover how to install TestNG in IntelliJ. For IntelliJ, we will run tests on Java where we will download the JAR from the Maven Repositories website and add it.

But before proceeding to add the JAR, make sure you already have IntelliJ installed on the system. If not, you can download IntelliJ by navigating to the JetBrains website.

Step 1: Visit Maven Repositories, search for TestNG and select the latest most used version (i.e. 7.1.0). In this tutorial, I will be using TestNG version 6.14.3. Hence, the POM file that I will be using for demonstration will be having traces of TestNG 6.14.3.

TestNG in IntelliJ

TestNG with IntelliJ

Step 2: Select and download the jar file. We will be adding this jar to IntelliJ.

TestNG & IntelliJ

Step 3: Launch IntelliJ, open the File menu, and select Project Structure.

Using TestNG & IntelliJ

Step 4: Open the Modules panel, go to the Dependencies tab, click on + sign, and select JARs or directories.

Use TestNG & IntelliJ

Step 5: Navigate to the path where we have the downloaded jar from previous steps and select it.

Step 6: It would appear on the list of dependencies. Just check it and click the Apply button followed by the OK button.

IntelliJ & TestNG

And with this, we are done with installation on both IntelliJ and Eclipse.

Project Setup in Eclipse

For this TestNG in Selenium tutorial, we would be going ahead with Eclipse IDE. In this section, we will learn how to create a TestNG project in Eclipse. However, if you are more comfortable using IntelliJ, then please go ahead with it. The steps mentioned would be the same irrespective of the IDE used. So let us create the project structure and add a test class to verify if it is being executed as TestNG Run or not.

In this entire tutorial, we will be using the same project structure as discussed above and add a new class file each time we want to practice something by executing and analyzing the output.

  • Create a Maven project in Eclipse and name it testng_selenium.
  • Update the pom.xml to have Selenium and TestNG dependencies.

POM, short for Project Object model, is an XML file that makes the basic fundamental unit of Maven. It stores the information about the project like version, description, etc., and also contains configuration details like dependencies, which Maven uses to build the project and make our scripts work.


Step 1: Create a package and name it test. This package will contain all our test class files.

TestNG Setup in Eclipse

Step 2: Inside this package, add a new class VerifySetup, add the following code, and save it.

Step 3: Now execute this as a TestNG test to verify the setup. For this select the class name, right-click on it, select Run As and then click on TestNG Test.

Setup in Eclipse for TestNG

Upon successful execution, the output should look like the below. This verifies our TestNG setup and the same step can be done to execute more classes that we add in this tutorial.

Eclipse & TestNG

TestNG Annotations and Attributes

Annotation means a note or a comment in any document that is used to provide it certain meaning. TestNG annotations also serve a similar purpose. These are used to provide meaning to any function in the test script and describe its behavior.

Annotations in TestNG are some predefined keywords that help to control the execution of the automation script in a proper flow. One must note, any test which is not annotated in TestNG is ignored while running tests. So it is mandatory to add required annotations for the test cases. Here is one example to understand this.

As you can see, in the above example, we have added two test cases, one with @Test annotation and one without it. So the expectation here is only the one with annotation is executed and another one is skipped. Let’s execute the class and verify this.

TestNG Annotations

It is clear from the output, that only the case with annotation got executed. One more thing to note here is that the Test run on the output also shows the total number of test cases as 1 since TestNG reads the number of test cases from @Test annotation and we had only one such case in the file.

TestNG annotations also provide relevant information about the method, class, or test suite they are associated with. These are represented by adding @ prefix. Let us have a look at all these annotations one by one, which we will use in this Selenium TestNG tutorial.

Types of Annotations

TestNG currently supports ten types of annotations.


This is the most important annotation as it covers the actual test case logic and executes the automation run. There are some attributes associated with this annotation that helps to serve various use cases. We will learn these attributes in the following sections.


This annotation is executed before running any test with the @Test annotation in a class. One example of using this can be to maximize the browser for test cases in a class.


This annotation is executed after all the test cases in a class are executed. One use case of this annotation would be to write the method to compile a test run report after all tests are run.


This annotation is executed before any and every test method, i.e., before every @Test annotation method in the class. For example, we have cases to test some dashboard that comes up after login. So instead of writing login steps in each test case, @BeforeMethod annotation can be used to write those steps once in a method to be called before every test case.


This annotation is executed after any and every test method, i.e., after every @Test annotation method in the class. In continuation to the above example, let’s say we need to log out after every test case. In such a scenario, this annotation would help.


This annotation is executed before the first test method with @Test annotation in the class is executed. For example, this can be used to navigate to the URL before executing the test.


This annotation is executed after the last @Test method is executed. The method with this annotation can be used to close the driver after the automation run.


This annotation marks the entry point of any automation script execution on TestNG in Selenium. Any method with this annotation is executed before any test in all the classes in that suite. This can be used to do generic setup steps for the execution, like initializing WebDriver for execution.


This annotation in TestNG is the last to be executed after all test methods of all the classes within the suite have been executed. A method with this annotation is mostly used for cleanup activities like closing the active driver sessions.


TestNG provides the capability to group the tests with similar functionalities in a group with the help of the group attribute of @Test annotation. This helps when we want to execute only a particular type of test case. The method with @BeforeGroup annotation is executed before the first test case for the given group is executed. We can mention the group name inside brackets next to the annotation.


This annotation in TestNG is executed after all the test cases of a given group are executed.

With this, we have understood all the annotations used in TestNG. These annotations are highly usable as:

  • Very easy to learn as there are no rules on which annotation to use with which method. The tester can use them as per free will and understanding.
  • Ease of parametrization, which helps in creating groups and other tasks.
  • Strongly typed, which saves time by highlighting errors while implementing only.
  • Saves the time and effort to extend classes to define the order of execution, unlike JUnit

Hierarchy of Annotations

TestNG annotations have a predefined hierarchy of execution which helps in executing the source code. These are executed in the below order :

Let us have a look at the following code example to demonstrate the hierarchy better.

Upon execution, this is how the output for this code snippet would be where all the methods get executed as per the hierarchy of the annotations.

TestNG Annotations Hierarchy

Looking at this code example and the output, it is evident that TestNG religiously follows the order of execution of annotations.

One might wonder when you see this example is that since every annotation has a particular sequence defined, then how do the two @Test methods in this example decide which one to execute when. This is something that is handled using test case priority in Selenium TestNG. The same will be covered in the upcoming sections of the blog.

TestNG Attributes

Just like a method in Java, TestNG annotations have attributes that help in making our tests more defined and provide more information about the test script while performing Selenium automation testing with TestNG. These attributes are:

  • description: It helps to define the purpose of the test
  • groups: This attribute helps to group test cases of the same functionality under a common group by giving it a group name. It is very useful when the user wants to execute cases of a particular group or wants to exclude cases of a particular group from execution.
  • dependsOnMethods: The test having this attribute is only executed if the dependent test method passes. If it fails or is not executed, then the test with this attribute is skipped from execution.
  • alwaysRun: It ensures that this test method always runs and is not dependent on the result of the methods that it depends upon.
  • dataProvider: This attribute is used when we want to run a test case on a particular data set (e.g. running the same test case on different browsers). This attribute provides the data to the test case for which it is used with the help of another method annotated with @DataProvider annotation of TestNG.
    We will learn more about DataProviders in a further section of this blog.
  • enabled: This attribute is helpful when we want to skip executing a particular test case. It helps to do so by setting the value to false.
  • timeOut: This attribute is used to define the max execution time for a test case. In other words, the test case should be completed in the given time, which is given in milliseconds, else the test execution would be terminated, and the case would be marked as failed with timeout exception (org.testng.internal.thread.ThreadTimeoutException)
  • invocationCount: This attribute is used when we want to execute the test case in a loop for a given number of times.
    Would execute the test case 3 times
  • invocationTimeOut: This attribute is used in combination with the invocationCount attribute. This attribute defines the timeframe within which the test should execute for a given number of times.
    Means execute the case 3 times in 20 milliseconds.
  • expectedExceptions: This attribute helps to handle the exceptions that the test method is expected to throw. If the test method throws the exception that is set on the attribute, then the test case passes, and for any other unhandled exception, the test method fails.
  • priority: This attribute is used to define the order of execution by assigning priorities to the test cases. A test method with lower priority is always executed first.
    In this case, the test with priority 1 is executed first, followed by the test with priority 2. Move to the next section to know more about priorities.

Consider the below code example to understand attributes implementation.

Running the above code would give the following output:

TestNG Attributes Example

TestNG Priorities

As mentioned before in this TestNG in Selenium tutorial, test case execution in TestNG is governed by the priority of test cases. But what if no priority is assigned to test cases. Then what decides the order of execution.

If there are multiple test cases, then TestNG executes them in alphabetical order. So for the following code, aTest is executed before bTest.

Now, let us assume a scenario where we want to execute bTest first. In such scenarios, a priority attribute is used. We can achieve this by assigning lower priority to the test we want to execute first. Priorities in TestNG can be assigned starting from 0, where lower priority cases are executed first.

In this case, bTest will be executed before aTest.

The next question that comes up is, what if the same priority is assigned to both test cases.

In such a case, TestNG runs the cases in alphabetical order, i.e., aTest will be executed before bTest.

Another scenario could be that the test class file is a combination of test cases with and without priority like below

In such a case, TestNG, by default, assigns a priority = 0 to such cases that do not have the attribute. Those are executed first in alphabetical order, followed by other cases as per priorities logic. So the output for the above code would be

Example of TestNG Properties

This is all about priorities, and these would be highly beneficial to use in any automation script while working with TestNG in Selenium. To learn more about the test case priority, you can go through our earlier blog on how to set test case priority in TestNG with Selenium.

DataProviders in TestNG

DataProviders in TestNG is a part of the inbuilt TestNG data-driven testing approach and is used to pass different values to the same TestNG test class through @DataProvider annotation.

TestNG DataProvider sample:

This annotation consists of only one attribute- name and it is always of type String. If the name is not provided then the name of the method becomes the data provider name by default, dataProviderMethod in this case.

DataProvider always returns a 2-D list of objects, on which the test method performs the execution using a data-driven approach on each passed argument.

Let’s take an example of a simple cross browser code where we want to navigate to a website on both Chrome and Firefox browsers.

In the above code, we have implemented DataProvider having a name as browserName, which passes 2 values chrome and firefox. These values are passed to the test method crossBrowserTest using the parameter value and the test case is executed one by one on both browsers.

This is a classic example of cross-browser testing code implementation where we want to execute the same script on multiple browsers.

Executing the above code would give an output like:

DataProvider Example of TestNG

One thing to note here is that instead of having only 1 @Test annotated test case, the execution shows two test cases are executed. This is because we passed 2 different values to the test case and hence TestNG treats it as 2 separate test cases since the test data is different.

Also readParameterization In TestNG – DataProvider and TestNG XML (With Examples)

TestNG Assertions

Assertions provide the means to verify if the actual and expected results of the test case execution are a match or not thus deciding the outcome of the test case whether it is a pass or fail. These are widely used to validate the results in any script of TestNG in Selenium.

An example of TestNG Assertions would be to validate the username on the dashboard after a user logs in to a website.

The syntax for TestNG Assert:

  • actual: the actual value which is fetched from the test case as a part of the execution.
  • expected: the expected value with which we want to compare the actual value.
  • message: string message to be displayed in case of assert failure.

Types of TestNG Asserts

Hard Assert: These are the default type of asserts in TestNG. These assert stop the test case execution when any assert fails and subsequent statements are not executed/validated. These are used in cases when subsequent steps in the test cases are not to be executed when a particular step assertion fails.

Soft Assert: These are the opposite of Hard Asserts. In the case of soft assert, the test case execution of subsequent steps continues even if some assert fails in between. These are used when we want to continue the execution and see the results for all steps even if a few fails in between. To use soft asserts, org.testng.asserts.SoftAssert package needs to be included.

The most commonly used TestNG asserts are:

This assertion verifies whether the given condition is true. If false, it will fail the test case.

This assertion verifies whether the given condition is false. If true, it will fail the test case.

This assertion verifies if the actual and expected are a match. If they match, it passes the test case otherwise fails.

This assertion verifies if the actual and expected are not a match. If they match, it fails the test case otherwise passes.

ReadAssertions in TestNG with Selenium

TestNG Selenium Demonstration

By now, this TestNG in Selenium tutorial has covered almost all the main features of the TestNG framework that you would need to write your first Java automation script. In the following code example, we will combine all the annotations, attributes, priorities, data providers, and assertions that we have learned so far.

The following test scenario will be executed by this code (To print and verify the sum of 2 numbers)

  1. Create a dataProvider that passes values as firstValue, secondValue, expectedSum.
  2. Navigate to
  3. Run Test 1 with the first set of values, {1, 4, 5}. This test should pass as the expected sum is correct.
  4. Clear the old values from the form.
  5. Run Test 2 with the second set of values, {2, 4, 7}. This test should fail as the expected sum is not correct. We are intentionally passing incorrect expected values to see the differences in output and test reports for passed and failed cases. Always remember that ideally the expected should always be the correct value. This is done just for the tutorial purpose here.
  6. Execute the code and analyze the reports.

In addition to this test case, we will add two more test cases with priority to showcase the usage, along with one of them having invocationCount attribute to show case getting executed multiple times as per priority.

You can see that this code uses a number of annotations, dataProviders, assertions, priorities and attributes which we have learnt and gives an idea of how these can be used together while working with automation scripts on TestNG in Selenium to use the features at full extent.

In this TestNG in Selenium tutorial, I have created an instance of Selenium RemoteWebDriver as the TestNG framework is best utilized with a Cloud Selenium Grid. The reason being, along with the advantages of Selenium Grid like executing multiple cases together across a variety of browsers and OS, a Cloud Grid also offers the speed and scalability which makes executions faster and more reliable with improved debugging and troubleshooting support.

So, in this code example, we will be executing our code on LambdaTest, which is one such Selenium Cloud Grid for performing cross browser testing at scale. It helps to execute testing across 3000+ real online browsers and OS without the hassle of maintaining in-house Selenium Grid. Selenium testing tools like LambdaTest provides interactive reporting dashboards for results analysis, a great variety of logs, and integrations with top CI/CD tools, project management tools, etc.

Next, we add the LambdaTest username and access key from the LambdaTest profile section to be used to run the script.

Code Walkthrough

Inside the setUp() method, browser capabilities are being set, and a remote grid is launched, which is to be used for executing the run.

Automation Using Selenium TestNG

browserProfile() method will help to execute the step to maximize the browser window once it is launched.

Automation & Selenium TestNG

We have used the navigateToUrl() method to open the website on which the test would be executed.

Selenium TestNG Automation

We have added two test cases that will run on local only to showcase the usage of priority and invocationCount usage, namely testPriotity() and testInvocationCount().

Selenium TestNG Automation Guide

sanityTestDataProvider() provides two sets of test data on which the test method would be executed of which one should pass and the other should fail, as already stated above.

Guide Selenium TestNG Automation

testSumOfTwoValues() is the actual test method where we have added the steps to enter the values and fetch the results to be compared using assertion. This method is executed once for each type of test data and helps to decide whether it is a pass or fail by means of assert.

Selenium TestNG for Automation

Next to fetch the result of execution, we use the getText() method with the identifier and store it as a String. Post this, we add Assertion to help validate the result.

Automation - Selenium TestNG

In this code, we have implemented assertEquals which will compare the values and print a message if there is a mismatch.

Selenium TestNG

clearOldData() function helps to clear the data filled in the input text boxes as a part of previous test execution.

Selenium TestNG- Automation

After every execution, it is a good practice to close any driver instances that we have created during the course of execution. The same has been implemented by the closeDriver() method.

Automate Using Selenium TestNG

In order to signify the completion of the executed test run and provide the tester with a message that the test report is ready to be analyzed, we have used the testReporting() method.

The two most important parts of the sample code are the dataProvider method and the main test method.

The above helps to find the web elements and enter data in the input boxes and then it by clicking the button. To fetch web element locators, we navigate to the page and right-click, and then do Inspect to find the locators as desired by any strategy like id, XPath, CSS, etc.

Automate - Selenium TestNG

This Selenium testing tutorial for beginners will help you interact with web elements in Selenium WebDriver.

You can follow the LambdaTest YouTube Channel and stay updated with the latest tutorials around Selenium testing, Cypress testing, CI/CD, and more.

Having understood the sample code, now execute the class as TestNG Test to get output.

Automate by Selenium TestNG

In this screenshot, we can see the test case gets executed as per priority, and the one with invocation count as 2 gets executed two times.

You can see that Default test numbers show total Tests run: 3 since we have three @Test annotated test cases in the class of which 1 has failures and hence Failures: 1.

However, the Default suite section shows the actual number of test cases along with the actual passed and failed number of cases.

Total tests run: 5 -> testPriotity() executed 1 time + testInvocationCount() executed 2 times + testSumOfTwoValues() executed once for each data set (total 2 times) -> 1 + 2 + 2 = 5

For the failed test case, the log shows the message we added on the assert to appear in case of failure: Expected and actual results do not match. Expected: 7 and Actual: 6

Also, we can log in to the LambdaTest Dashboard to check the execution results for the automation script we have executed.

Test Automation Tool

You can see more detailed logs and reporting for this run by clicking on the recent test run and navigating to the LambdaTest Automation Logs section, as shown below.

Free Test Automation

In addition to the Automation Logs and related info with it, the LambdaTest Dashboard now provides users with more detailed views and data about test case execution and associated trends in graphical representation by means of LambdaTest Analytics Dashboard available under the Automation Tab.

Free Test Automation

Analytics also provides you the ability to analyze the test trends and resource utilization by means of multiple filters like users, OS, browsers, etc.

Selenium -TestNG- Automation

TestNG Reports Generation in Selenium

The same results can be analyzed in a more formatted way with the help of HTML reports generated by TestNG. To access the same, refresh your project by right-clicking on the project name. This is advised to update the report to the latest run.

Next, navigate to the test-output folder and from it open the index.html file. This is the default format report for the TestNG run.

Platform For Test Automation

If you select the test from the Info section, it shows the number of classes executed and the number of failed tests.

Running Selenium TestNG Automation

Selecting the group section shows the name of the group and test case methods executed as a part of it.

Running Selenium- TestNG Automation

Chronological View shows the order in which all the methods in the class were executed, along with the time taken by each function on each step.

Test Automation Results

Last, and most important is the Results section, which shows the complete detail of the executed cases with proper exception/asserts failure in case of failed test cases. We can hide/show only passed or only failed test cases as well using this section as per the requirement.

Test Automation- Results

Results For Test Automation

Also readHow To Use TestNG Reporter Log In Selenium

If you are a test automation expert and want to demonstrate your proficiency in the TestNG framework, you can earn this TestNG certification offered by LambdaTest for FREE

TestNG Certification

With TestNG certification, you can challenge your skills in performing automated testing with TestNG and take your career to the next level.

Here’s a short glimpse of the TestNG certification from LambdaTest:


With this Selenium TestNG tutorial, we can easily conclude that using TestNG in Selenium makes the tests more structured and easy to maintain. Moreover, with the support of its additional features like reporting, dataProviders, more robust annotations, etc., over its predecessors, TestNG makes the automation testing experience user-friendly. Having said that, it is time for you to get started with your first TestNG in Selenium automation project, and remember to use the annotations wisely as they are to make life easier.

Happy Testing!!

Frequently Asked Questions (FAQs)

What is TestNG used for?

TestNG is one of the leading testing frameworks available because it is built with powerful annotations and has a clear, understandable syntax. It allows testers to quickly configure tests without needing several lines of code. Using TestNG’s dataprovider annotation, testers are able to acquire data while they’re structuring their tests and therefore eliminate the need to write multiple lines of code just to acquire data.

What is difference between TestNG and Selenium?

TestNG is a Java framework that allows you to write the automation code for testing applications and packages. Selenium is a testing framework specifically designed to test the UI of the application and how it behaves on browsers.

Why TestNG is used in Selenium?

TestNG is one of the most popular open-source testing frameworks for modern Java developers, and it’s especially popular in the Selenium community. TestNG offers a more complete suite of testing capabilities than JUnit, including detailed reporting, functional testing tools, and test-coverage reporting. It also provides third-party plugins, so it’s compatible with existing Maven/Gradle workflows and CI servers such as Jenkins.

Author’s Profile

Vipul Gupta

Vipul Gupta is a passionate Quality Engineer with 6+ years of experience and keen interest in automation testing of Web and API based applications. He is having experience in designing and maintaining various automation frameworks. Currently working as Sr. SDET, he enjoys reading and learning about new test practices and frameworks.

Blogs: 3


Test your websites, web-apps or mobile apps seamlessly with LambdaTest.

  • Selenium, Cypress, Playwright & Puppeteer Testing
  • Real Devices Cloud
  • Native App Testing
  • Appium Testing
  • Live Interactive Testing
  • Smart Visual UI Testing
Book a Demo