How To Group Test Cases In TestNG [with Examples]

TestNG is an open-source automation testing framework inspired by JUnit and NUnit. The framework supports data-driven testing, parallel test execution, testing integrated classes, provides access to HTML reports, amongst others. TestNG can be seamlessly integrated with Jenkins, Eclipse, IntelliJ IDEA, Maven, etc.

Source

TestNG Annotations are useful for controlling the test methods’ execution flow and running multiple tests in Parallel. Parameterized TestNG tests with Selenium WebDriver are handy for running test(s) against different browser and platform combinations. As a part of our TestNG tutorial series, we look at how to group test cases in TestNG. The usage of groups in TestNG is demonstrated with relevant TestNG group examples. If you are preparing for an interview you can learn more through TestNG interview questions.

What Are TestNG Groups?

In TestNG, groups allow for the logical grouping of test methods. This feature enables selective test execution and better organization, especially in large test suites. By defining groups in the test code, users can easily include or exclude specific groups during the test run, facilitating targeted testing and efficient management of test cases.

Consider a scenario where the test suite comprises different test types, e.g., unit test, integration test, smoke test, etc. Instead of adding all the test scenarios in a single test suite (or multiple test suites), you can perform ‘grouping’ test methods. This helps in segregating the test scenarios into different test categories – test type (e.g., smoke, functional, etc.), functionality being tested (login, checkout, etc.), and test combinations (e.g., browser set to Chrome, Firefox on Windows 10, etc.).

The sophisticated groupings of test methods through TestNG groups are used for:

• Declaring methods that belong to groups.
• Specifying groups that contain other groups.
• Running tests within the same group or multiple groups.
• Including (or excluding) tests in a group.
• Including a certain set of groups (or regular expressions) while excluding another group
• Running tests that depend on the other tests and avoiding unnecessary test execution if the dependent test has failed.

Advantages of TestNG Groups

Here are some of the major advantages of groups in TestNG:

• Flexibility to partition tests, thereby reducing the efforts involved in the maintenance of the tests.
• Avoids the need to recompile the code if two different sets of tests are run back to back.

TestNG groups can also be used for running tests in Parallel at the ‘Test’ level or ‘Method’ level. In further sections of this ‘How to group test cases in TestNG’ tutorial, we would demonstrate parallel testing in TestNG and Selenium using Test Groups.

With TestNG certification, you can challenge your skills in performing automated testing with TestNG and take your career to the next level.

Here’s a short glimpse of the TestNG certification from LambdaTest:

How to group test cases in TestNG?

Groups in TestNG are specified in testng.xml under the <suite> or <test> tag. Groups under the <suite> tag apply to all the tests included under the <test> tag in that particular <suite>.

To group tests in the source code, you have to use the @groups attribute of the @Test annotation. TestNG provides the option to structure the test such that the entire test class belongs to a particular TestNG group or a couple of methods belong to the TestNG group. Also, test methods can belong to one or more TestNG groups.

Let’s take a simple example where the test code has three test methods (test_method1, test_method2, and test_method3). We create two TestNG groups – group1 and group2.

Here is how to group test cases in TestNG. As shown below, test_method1() is part of both the groups and test_method2() & test_method3() are a part of the group2 & group1 respectively.

The TestNG groups are invoked in testng.xml:

As seen in testng.xml, the groups are included under the <group> tag. As we want to run the tests included under both the groups (group1 and group2), we include the required group names under the <run> tag.

Demonstration: How to use TestNG Groups

To demonstrate running Selenium tests in TestNG within the same groups, within multiple groups, including (and excluding) test methods, etc., we use the cross browser testing scenarios that will be executed on the cloud-based Selenium Grid by LambdaTest.

• Test Group – 1 (GroupName – Search)

Test Scenario – 1 [OS – Windows 10, Browser – Chrome, Version – Latest]

1. Go to Google.
2. Enter the search string as ‘LambdaTest.’
3. Assert if the page title does not match the expected title.

Test Scenario – 2 [OS – MacOS Catalina, Browser – Safari, Version – 13.0]

1. Go to Bing.
2. Enter the search string as ‘LambdaTest Blog.’
3. Assert if the page title does not match the expected title.

• Test Group – 2 (GroupName – ToDo)

Test Scenario – 1 [OS – Windows 10, Browser – Firefox, Version – 68.0]

1. Go to LambdaTest ToDo App.
2. Mark the First Item (li1) as Done.
3. Mark the Second Item (li2) as Done.

Test Scenario – 2 [OS – MacOS Catalina, Browser – Safari, Version – 13.0]

1. Go to LambdaTest ToDo App.
2. Mark the First Item (li1) as Done.
3. Mark the Second Item (li2) as Done.
4. Mark the Third Item (li3) as Done.

Here is the overall project structure which we created in IntelliJ IDEA IDE:

In the TestNGGroups project, we create a package named org.testnggroup, where we create the following class files (that contain the implementation of the relevant test scenarios):

(Implementation of Test Scenarios in ‘Test Group 1’)

(Implementation of Test Scenarios in ‘Test Group 2’)

(Implementation of Helper Functions for creating an instance of RemoteWebDriver)

Here are the three methods implemented in Helper.java:

1. setupThread: This method takes five arguments (Build, Name, platformName, browserName, and browserVersion), which are inline with the requirements of the LambdaTest capabilities generator. The cross browser tests would be executed on Selenium 4 Grid.

   It instantiates a new ThreadLocal for each test class since the tests (in the two-class files) are executed in Parallel at the ‘Method’ level.

   

2. getDriver: This method returns the WebDriver object as a ThreadLocal for the parallel tests.
3. tearDownThread: This method releases the resources held by the current WebDriver object.

How To Run Test Cases Within The Same Group?

As part of TestNG group examples, we explore the most basic use of groups in TestNG. In a project, you could have multiple Groups, but you might be interested in running tests that belong to a particular TestNG Group. In such cases, you can group the tests to selectively run ‘all the tests’ within a specific group.

In our example, we have two TestNG groups – Search and ToDo. Let’s look at how to run test cases within the same group (e.g., Search) in Parallel. Parallel testing in Selenium is preferred so that tests can be executed faster, and you can also make the most of the capabilities offered by the cloud-based Selenium Grid.

In testng.xml, we set the thread-count attribute to 2 and the parallel attribute to “methods.” Since we want to run the test cases implemented under the ‘Search’ group, the group is included in the <run> tag under <groups> .

There are two test methods under the TestNG Group “Search”:

• test_GoogleSearch – Priority is set to ‘1.’
• test_BingSearch – Priority is set to ‘2.’

The specified group tests will run in Parallel as per the priority specified in the @Test annotation.

As seen in the execution snapshot, two test methods under the TestNG tag – “Search” are running in Parallel:

Shown below are the execution snapshots from the IDE and the Automation Dashboard from LambdaTest it indicates that their tests were executed successfully:

How To Run Test Cases Within Multiple Groups?

TestNG lets you run test methods that belong to multiple groups. The group names are provided as an array in the groups attribute of the @Test annotation. In our case, we have four test methods that are grouped under two TestNG groups. Since the test methods are a part of two different classes, the two Groups are included under two separate <test> tags.

The test methods of two different classes are run under different threads. Since we have set the parallel attribute to ‘methods,’ the methods (as per the set priority) in the respective classes are executed in parallel.

At first, the test methods under the group “Search” are executed in parallel. Post the successful execution of those methods, the test methods under the TestNG group “ToDo” are run in parallel.

Here are the execution screenshots that indicate that all four test methods under the groups “Search” and “ToDo” were executed successfully:

Groups Within Groups In TestNG

In this section of the TestNG tutorial, we look at how to use ‘MetaGroups’ in TestNG. Groups can include other groups. The combined group is called ‘MetaGroup.’ Groups within groups in TestNG can be helpful in cases where a ‘collection’ of test scenarios from different groups have to be run as a part of the test case.

For demonstrating how to group test cases in TestNG when Groups are within Groups, we create four test groups – Search1 and Search2 (for including the test methods that demonstrate the search functionality) & ToDo1 and ToDo2 (for including the test methods that demonstrate the ToDo App functionality).

Shown below are the code modifications for creating four TestNg groups:

A new group named “Group1” includes the test methods implemented under the “Search1” and “Search2” groups. Another group, “Group2,” includes the test methods implemented under the “ToDo1” and “ToDo2” groups. The groups Group1 and Group2 are included as a part of a new group named ‘SuperGroup.’

The four tests which are a part of ‘SuperGroup’ are run by including the SuperGroup under the < run > tag.

The four test methods that are a part of the nested group named SuperGroup are executed in parallel:

All four test scenarios under ‘SuperGroup’ are executed successfully:

How To Exclude/Include Test Cases With Groups?

TestNG Groups also provides provision to ignore test scenario(s)/test methods using the <exclude> tag in the <methods> section. On similar lines, the <include> tag in the <methods> section of testng.xml is used for including the test methods as a part of the test execution process.

1. TestNG_SearchGroup class: To run test methods whose name includes “.*Bing.*” and exclude test method(s) whose name contains “.*Google.*”, we use method groups in the following manner:

The matching method in the class is test_BingSearch().

1. TestNG_ToDoGroup class: To run test methods whose name includes “.*ToDoApp.*” and exclude test method(s) whose name contains “.*Test1.*”, we use method groups in the following manner:

The matching method in the class is test_Selenium4_ToDoApp_Test2().

Here is the execution snapshot, which indicates that the two matching test methods are run successfully on the cloud-based Selenium Grid:

How to Exclude a TestNG Group?

The TestNG framework also lets you include as well as exclude them. Exclusion of Groups in TestNG is useful in cases where there is a temporary breakage in the tests (due to some recent changes), and you do not have the time to fix them.

The tests in those groups can be temporarily deactivated to be later reactivated once the issues have been fixed. Groups can also be excluded using Regular expressions.

For demonstrating exclusion of TestNG groups, we use the <include> tag in <groups> to include TestNG groups that match the regular expression “.*ToDo.*”. The <exclude> tag in <groups> is used for excluding the TestNG group with the name “Search”.

The TestNG Group named “ToDo” matches the criteria set (i.e. “.*ToDo.*”) in the <include> tag. On the other hand, the test methods implemented in TestNG_SearchGroup are excluded as they meet the <exclude> tag requirements.

As seen below, the test methods from the matching TestNG Groups were executed successfully on the LambdaTest Selenium Grid:

Regular Expressions and TestNG Groups

TestNG uses Regular Expressions (i.e., anything that matches “.*” – Dot Star) and not WildMats (i.e., “*”) for running test methods that match the specified pattern. For demonstrating the usage of regular expressions with TestNG groups, we run test methods that include the term “Search”.

testng.xml to run test cases by matching Group Names using Regular Expressions

As seen in the testng.xml, tests under the groups – “Search” will be executed. Hence, the test methods test_GoogleSearch() and test_BingSearch() are run as per the assigned priority.

As shown below, the test methods matching the corresponding TestNG Group name are run successfully:

How To Group Your Tests On LambdaTest?

LambdaTest lets you group the automation tests using custom tags. The advantage of using custom tags is that it helps ease the Search for your test cases on the LambdaTest automation dashboard.

To group tests on Lambdatest using custom tags, create a String array containing the names of the custom tags, separated by a comma.

Now add this custom tag in the Desired Capabilities instance, as shown below:

In our case, we set the capabilities in the setupThread() method, defined in the Helper class. We create a Custom Tag that consists of the combination of Platform Name and Browser Name & Browser Version used in the cross browser testing.

For viewing the custom tags on the automation logs, navigate to the Automation logs on the automation dashboard, and check the applied custom tags below the test names in the left panel.

You also have the flexibility to filter the tests by selecting multiple tags at once from the filter toolbar.

Here we filter the tags matching ‘MacOS’ to locate the tests run on the macOS platform:

Custom Tags should be efficiently used for grouping test cases on LambdaTest so that it becomes easy to filter ‘required’ tests from the numerous tests executed on the Grid.

Check a complete guide for your first TestNG automation script.

Wrapping up

Source

In this TestNG tutorial, we had a detailed look at how to group test cases in TestNG. Groups in TestNG are useful in projects where you want to run test scenarios that match certain criteria. For example, all the cross browser tests can be put under a group named “crossbrowser” so that the tests can be easily identified. Grouping of tests also helps in temporarily disabling tests that might be causing breakage in the other tests.

As evident from the TestNG group examples, you can run tests within the same group, within multiple groups, within nested groups, and more. Regular expressions with groups and test methods are convenient when you want to include/exclude test cases (or groups) that match a certain condition.