Getting Started with MongoDB Testing Using Selenium WebDriver

Daniel Mascarenhas

Posted On: September 25, 2023

view count164192 Views

Read time9 Min Read

Test automation has become an important part of product quality. Without good test automation coverage, the product cannot scale in terms of new features. In the last 20 years, I have seen how test automation teams have become the center of attention in the development team.

Software organizations are pressured to adopt new cutting-edge technologies to stay ahead. Continuous changes to the product are, hence, inevitable. Change to product functionality means change to test cases & so to the automation suite.

This change can be in any form: change in the testcase, test data, test config data, or the test report itself. Let’s say change in the test case can be handled by refactoring test cases. But what about the other changes? Can those unstructured changes be handled in a more structured way?

The answer to this question is Yes. Thanks to the new DB, which supports such data. It’s called MongoDB.

In this blog on MongoDB testing, we will take a closer look at different features offered by MongoDB and how to perform MongoDB testing using the Selenium test automation framework. If you are preparing for an interview you can learn more through Selenium interview questions.

What is MongoDB?

MongoDB is an open-source database that can store large amounts of unstructured data. As opposed to rows and columns in typical SQL format, MongoDB uses non-SQL format, i.e., it stores data in collections and documents. Each collection contains a set of documents, and each document, in turn, has key-value pairs. Each document has an ID, which is the primary key and represents a unique value in a document.

As per the Stack Overflow developer survey conducted in 2023, MongoDB has been rated as the database most wanted by developers. Relational databases have been around for decades. MongoDB is a scalable, flexible NoSQL document database platform designed to overcome the relational databases approach.

It is a non-relational database. In relational databases, you need to define a schema beforehand. MongoDB doesn’t need such a requirement, making it flexible for introducing new changes. It costs less time and money.

The difference between SQL & NoSQL databases can be described below:

SQL NoSQL
Stands for Structured Query Language Stands for Not Only SQL
Also called a Relational Database Management System (RDBMS) Also called a Non-Relational Database Management System
Suitable for structured data with predefined schema Suitable for unstructured and semi-structured data
Data is stored in tables with columns and rows Data is stored in collections or documents
Requires vertical scaling to handle large volumes of data Horizontal scaling makes it possible to handle large volumes of data
Examples: PostgreSQL, Oracle, Microsoft SQL Server Examples: MongoDB, Cassandra, Amazon DynamoDB, Redis

Let us now look at the key features of MongoDB.

MongoDB Features

MongoDB is an open-source database designed to support unstructured data. This DB has driver support for all popular languages like Java, Python, PHP, Node.js, C, C++, etc.

Here are some of its salient features.

  • Scalability
  • There is no need to define a schema before adding data. We can easily add fields to documents as per our needs. If the tester is referring to test data and test config in his test script from some database, he doesn’t have to worry about schema changes in case of any changes.

  • Ad-hoc queries for optimized, real-time analytics
  • An ad hoc query is a short-lived command whose value depends on a variable. MongoDB uses MongoDB Query Language (MQL). It supports field queries, range queries, and regular expression searches. When designing the schema of a database, it is impossible to know in advance all the queries that will be performed by end users.

    Let’s say a tester is storing his test results in MongoDB, using these ad-hoc queries, he can dwell on introducing test analytics.

  • Native document validation and schema examination with Compass
  • MongoDB provides Compass as a client through which we can connect to a particular database and view documents and their fields.

  • Easy replication
  • MongoDB allows us to replicate data easily. Due to this, it helps us to distribute data across geographical regions.

Info Note

Run automated test suites on 3000+ environments. Try LambdaTest Today!

Why use MongoDB with Selenium?

Integrating MongoDB with Selenium, the powerful web automation framework, offers a compelling synergy that revolutionizes how we handle test data in the world of test automation.

While Selenium is renowned for automating web applications with precision and efficiency, MongoDB, a NoSQL database, is celebrated for its flexibility and scalability in data storage.

Combining these two technologies empowers testers and developers alike to manage test data dynamically, streamline test case execution, and enhance the overall efficiency of the testing process.

This section of the MongoDB testing tutorial will explore why incorporating MongoDB into your Selenium automation stack can lead to more robust and adaptable testing solutions.

  • Schema Flexibility
  • As described initially, test cases rely on test data and config info. Most of the time, this info is stored in traditional DB. Test scripts refer to those data via DB. In case of continuous application changes, changing schema can be tedious. If we use MongoDB to store such data, we don’t have to worry about schema updates.

  • Detailed Test Reports
  • The second most important problem MongoDB solves is test reporting. Test Automation is usually run every day. It creates a lot of test reports and test execution data. In the usual test automation framework, the earlier test run’s report gets overwritten. If we push every test run’s results into MongoDB, we can build detailed test reports with historical data.

  • Business Intelligence
  • We need a mechanism that can store volumes of test reports and test execution data so that we can apply some intelligence & gain insights into our test execution. MongoDB is unstructured and capable of dealing with large data. It provides tools and ways to build sophisticated analytics queries. We can apply business intelligence to the unstructured data stored in MongoDB.

    Let’s see some use cases in action in the next section of this MongoDB testing tutorial.

How to set up MongoDB with Selenium?

MongoDB’s NoSQL architecture pairs seamlessly with Selenium’s web automation capabilities, offering a dynamic approach to managing test data. In this section of the MongoDB testing tutorial, I will walk you through the steps to set up MongoDB with Selenium, including installing MongoDB and MongoDB Compass and creating a sample MongoDB database using Compass.

Installing MongoDB

When writing this blog on MongoDB testing, MongoDB version 6.0.6 is available. However, the Klov server we will use in this blog is incompatible with that MongoDB version. Hence, we will install MongoDB version 5.0.17 on Windows for this exercise.

  1. Go to this link:
  2. https://www.mongodb.com/download-center/community/releases/archive.

  3. Locate the entry, mongodb-windows-x86_64-5.0.17.zip. Download and copy the zip file to C:\MongoDB. Extract it.
  4. Let’s create a data folder for MongoDB to store data. Create folder structure as C:\MongoDB\data.
  5. We are ready to start MongoDB. Go to the extracted folder. Go inside the bin folder. Start CMD from this path.
  6. Run the below command.
  7. mongod –bind_ip_all –port 27017 –dbpath C:\MongoDB\data

Note: –bind_ip_all flag is required if MongoDB installed is on Windows to avoid connectivity issues. The default port is 27017. So even if it is not passed explicitly, that’s fine.

Command output would look something like this.:

Installing MongoDB

Installing MongoDB Compass

MongoDB Compass is the UI client for MongoDB, where we visualize our data. We can interact with data using full CRUD functionality. We can run ad-hoc queries. We can also view and optimize the query performance. It is available on Linux, Mac, or Windows. Compass empowers us to make smarter decisions about indexing, document validation, and more. For this exercise, we will use Windows Server 2016; the steps might differ if you are using any other OS.

  1. Let’s now install ‘MongoDB Compass’. For this exercise, try to install it on the same machine where MongoDB is installed.
  2. Open Windows Powershell.
  3. Go to the location where MongoDB was extracted. Go to the bin folder. Locate Install-Compass.psl. Run Install-Compass.psl script from Windows Powershell. Sometimes, while running the script on Windows, it gives an error such as ‘PowerShell script is not digitally signed.’. To avoid this, first run the below command from powershell cmd.
  4. Set-ExecutionPolicy -ExecutionPolicy unrestricted

  5. Run Install-Compass.psl. This will install a MongoDB client.

Installing MongoDB Compass

Creating a sample MongoDB database using Compass

  1. Open the MongoDB Compass. It will launch the connect page, as shown below.
  2. Open the MongoDB Compass

  3. Click on Connect. It will connect to the MongoDB server.
  4. Click on ‘+’ to create a new database. Give DB details to the database named ‘Demo’ as below and say ‘Create Database’.
  5. Create Database

  6. MongoDB Compass would create DB & would show a blank DB page as below.
  7. MongoDB Compass would create DB

Our MongoDB is set up!

How to write a Selenium script to connect to MongoDB?

Let’s dive in and learn how to create Selenium scripts that establish a robust connection to MongoDB, unlocking possibilities for your MongoDB testing endeavors.

Code Setup

  1. Download the code from GitHub.
  2. Open the project in IntelliJ.
  3. In IntelliJ, use Java 11.0.2 as SDK and Maven 3.3.9.
  4. Project structure would look like this.

Project structure

Code Walkthrough

File name: pom.xml

For Selenium and TestNG libraries, we have added below the entries into pom.xml.

For MongoDB Java drivers, the below library is added.

Besides the above libraries in pom.xml, we have added Maven plugins to build the project.

Finally, Java version 11 has been set to be used for compiling.

File name: MongoDBTest.java

Import statements: We need to import several libraries to accomplish our exercise.

connectToMongoDB(): Function to initialize and connect to MongoDB.

This function will initialize MongoClient and establish a DB connection with the MongoDB database and collection.

Provide the connection parameters to it.

{MongoDBHost}: IP Address of the machine where MongoDB is hosted.

Sometimes, we face issues binding to localhost on Windows. To avoid this, on the MongoDB host, open cmd and run ipconfig. Use the ‘ipv4 address’ as IP for connecting to MongoDB via script.

{MongoDBName}: Database ‘Demo’ created in earlier steps.

{MongoDBPort}: 27017

browserSetup(): Function to initialize browser setup.

We use WebDriver Manager to instantiate Chrome WebDriver instances. We will launch the browser in a headless manner.

Test1() and Test2(): Test methods

Let’s look at the two test methods, Test1 and Test2. These will perform simple checks on the LambadaTest application. It would also append test cases and test execution info into MongoDB.

In Test1(), we first create a document by passing the document and test names. We can then append details in key-value pairs. Once a pair of values is appended, we insert that document into the MongoDB collection at the end.

otherCRUDOperations(): This method showcases how different CRUD operations can be performed on MongoDB.

After updating connection details, execute the script via IDE.

execution data

Once the test is executed, go back to MongoDB to check the test execution data.

As we can see below, data has been populated into MongoDB.

 new execution

Let’s rerun the same test class to see how new execution data is getting appended into MongoDB.

We can see below MongoDB has inserted data for the next run. Since MongoDB assigned an object id for every record, it becomes easier to track historical data. We can use this historic execution data to build new reports or apply analytics.

demo MongoDB

So, this is how we can interact with MongoDB via Selenium scripts. MongoDB can help testers build test scripts that are resilient to any change. We can even build reports from scratch by leveraging MongoDB documents and collections.

Let’s now take an example of one third-party test automation reporting tool leveraging MongoDB to build sophisticated test automation reports.

Info Note

Automate your test scripts with Selenium on the cloud. Try LambdaTest Today!

Third-Party Reporting Tools with MongoDB

As some might be familiar, Extent Report is one open-source reporting library useful for test automation. It can be easily integrated with major test automation frameworks like JUnit, TestNG, JGiven, etc. These reports have pie chart representation, test stepwise report generation, screenshots for failing test cases, etc. Due to a good, presentable user interface, these reports can be shared with all stakeholders.

Extent reports have gone one step ahead and leveraged MongoDB to build a real-time analytic report server called ‘Klov’.

What is Klov?

Klov is a reporting server for extent reports. It provides a detailed analysis of our most recent builds. It enables us to leverage historical data to analyze how tests have performed against AUT (Application under test). It works with the Extent Report Library to push data into a MongoDB instance. And then, these reports are published on the Klov server.

To use the Klov server, we should use extent reports in our tests. Let’s see how we can rewrite the earlier test using Klov for MongoDB.

How to set up Klov?

Installation of Klov is very simple. As the community version is Docker-based, we must deploy the container image to Docker.

Installing the Docker Engine on a CentOS machine

  1. Uninstall any older versions of Docker before attempting to install a new version.
  2. sudo yum remove docker \

      docker-client \
      docker-client-latest \
      docker-common \
      docker-latest \
      docker-latest-logrotate \
      docker-logrotate \
      docker-engine


    docker-engine

    Note: yum might report that you have none of these packages installed as above. You can ignore that.

  3. Set up the rpm repository.
  4. sudo yum install -y yum-utils

    sudo yum install

    sudo yum-config-manager –add-repo
    https://download.docker.com/linux/centos/docker-ce.repo

    sudo yum-config-manager

  5. Install the Docker Engine.
  6. sudo yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

    install docker-ce docker

    It would prompt the below question, type Y.

    package

  7. Start Docker.
  8. sudo systemctl start docker

    Verify that the Docker Engine installation is successful by running the hello-world image.

    sudo docker run hello-world

    Start Docker.

Installing Klov image

  1. Download the Docker-compose.yml file to install the Klov image. Run the below command.
  2. curl
    https://raw.githubusercontent.com/extent-framework/klov-server/master/docker-compose.yml -o docker-compose.yml

    Download the Docker-compose.yml file

  3. Open the yml file. Update the MongoDB connection details and the port where Klov will be hosted.
  4. version: '2'
    services:
    klov:
    image: anshooarora/klov:1.0.1
    container_name: klov
    environment:
    - SPRING_DATA_MONGODB_URI=mongodb://{MongoDBHost}:{port}
    ports:
    - 80:80

    In place of {MongoDBHost}, give the IP (ipv4) address of the Windows host machine where MongoDB is hosted (used earlier in the Selenium script). {port} would be the default MongoDB port, 27017.

  5. Start Klov.
  6. docker-compose up

    It should show a message such as “org.springframework.boot.StartupInfoLogger: Started KlovApplication in 9.266 seconds (JVM running for 10.14).”

    Start Klov.

  7. Open Klov at the $PORT you specified in docker-compose.yml or default:80. Give url as: http:///projects
  8. It would look something like this.

    $PORT

    Changes required on the test project side for Klov

    In pom.xml, in addition to earlier dependencies, we have below entries added for Klov.

    Code Walkthrough

    File name: Test_Extent_Klov.java

    Import statements: We are using several libraries to accomplish our task.

    klovSetup(): This function will initialize the Klov server.

    Method klovSetup() has all the methods required to establish and initialize the Klov server connection. It provides the below connection parameters.

    • {MongoDBHost}: IP Address of the machine where MongoDB is hosted. Same as earlier.
    • {KlovHost}: CentOS machine hostname where Klov Docker image is hosted.
    • {KlovPort}: Port where Klov Server listens as mentioned in the Docker-compose.yml file. By default, it is 80.

    browserSetup(): Function to initialize browser setup. We are using WebDriver Manager to instantiate Chrome WebDriver instances. We will launch the browser in a headless manner.

    Test1(), Test2(): Let’s see how test execution details can be pushed to extent reports and then to the Klov server.

    Provide all the connection details as discussed above and execute the above script via IntelliJ.

    After execution

    After execution, access the Klov server URL now. You will see new entry called ‘LambdaTest.’

    new entry called ‘LambdaTest.’

    Let’s rerun the same test script to have more execution data and revisit the Klov server.

    revisit the Klov server

    If we navigate further, we will see detailed test execution reports with analytics.

    test execution reports with analytics

    Klov created the above reports based on MongoDB. It created a good schema in MongoDB to store different aspects of test execution reports.

    good schema in MongoDB

    Conclusion

    With this, we’ve reached the conclusion of this blog post on MongoDB testing. We saw why Selenium with MongoDB is a powerful combination. We can build test scripts with MongoDB to refer to dynamic test config and test data. We can also store and look into historical data of test execution results. We can build it from scratch or use a reporting tool, Klov, which uses MongoDB internally.

    Frequently Asked Questions (FAQs)

    What is MongoDB in testing?

    In the context of testing, MongoDB refers to a NoSQL, document-oriented database commonly used to store and manage test data. MongoDB is particularly popular in testing and quality assurance due to its flexibility, scalability, and ability to handle unstructured or semi-structured data.

    How to create a test database in MongoDB?

    In MongoDB, you can create a test database by following these steps:

    1. Start MongoDB
    2. Access the MongoDB Shell
    3. Switch to the Admin Database (Optional)
    4. Create a Test Database
    5. Verify the New Database
    6. Insert Data (Optional)
Author Profile Author Profile Author Profile

Author’s Profile

Daniel Mascarenhas

Daniel has experience in Testing & quality assurance for software products (Web, Enterprise & SaaS products) spread across design, development & validation. He has also implemented Test automation solutions at various Government & private sector companies to improve their quality process.

Blogs: 2



linkedintwitter