How to Integrate LambdaTest with Calliope.pro?
Ramit Dhamija
Posted On: September 12, 2019
18009 Views
9 Min Read
Collaboration is pivotal for any successful release. Can you imagine going through a sprint without consulting or informing any other team involved in the project about what you did? You can’t right because it is not a pretty picture. Modern SDLCs demand various teams to coordinate as they try to deliver a product as quickly as possible in the market, with assured quality.
Your job as an automation tester doesn’t end up by merely running automation scripts for your web application, you also have to share the test results across different teams such as the development team, or business analysts. Generating and presenting reports requires a lot of effort as your reports need to clearly indicate which modules are working fine and which ones are failing.
This week I added a new tool in my testing checklist to help me share across the results of my Selenium automation testing across my team as I performed automated cross browser testing, and the tool is Calliope. In this article, I am going to help you integrate your LambdaTest account with Calliope, so you could share reports of test automation scripts executed at LambdaTest Selenium Grid across your teammates in a jiffy. Let’s start with the big question.
What Is Calliope?
Calliope is cloud-based tool providing you with a collective dashboard for sharing, and monitoring, and comparing the results of your automation script execution. Offering compatibility with major test automation frameworks, Calliope enables your organization to have a unified view of everything that is going around with your automation test scripts. Your stakeholders can analyze it too for comparing the current state of your test cycles with a historic state.
Calliope ensures that your entire team is on the same page when it comes to analyzing your test results. Here is how:
- Sharing your test results – Your test results are presented in Gherkin syntax, making it easier to understand by your stockholders as well.
- Customize your own dashboard – Invite your colleagues and structure the dashboard in a manner as you structure your team.
- Compare Historical Data – You can compare the current health status of your test builds with historical status. Plus, all your test cases are gathered in a single region allowing even your non-technical stakeholders with a clear picture of the overall health for your automation scripts.
- Easy Regression Check – Calliope stores all your test imports on their clouds making it easier for you to reflect back anytime for validating your regression test cycles.
- API integration – Calliope API will help you run your test suites on third-party tools instantly through the test results dashboard presented in your Calliope instance.
- CI/CD Integration with GitLab – Like LambdaTest, Calliope also offers integration with GitLab CI to help you import your test cases directly from your CI/CD pipeline to Calliope instance.
Now, that we have an idea about the various features offered by Calliope. It is about time we get to know about integrating LambdaTest with Calliope. However, to those of us who are not aware of LambdaTest.
What Is LambdaTest?
We offer a cloud-based cross browser testing platform to help you perform browser compatibility testing by both manual and Selenium. LambdaTest offers a Selenium Grid of 3000+ real browsers for both mobile and desktop, running on real operating systems. Here is a quick video tutorial about us:
Perform Cross Browser Testing with LambdaTest on 3000+ Browsers With A Free Sign-up.
Why Should You Integrate LambdaTest With Calliope?
If not well-orchestrated, cross browser testing can turn out to be a mess. You will require a thorough plan and strategy, design a cross browser compatibility matrix to figure out which browsers are of higher priority and which ones are of the lowest. Depending on the amount of test coverage, and time in-hand, you will need to make a decision to opt for automated cross browser testing. To pace things even faster you would require parallel execution with Selenium, by which you can run multiple test cases simultaneously. And even after following up the right plan, you would need a testing dashboard where you can club all of your test cases in one place, for everyone to have a clear and concise view. This is where LambdaTest integration with Calliope comes into the picture.
Using LambdaTest integration with Calliope you can analyze the test results of automation test scripts being executed in parallel on more than 3000 browsers + OS combinations. Let’s get started.
Import LambdaTest Automation Test Results Into Calliope Dashboard
So, now you are aware of LambdaTest and Calliope’s services, let’s have a look at how you can import your LambdaTest automation test results directly to Calliope dashboard.
For fetching LambdaTest automation test results details from scratch we need to call LambdaTest API which provides us our recently executed test session details such as name, status, os, browser, version and all generated logs endpoint. We will be making use of the below API:
Get/sessions/{session_id}
It will provide us with information specific to our test sessions. This would require you to give the session ID for the test session you want to retrieve the details for. You can refer to LambdaTest API Documentation to check the example value or schema.
Below is the test session details that would be fetched through LambdaTest API and will get imported to Calliope dashboard.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
{ data:{ name=Calliope-Sample-Test, duration=24, platform=win10, browser=chrome, browserVersion=74.0, device=, statusInd=running, sessionId=ca7e3b1df8ddc533fc5394141601431f, buildName=LambdaTest Integration with Calliope, consoleLogsUrl=https://api.lambdatest.com/automation/api/v1/sessions/ca7e3b1df8ddc533fc5394141601431f/log/console, networkLogsUrl=https://api.lambdatest.com/automation/api/v1/sessions/ca7e3b1df8ddc533fc5394141601431f/log/network, commandLogsUrl=https://api.lambdatest.com/automation/api/v1/sessions/ca7e3b1df8ddc533fc5394141601431f/log/command, seleniumLogsUrl=https://api.lambdatest.com/automation/api/v1/sessions/ca7e3b1df8ddc533fc5394141601431f/log/selenium, videoUrl=https://automation.lambdatest.com/public/video?testID=3K6C3-HBAMX-BUS4B-FWY5Z, screenshotUrl=https://d15x9hjibri3lt.cloudfront.net/3K6C3-HBAMX-BUS4B-FWY5Z/screenshots.zip }, message:Retrieve session was successful, status:success } |
You might be wondering how you can send the above test session details directly to Calliope dashboard after retrieving from LambdaTest API. So, here is the Java-TestNG code for you that can help you setting up the test environment which further includes your username, accesskey, gridURL and test configurations such as browser, browser version, OS etc.
Once you have set the test environment, you can now write your test cases. We have been using SessionId Java class that provides the session ID for running session, we can use the same session ID for calling LambdaTest GET Session API whose URL is:
https://api.lambdatest.com/automation/api/v1/sessions/
For accessing the LambdaTest API, you need to get authorized with your credentials that include your username and access key. We have used Base64 class to encode the credentials which are universally accepted format by web servers and browsers.
Now, we can use our encoded basic authorization with session URL to send the GET request for the test session details.
Since, we are using Java-TestNG here, all our test results and logs get saved in testng-results.xml file. The purpose of this file is to capture the test session data and can further be used with Calliope API in order to transfer the saved test session data to Calliope Dashboard. In the below code, we have also used Reporter.log which would report the test logs and all other session details to the testng-results.xml file.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 |
BaseTest.java package calliopeIntegration; import java.io.BufferedReader; import java.io.InputStreamReader; import java.net.MalformedURLException; import java.net.URL; import java.util.Base64; import org.apache.http.HttpResponse; import org.apache.http.client.HttpClient; import org.apache.http.client.methods.HttpGet; import org.apache.http.impl.client.HttpClientBuilder; import org.openqa.selenium.By; import org.openqa.selenium.JavascriptExecutor; import org.openqa.selenium.remote.DesiredCapabilities; import org.openqa.selenium.remote.RemoteWebDriver; import org.openqa.selenium.remote.SessionId; import org.testng.Reporter; import org.testng.annotations.AfterMethod; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Listeners; import org.testng.annotations.Test; import com.fasterxml.jackson.databind.DeserializationFeature; import com.fasterxml.jackson.databind.ObjectMapper; import calliopeIntegration.session.SessionResponse; @Listeners({IntegrationExecution.class}) public class BaseTest { public String username = "Your_Username"; //LambdaTest Username public String accesskey = "Your_AccessKey; //LambdaTest AccessKey public static RemoteWebDriver driver = null; public String gridURL = "@hub.lambdatest.com/wd/hub"; //GridURL boolean status = false; @BeforeMethod public void setUp() { DesiredCapabilities capabilities = new DesiredCapabilities(); // Setting configurations capabilities.setCapability("browserName", "chrome"); capabilities.setCapability("version", "74.0"); capabilities.setCapability("platform", "win10"); // If this cap isn't specified, it will just get the any available one capabilities.setCapability("build", "LambdaTest Integration with Calliope"); capabilities.setCapability("name", "Calliope-Sample-Test"); capabilities.setCapability("network", true); // To enable network logs capabilities.setCapability("visual", true); // To enable step by step screenshot capabilities.setCapability("video", true); // To enable video recording capabilities.setCapability("console", true); // To capture console logs try { driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities); } catch (MalformedURLException e) { System.out.println("Invalid grid URL"); } catch (Exception e) { System.out.println(e.getMessage()); } Reporter.log("Build and Session Information retrieved from LambdaTest API's",true); } // Test cases @Test public void BuildSession() throws Exception { try { driver.get("https://www.apple.com/"); driver.manage().window().maximize(); Thread.sleep(5000); driver.findElement(By.xpath("//*[@id=\'ac-globalnav\']/div/ul[2]/li[3]")).click(); Thread.sleep(2000); driver.findElement( By.cssSelector("#chapternav > div > ul > li.chapternav-item.chapternav-item-ipad-air > a")).click(); Thread.sleep(2000); driver.findElement(By.linkText("Why iPad")).click(); Thread.sleep(2000); } catch (Exception e) { System.out.println(e.getMessage()); } } @AfterMethod public void teardown() throws Exception { SessionId session = driver.getSessionId(); String sessionId = session.toString(); if (driver != null) { ((JavascriptExecutor) driver).executeScript("lambda-status=" + status); driver.quit(); } String usernameColonPassword = username+":"+accesskey ; // API Authorization String basicAuthPayload = "Basic " + Base64.getEncoder().encodeToString(usernameColonPassword.getBytes()); Reporter.log(basicAuthPayload); //Session Info String sessionURL = "https://api.lambdatest.com/automation/api/v1/sessions/"+sessionId; Reporter.log(sessionURL,true); String jsonResponse2 = sendGetRequest(sessionURL,basicAuthPayload); ObjectMapper objectMapper2 = new ObjectMapper(); objectMapper2.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false); SessionResponse response2 = objectMapper2.readValue(jsonResponse2, SessionResponse.class); Reporter.log(response2.toString(),true); } public static String sendGetRequest(String url, String basicAuthPayload) throws Exception { HttpClient client = HttpClientBuilder.create().build(); HttpGet request = new HttpGet(url); // add GET request header request.addHeader("Content-Type", "application/json"); request.addHeader("Authorization", basicAuthPayload); HttpResponse response = client.execute(request); BufferedReader rd = new BufferedReader(new InputStreamReader(response.getEntity().getContent())); StringBuffer result = new StringBuffer(); String line = ""; while ((line = rd.readLine()) != null) { result.append(line); } return result.toString(); } } |
Now, after executing the above code, we have our test executed on LambdaTest Selenium Grid along with the test testng-results.xml file saved in our system which includes all our recent test session details.
The next step is to call Calliope API to post the test data from testng-results.xml file to Calliope dashboard. Calliope support different result file format according to different frameworks. For instance, XML for Junit and TestNG, JSON for Cucumber. For more information on this, you can refer to Calliope’s documentation link.
We have used MediaType class to define the file type(XML) for our TestNG framework. Now, we need a testng-results.xml file to GET and POST test session data. So, we need to define the directory where this file is saved to access the file by calling the Calliope API.
We have defined API command as an endpoint URL which includes your profile number along with OS, platform and build name. You can extract your Calliope API from their documentation for API import.
Make sure you have entered the correct API Key, otherwise, your test would fail due to unmatch of API Key with your profile number.
After setting up the endpoint URL and the API Key, it’s time to call Calliope API now by sending a POST request for importing test results to the dashboard.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
CalliopeAPI.java package calliopeIntegration; import java.io.File; import java.io.IOException; import org.testng.annotations.Test; import okhttp3.MediaType; import okhttp3.MultipartBody; import okhttp3.OkHttpClient; import okhttp3.Request; import okhttp3.RequestBody; import okhttp3.Response; public class CalliopeAPI { @Test public void calliopeAPIcall() { final MediaType MEDIA_TYPE_XML = MediaType.parse("application/xml"); // for xml files final MediaType MEDIA_TYPE_JSON = MediaType.parse("application/json"); // for json files MultipartBody requestBody = null; try { String report_filename = "C:\\Users\\Lenovo-I7\\git\\Calliope-Integration-with-LambdaTest\\CalliopeSample\\test-output\\testng-results.xml"; // Result File Address requestBody = new MultipartBody.Builder() .setType(MultipartBody.FORM) .addFormDataPart("file", report_filename, RequestBody.create(MEDIA_TYPE_XML, new File(report_filename))) .build(); } catch (Exception e1) { e1.printStackTrace(); } OkHttpClient client = new OkHttpClient(); String endpoint_url = "https://app.calliope.pro/api/v2/profile/577/report/import?os=myos&platform=myplatform&build=mybuild"; // Calliope API final String API_KEY = "ZDk4ZGVhM2VlMzRlYjlkZGI0Y2MxZTA4Yjg1OTYxNjUyMzQzMGZhZmE0NTY0MTk4Y2MyMmM0NGQ3OTlmNTk2N2Jm"; //Calliope AccessKey Request request = new Request.Builder().url(endpoint_url).post(requestBody).addHeader("x-api-key", API_KEY).build(); // POST request Response response = null; try { response = client.newCall(request).execute(); System.out.println("============="); System.out.println(response); String response_body = response.body().string(); System.out.println(response_body); System.out.println("============="); if (response.isSuccessful()){ System.out.println("created"); } else { throw new IOException("Unexpected HTTP code " + response); } } catch (IOException e) { e.printStackTrace(); } } } |
To make sure that CalliopeAPI.java file is getting executed after BaseTest.java file finishes its execution, we have used TestNG Listeners for this. Just for the recap, the BaseTest.java file includes the test configurations, test cases and the procedure for calling the LambdaTest GET session API. So, unless the execution of BaseTest.java doesn’t end up we cannot call CalliopeAPI.java, this is the reason that we have used IExecutionListener which provides two methods and onExecutionFinish().
- Method onExecutionStart() gets invoked before TestNG run starts.
- Method onExecutionFinish() gets invoked when all the suites have been run.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
IntegrationExecution.java package calliopeIntegration; import org.testng.IExecutionListener; public class IntegrationExecution implements IExecutionListener { public void onExecutionStart() { System.out.println("Fetching LambdaTest Automation Test Data and Sending to Calliope Dashboard"); } public void onExecutionFinish() { CalliopeAPI object = new CalliopeAPI(); object.calliopeAPIcall(); // Calling Calliope API after test execution on LambdaTest } } |
Since we are retrieving data from LambdaTest API whose schema is in JSON format, so we need to set up and organize the test session retrieving data using JsonPropertyOrder.
Below is the code which would show you the hierarchy or list in which the test session data would get organized.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 |
SessionData.java package calliopeIntegration.session; import com.fasterxml.jackson.annotation.JsonInclude; import com.fasterxml.jackson.annotation.JsonProperty; import com.fasterxml.jackson.annotation.JsonPropertyOrder; @JsonInclude(JsonInclude.Include.NON_NULL) @JsonPropertyOrder({ "name", "duration", "platform", "browser", "browser_version", "device", "status_ind", "session_id", "build_name", "console_logs_url", "network_logs_url", "command_logs_url", "selenium_logs_url", "video_url", "screenshot_url" }) public class SessionData { @JsonProperty("name") private String name; @JsonProperty("duration") private Integer duration; @JsonProperty("platform") private String platform; @JsonProperty("browser") private String browser; @JsonProperty("browser_version") private String browserVersion; @JsonProperty("device") private String device; @JsonProperty("status_ind") private String statusInd; @JsonProperty("session_id") private String sessionId; @JsonProperty("build_name") private String buildName; @JsonProperty("console_logs_url") private String consoleLogsUrl; @JsonProperty("network_logs_url") private String networkLogsUrl; @JsonProperty("command_logs_url") private String commandLogsUrl; @JsonProperty("selenium_logs_url") private String seleniumLogsUrl; @JsonProperty("video_url") private String videoUrl; @JsonProperty("screenshot_url") private String screenshotUrl; @JsonProperty("name") public String getName() { return name; } @JsonProperty("name") public void setName(String name) { this.name = name; } @JsonProperty("duration") public Integer getDuration() { return duration; } @JsonProperty("duration") public void setDuration(Integer duration) { this.duration = duration; } @JsonProperty("platform") public String getPlatform() { return platform; } @JsonProperty("platform") public void setPlatform(String platform) { this.platform = platform; } @JsonProperty("browser") public String getBrowser() { return browser; } @JsonProperty("browser") public void setBrowser(String browser) { this.browser = browser; } @JsonProperty("browser_version") public String getBrowserVersion() { return browserVersion; } @JsonProperty("browser_version") public void setBrowserVersion(String browserVersion) { this.browserVersion = browserVersion; } @JsonProperty("device") public String getDevice() { return device; } @JsonProperty("device") public void setDevice(String device) { this.device = device; } @JsonProperty("status_ind") public String getStatusInd() { return statusInd; } @JsonProperty("status_ind") public void setStatusInd(String statusInd) { this.statusInd = statusInd; } @JsonProperty("session_id") public String getSessionId() { return sessionId; } @JsonProperty("session_id") public void setSessionId(String sessionId) { this.sessionId = sessionId; } @JsonProperty("build_name") public String getBuildName() { return buildName; } @JsonProperty("build_name") public void setBuildName(String buildName) { this.buildName = buildName; } @JsonProperty("console_logs_url") public String getConsoleLogsUrl() { return consoleLogsUrl; } @JsonProperty("console_logs_url") public void setConsoleLogsUrl(String consoleLogsUrl) { this.consoleLogsUrl = consoleLogsUrl; } @JsonProperty("network_logs_url") public String getNetworkLogsUrl() { return networkLogsUrl; } @JsonProperty("network_logs_url") public void setNetworkLogsUrl(String networkLogsUrl) { this.networkLogsUrl = networkLogsUrl; } @JsonProperty("command_logs_url") public String getCommandLogsUrl() { return commandLogsUrl; } @JsonProperty("command_logs_url") public void setCommandLogsUrl(String commandLogsUrl) { this.commandLogsUrl = commandLogsUrl; } @JsonProperty("selenium_logs_url") public String getSeleniumLogsUrl() { return seleniumLogsUrl; } @JsonProperty("selenium_logs_url") public void setSeleniumLogsUrl(String seleniumLogsUrl) { this.seleniumLogsUrl = seleniumLogsUrl; } @JsonProperty("video_url") public String getVideoUrl() { return videoUrl; } @JsonProperty("video_url") public void setVideoUrl(String videoUrl) { this.videoUrl = videoUrl; } @JsonProperty("screenshot_url") public String getScreenshotUrl() { return screenshotUrl; } @JsonProperty("screenshot_url") public void setScreenshotUrl(String screenshotUrl) { this.screenshotUrl = screenshotUrl; } @Override public String toString() { StringBuilder builder = new StringBuilder(); builder.append("{\nname="); builder.append(name); builder.append(",\nduration="); builder.append(duration); builder.append(",\nplatform="); builder.append(platform); builder.append(",\nbrowser="); builder.append(browser); builder.append(",\nbrowserVersion="); builder.append(browserVersion); builder.append(",\ndevice="); builder.append(device); builder.append(",\nstatusInd="); builder.append(statusInd); builder.append(",\nsessionId="); builder.append(sessionId); builder.append(",\nbuildName="); builder.append(buildName); builder.append(",\nconsoleLogsUrl="); builder.append(consoleLogsUrl); builder.append(",\nnetworkLogsUrl="); builder.append(networkLogsUrl); builder.append(",\ncommandLogsUrl="); builder.append(commandLogsUrl); builder.append(",\nseleniumLogsUrl="); builder.append(seleniumLogsUrl); builder.append(",\nvideoUrl="); builder.append(videoUrl); builder.append(",\nscreenshotUrl="); builder.append(screenshotUrl); builder.append("\n}"); return builder.toString(); } } |
In SessionData.java only session data variables are declared, though they are defined in a hierarchical manner but they do not contain any values yet. We have now created a new class SessionResponse.java that would set values to the previously defined session data variables.SessionResponse.java class has been called by BaseTest.java class for setting the values for session data variables after retrieving from LambdaTest GET session API.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 |
SessionResponse.java package calliopeIntegration.session; import com.fasterxml.jackson.annotation.JsonInclude; import com.fasterxml.jackson.annotation.JsonProperty; @JsonInclude(JsonInclude.Include.NON_NULL) public class SessionResponse { @JsonProperty("data") private SessionData sessionData; @JsonProperty("message") private String message; @JsonProperty("status") private String status; @JsonProperty("data") public SessionData getData() { return sessionData; } @JsonProperty("data") public void setData(SessionData sessionData) { this.sessionData = sessionData; } @JsonProperty("message") public String getMessage() { return message; } @JsonProperty("message") public void setMessage(String message) { this.message = message; } @JsonProperty("status") public String getStatus() { return status; } @JsonProperty("status") public void setStatus(String status) { this.status = status; } @Override public String toString() { StringBuilder builder = new StringBuilder(); builder.append("{\ndata:"); builder.append(sessionData); builder.append(", \nmessage:"); builder.append(message); builder.append(", \nstatus:"); builder.append(status); builder.append("\n}"); return builder.toString(); } } |
Now, if you look at the Calliope dashboard you will be able to notice a screenshot below, representing the executions of your automation test script at LambdaTest Selenium Grid for automated cross browser testing.
How Was That?
Kudos! You have successfully imported your automation test results of the Selenium Grid offered by LambdaTest on your Calliope dashboard. Now, you can effortlessly collaborate your teammates while performing cross browser testing with LambdaTest. Let me know your thoughts on this integration and how it helped to fast track your test cycles. Stay tuned for more articles by hitting the notification bell at the bottom. Cheers and happy testing! 🙂
Got Questions? Drop them on LambdaTest Community. Visit now