How to use someothertest method in Lemoncheesecake
Best Python code snippet using lemoncheesecake
gtest_utils_test.py
Source:gtest_utils_test.py 
1#!/usr/bin/env python2# Copyright (c) 2012 The Chromium Authors. All rights reserved.3# Use of this source code is governed by a BSD-style license that can be4# found in the LICENSE file.5"""Unit tests for classes in gtest_command.py."""6import unittest7import test_env # pylint: disable=W06118from common import gtest_utils9FAILURES = ['NavigationControllerTest.Reload',10 'NavigationControllerTest/SpdyNetworkTransTest.Constructor/0',11 'BadTest.TimesOut',12 'MoreBadTest.TimesOutAndFails',13 'SomeOtherTest.SwitchTypes',14 'SomeOtherTest.FAILS_ThisTestTimesOut']15FAILS_FAILURES = ['SomeOtherTest.FAILS_Bar']16FLAKY_FAILURES = ['SomeOtherTest.FLAKY_Baz']17TIMEOUT_MESSAGE = 'Killed (timed out).'18RELOAD_ERRORS = ('C:\b\slave\chrome-release-snappy\build\chrome\browser'19'\navigation_controller_unittest.cc:381: Failure' + """20Value of: -121Expected: contents->controller()->GetPendingEntryIndex()22Which is: 023""")24SPDY_ERRORS = ('C:\b\slave\chrome-release-snappy\build\chrome\browser'25'\navigation_controller_unittest.cc:439: Failure' + """26Value of: -127Expected: contents->controller()->GetPendingEntryIndex()28Which is: 029""")30SWITCH_ERRORS = ('C:\b\slave\chrome-release-snappy\build\chrome\browser'31'\navigation_controller_unittest.cc:615: Failure' + """32Value of: -133Expected: contents->controller()->GetPendingEntryIndex()34Which is: 035""" + 'C:\b\slave\chrome-release-snappy\build\chrome\browser'36'\navigation_controller_unittest.cc:617: Failure' + """37Value of: contents->controller()->GetPendingEntry()38 Actual: true39Expected: false40""")41TIMEOUT_ERRORS = ('[61613:263:0531/042613:2887943745568888:ERROR:/b/slave'42'/chromium-rel-mac-builder/build/src/chrome/browser/extensions'43'/extension_error_reporter.cc(56)] Extension error: Could not load extension '44'from \'extensions/api_test/geolocation/no_permission\'. Manifest file is '45'missing or unreadable.')46MOREBAD_ERRORS = """47Value of: entry->page_type()48 Actual: 249Expected: NavigationEntry::NORMAL_PAGE50"""51TEST_DATA = ("""52[==========] Running 7 tests from 3 test cases.53[----------] Global test environment set-up.54[----------] 1 test from HunspellTest55[ RUN ] HunspellTest.All56[ OK ] HunspellTest.All (62 ms)57[----------] 1 test from HunspellTest (62 ms total)58[----------] 4 tests from NavigationControllerTest59[ RUN ] NavigationControllerTest.Defaults60[ OK ] NavigationControllerTest.Defaults (48 ms)61[ RUN ] NavigationControllerTest.Reload62%(reload_errors)s63[ FAILED ] NavigationControllerTest.Reload (2 ms)64[ RUN ] NavigationControllerTest.Reload_GeneratesNewPage65[ OK ] NavigationControllerTest.Reload_GeneratesNewPage (22 ms)66[ RUN ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/067%(spdy_errors)s68[ FAILED ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0 (2 ms)69[----------] 4 tests from NavigationControllerTest (74 ms total)70 YOU HAVE 2 FLAKY TESTS71[----------] 1 test from BadTest72[ RUN ] BadTest.TimesOut73%(timeout_errors)s74""" % {'reload_errors': RELOAD_ERRORS,75 'spdy_errors': SPDY_ERRORS,76 'timeout_errors': TIMEOUT_ERRORS} +77'[0531/042642:ERROR:/b/slave/chromium-rel-mac-builder/build/src/chrome'78'/test/test_launcher/out_of_proc_test_runner.cc(79)] Test timeout (30000 ms) '79'exceeded for BadTest.TimesOut' + """80Handling SIGTERM.81Successfully wrote to shutdown pipe, resetting signal handler.82""" +83'[61613:19971:0531/042642:2887973024284693:INFO:/b/slave/chromium-rel-mac-'84'builder/build/src/chrome/browser/browser_main.cc(285)] Handling shutdown for '85'signal 15.' + """86[----------] 1 test from MoreBadTest87[ RUN ] MoreBadTest.TimesOutAndFails88%(morebad_errors)s89""" % {'morebad_errors': MOREBAD_ERRORS} +90'[0531/042642:ERROR:/b/slave/chromium-rel-mac-builder/build/src/chrome/test'91'/test_launcher/out_of_proc_test_runner.cc(79)] Test timeout (30000 ms) '92'exceeded for MoreBadTest.TimesOutAndFails' + """93Handling SIGTERM.94Successfully wrote to shutdown pipe, resetting signal handler.95[ FAILED ] MoreBadTest.TimesOutAndFails (31000 ms)96[----------] 5 tests from SomeOtherTest97[ RUN ] SomeOtherTest.SwitchTypes98%(switch_errors)s99[ FAILED ] SomeOtherTest.SwitchTypes (40 ms)100[ RUN ] SomeOtherTest.Foo101[ OK ] SomeOtherTest.Foo (20 ms)102[ RUN ] SomeOtherTest.FAILS_Bar103Some error message for a failing test.104[ FAILED ] SomeOtherTest.FAILS_Bar (40 ms)105[ RUN ] SomeOtherTest.FAILS_ThisTestTimesOut106""" % {'switch_errors' : SWITCH_ERRORS} +107'[0521/041343:ERROR:test_launcher.cc(384)] Test timeout (5000 ms) '108'exceeded for SomeOtherTest.FAILS_ThisTestTimesOut' + """109[ RUN ] SomeOtherTest.FLAKY_Baz110Some error message for a flaky test.111[ FAILED ] SomeOtherTest.FLAKY_Baz (40 ms)112[----------] 2 tests from SomeOtherTest (60 ms total)113[----------] Global test environment tear-down114[==========] 8 tests from 3 test cases ran. (3750 ms total)115[ PASSED ] 4 tests.116[ FAILED ] 4 tests, listed below:117[ FAILED ] NavigationControllerTest.Reload118[ FAILED ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0119[ FAILED ] SomeOtherTest.SwitchTypes120[ FAILED ] SomeOtherTest.FAILS_ThisTestTimesOut121 1 FAILED TEST122 YOU HAVE 10 DISABLED TESTS123 YOU HAVE 2 FLAKY TESTS124program finished with exit code 1125""")126TEST_DATA_CRASH = """127[==========] Running 7 tests from 3 test cases.128[----------] Global test environment set-up.129[----------] 1 test from HunspellTest130[ RUN ] HunspellTest.Crashes131Oops, this test crashed!132"""133VALGRIND_HASH = 'B254345E4D3B6A00'134VALGRIND_SUPPRESSION = """Suppression (error hash=#%(hash)s#):135{136 <insert_a_suppression_name_here>137 Memcheck:Leak138 fun:_Znw*139 fun:_ZN31NavigationControllerTest_Reload8TestBodyEv140}""" % {'hash' : VALGRIND_HASH}141TEST_DATA_VALGRIND = """142[==========] Running 5 tests from 2 test cases.143[----------] Global test environment set-up.144[----------] 1 test from HunspellTest145[ RUN ] HunspellTest.All146[ OK ] HunspellTest.All (62 ms)147[----------] 1 test from HunspellTest (62 ms total)148[----------] 4 tests from NavigationControllerTest149[ RUN ] NavigationControllerTest.Defaults150[ OK ] NavigationControllerTest.Defaults (48 ms)151[ RUN ] NavigationControllerTest.Reload152[ OK ] NavigationControllerTest.Reload (2 ms)153[ RUN ] NavigationControllerTest.Reload_GeneratesNewPage154[ OK ] NavigationControllerTest.Reload_GeneratesNewPage (22 ms)155[ RUN ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0156[ OK ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0 (2 ms)157[----------] 4 tests from NavigationControllerTest (74 ms total)158[----------] Global test environment tear-down159[==========] 5 tests from 1 test cases ran. (136 ms total)160[ PASSED ] 5 tests.161%(suppression)s162program finished with exit code 255163""" % {'suppression': VALGRIND_SUPPRESSION}164FAILING_TESTS_OUTPUT = """165Failing tests:166ChromeRenderViewTest.FAILS_AllowDOMStorage167PrerenderBrowserTest.PrerenderHTML5VideoJs168"""169FAILING_TESTS_EXPECTED = ['ChromeRenderViewTest.FAILS_AllowDOMStorage',170 'PrerenderBrowserTest.PrerenderHTML5VideoJs']171TEST_DATA_SHARD_0 = ("""Note: This is test shard 1 of 30.172[==========] Running 6 tests from 3 test cases.173[----------] Global test environment set-up.174[----------] 1 test from HunspellTest175[ RUN ] HunspellTest.All176[ OK ] HunspellTest.All (62 ms)177[----------] 1 test from HunspellTest (62 ms total)178[----------] 1 test from BadTest179[ RUN ] BadTest.TimesOut180%(timeout_errors)s181""" % {'timeout_errors': TIMEOUT_ERRORS} +182'[0531/042642:ERROR:/b/slave/chromium-rel-mac-builder/build/src/chrome/test'183'/test_launcher/out_of_proc_test_runner.cc(79)] Test timeout (30000 ms) '184'exceeded for BadTest.TimesOut' + """185Handling SIGTERM.186Successfully wrote to shutdown pipe, resetting signal handler.187""" +188'[61613:19971:0531/042642:2887973024284693:INFO:/b/slave/chromium-rel-mac-'189'builder/build/src/chrome/browser/browser_main.cc(285)] Handling shutdown for '190'signal 15.' + """191[----------] 4 tests from SomeOtherTest192[ RUN ] SomeOtherTest.SwitchTypes193%(switch_errors)s194[ FAILED ] SomeOtherTest.SwitchTypes (40 ms)195[ RUN ] SomeOtherTest.Foo196[ OK ] SomeOtherTest.Foo (20 ms)197[ RUN ] SomeOtherTest.FAILS_Bar198Some error message for a failing test.199[ FAILED ] SomeOtherTest.FAILS_Bar (40 ms)200[ RUN ] SomeOtherTest.FAILS_ThisTestTimesOut201""" % {'switch_errors' : SWITCH_ERRORS} +202'[0521/041343:ERROR:test_launcher.cc(384)] Test timeout (5000 ms) exceeded '203'for SomeOtherTest.FAILS_ThisTestTimesOut' + """204[ RUN ] SomeOtherTest.FLAKY_Baz205Some error message for a flaky test.206[ FAILED ] SomeOtherTest.FLAKY_Baz (40 ms)207[----------] 2 tests from SomeOtherTest (60 ms total)208[----------] Global test environment tear-down209[==========] 7 tests from 3 test cases ran. (3750 ms total)210[ PASSED ] 5 tests.211[ FAILED ] 2 test, listed below:212[ FAILED ] SomeOtherTest.SwitchTypes213[ FAILED ] SomeOtherTest.FAILS_ThisTestTimesOut214 1 FAILED TEST215 YOU HAVE 10 DISABLED TESTS216 YOU HAVE 2 FLAKY TESTS217""")218TEST_DATA_SHARD_1 = ("""Note: This is test shard 13 of 30.219[==========] Running 5 tests from 2 test cases.220[----------] Global test environment set-up.221[----------] 4 tests from NavigationControllerTest222[ RUN ] NavigationControllerTest.Defaults223[ OK ] NavigationControllerTest.Defaults (48 ms)224[ RUN ] NavigationControllerTest.Reload225%(reload_errors)s226[ FAILED ] NavigationControllerTest.Reload (2 ms)227[ RUN ] NavigationControllerTest.Reload_GeneratesNewPage228[ OK ] NavigationControllerTest.Reload_GeneratesNewPage (22 ms)229[ RUN ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0230%(spdy_errors)s231""" % {'reload_errors' : RELOAD_ERRORS,232 'spdy_errors' : SPDY_ERRORS} +233'[ FAILED ] NavigationControllerTest/SpdyNetworkTransTest.Constructor'234'/0 (2 ms)' + """235[----------] 4 tests from NavigationControllerTest (74 ms total)236 YOU HAVE 2 FLAKY TESTS237[----------] 1 test from MoreBadTest238[ RUN ] MoreBadTest.TimesOutAndFails239%(morebad_errors)s240""" % {'morebad_errors': MOREBAD_ERRORS} +241'[0531/042642:ERROR:/b/slave/chromium-rel-mac-builder/build/src/chrome/test'242'/test_launcher/out_of_proc_test_runner.cc(79)] Test timeout (30000 ms) '243'exceeded for MoreBadTest.TimesOutAndFails' + """244Handling SIGTERM.245Successfully wrote to shutdown pipe, resetting signal handler.246[ FAILED ] MoreBadTest.TimesOutAndFails (31000 ms)247[----------] Global test environment tear-down248[==========] 5 tests from 2 test cases ran. (3750 ms total)249[ PASSED ] 3 tests.250[ FAILED ] 2 tests, listed below:251[ FAILED ] NavigationControllerTest.Reload252[ FAILED ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0253 1 FAILED TEST254 YOU HAVE 10 DISABLED TESTS255 YOU HAVE 2 FLAKY TESTS256""")257TEST_DATA_SHARD_EXIT = 'program finished with exit code '258TEST_DATA_CRASH_SHARD = """Note: This is test shard 5 of 5.259[==========] Running 7 tests from 3 test cases.260[----------] Global test environment set-up.261[----------] 1 test from HunspellTest262[ RUN ] HunspellTest.Crashes263Oops, this test crashed!"""264class TestGTestLogParserTests(unittest.TestCase):265 def testGTestLogParserNoSharing(self):266 # Tests for log parsing without sharding.267 parser = gtest_utils.GTestLogParser()268 for line in TEST_DATA.splitlines():269 parser.ProcessLine(line)270 self.assertEqual(0, len(parser.ParsingErrors()))271 self.assertFalse(parser.RunningTests())272 self.assertEqual(sorted(FAILURES), sorted(parser.FailedTests()))273 self.assertEqual(sorted(FAILURES + FAILS_FAILURES),274 sorted(parser.FailedTests(include_fails=True)))275 self.assertEqual(sorted(FAILURES + FLAKY_FAILURES),276 sorted(parser.FailedTests(include_flaky=True)))277 self.assertEqual(sorted(FAILURES + FAILS_FAILURES + FLAKY_FAILURES),278 sorted(parser.FailedTests(include_fails=True, include_flaky=True)))279 self.assertEqual(10, parser.DisabledTests())280 self.assertEqual(2, parser.FlakyTests())281 test_name = 'NavigationControllerTest.Reload'282 self.assertEqual('\n'.join(['%s: ' % test_name, RELOAD_ERRORS]),283 '\n'.join(parser.FailureDescription(test_name)))284 test_name = 'NavigationControllerTest/SpdyNetworkTransTest.Constructor/0'285 self.assertEqual('\n'.join(['%s: ' % test_name, SPDY_ERRORS]),286 '\n'.join(parser.FailureDescription(test_name)))287 test_name = 'SomeOtherTest.SwitchTypes'288 self.assertEqual('\n'.join(['%s: ' % test_name, SWITCH_ERRORS]),289 '\n'.join(parser.FailureDescription(test_name)))290 test_name = 'BadTest.TimesOut'291 self.assertEqual('\n'.join(['%s: ' % test_name,292 TIMEOUT_ERRORS, TIMEOUT_MESSAGE]),293 '\n'.join(parser.FailureDescription(test_name)))294 test_name = 'MoreBadTest.TimesOutAndFails'295 self.assertEqual('\n'.join(['%s: ' % test_name,296 MOREBAD_ERRORS, TIMEOUT_MESSAGE]),297 '\n'.join(parser.FailureDescription(test_name)))298 parser = gtest_utils.GTestLogParser()299 for line in TEST_DATA_CRASH.splitlines():300 parser.ProcessLine(line)301 self.assertEqual(0, len(parser.ParsingErrors()))302 self.assertTrue(parser.RunningTests())303 self.assertEqual(['HunspellTest.Crashes'], parser.FailedTests())304 self.assertEqual(0, parser.DisabledTests())305 self.assertEqual(0, parser.FlakyTests())306 test_name = 'HunspellTest.Crashes'307 self.assertEqual('\n'.join(['%s: ' % test_name, 'Did not complete.']),308 '\n'.join(parser.FailureDescription(test_name)))309 def testGTestLogParserSharing(self):310 # Same tests for log parsing with sharding_supervisor.311 parser = gtest_utils.GTestLogParser()312 test_data_shard = TEST_DATA_SHARD_0 + TEST_DATA_SHARD_1313 for line in test_data_shard.splitlines():314 parser.ProcessLine(line)315 parser.ProcessLine(TEST_DATA_SHARD_EXIT + '2')316 self.assertEqual(0, len(parser.ParsingErrors()))317 self.assertFalse(parser.RunningTests())318 self.assertEqual(sorted(FAILURES), sorted(parser.FailedTests()))319 self.assertEqual(sorted(FAILURES + FAILS_FAILURES),320 sorted(parser.FailedTests(include_fails=True)))321 self.assertEqual(sorted(FAILURES + FLAKY_FAILURES),322 sorted(parser.FailedTests(include_flaky=True)))323 self.assertEqual(sorted(324 FAILURES + FAILS_FAILURES + FLAKY_FAILURES),325 sorted(parser.FailedTests(include_fails=True, include_flaky=True)))326 self.assertEqual(10, parser.DisabledTests())327 self.assertEqual(2, parser.FlakyTests())328 test_name = 'NavigationControllerTest.Reload'329 self.assertEqual('\n'.join(['%s: ' % test_name, RELOAD_ERRORS]),330 '\n'.join(parser.FailureDescription(test_name)))331 test_name = (332 'NavigationControllerTest/SpdyNetworkTransTest.Constructor/0')333 self.assertEqual('\n'.join(['%s: ' % test_name, SPDY_ERRORS]),334 '\n'.join(parser.FailureDescription(test_name)))335 test_name = 'SomeOtherTest.SwitchTypes'336 self.assertEqual('\n'.join(['%s: ' % test_name, SWITCH_ERRORS]),337 '\n'.join(parser.FailureDescription(test_name)))338 test_name = 'BadTest.TimesOut'339 self.assertEqual(340 '\n'.join(['%s: ' % test_name,341 TIMEOUT_ERRORS, TIMEOUT_MESSAGE]),342 '\n'.join(parser.FailureDescription(test_name)))343 test_name = 'MoreBadTest.TimesOutAndFails'344 self.assertEqual(345 '\n'.join(['%s: ' % test_name,346 MOREBAD_ERRORS, TIMEOUT_MESSAGE]),347 '\n'.join(parser.FailureDescription(test_name)))348 parser = gtest_utils.GTestLogParser()349 for line in TEST_DATA_CRASH.splitlines():350 parser.ProcessLine(line)351 self.assertEqual(0, len(parser.ParsingErrors()))352 self.assertTrue(parser.RunningTests())353 self.assertEqual(['HunspellTest.Crashes'], parser.FailedTests())354 self.assertEqual(0, parser.DisabledTests())355 self.assertEqual(0, parser.FlakyTests())356 test_name = 'HunspellTest.Crashes'357 self.assertEqual('\n'.join(['%s: ' % test_name, 'Did not complete.']),358 '\n'.join(parser.FailureDescription(test_name)))359 def testGTestLogParserValgrind(self):360 parser = gtest_utils.GTestLogParser()361 for line in TEST_DATA_VALGRIND.splitlines():362 parser.ProcessLine(line)363 self.assertEqual(0, len(parser.ParsingErrors()))364 self.assertFalse(parser.RunningTests())365 self.assertFalse(parser.FailedTests())366 self.assertEqual([VALGRIND_HASH], parser.SuppressionHashes())367 self.assertEqual(VALGRIND_SUPPRESSION,368 '\n'.join(parser.Suppression(VALGRIND_HASH)))369 parser = gtest_utils.GTestLogParser()370 for line in FAILING_TESTS_OUTPUT.splitlines():371 parser.ProcessLine(line)372 self.assertEqual(FAILING_TESTS_EXPECTED,373 parser.FailedTests(True, True))374if __name__ == '__main__':...
gtest_utils_test.py.svn-base
Source:gtest_utils_test.py.svn-base
1#!/usr/bin/env python2# Copyright (c) 2012 The Chromium Authors. All rights reserved.3# Use of this source code is governed by a BSD-style license that can be4# found in the LICENSE file.5"""Unit tests for classes in gtest_command.py."""6import unittest7import test_env # pylint: disable=W06118from common import gtest_utils9FAILURES = ['NavigationControllerTest.Reload',10 'NavigationControllerTest/SpdyNetworkTransTest.Constructor/0',11 'BadTest.TimesOut',12 'MoreBadTest.TimesOutAndFails',13 'SomeOtherTest.SwitchTypes',14 'SomeOtherTest.FAILS_ThisTestTimesOut']15FAILS_FAILURES = ['SomeOtherTest.FAILS_Bar']16FLAKY_FAILURES = ['SomeOtherTest.FLAKY_Baz']17TIMEOUT_MESSAGE = 'Killed (timed out).'18RELOAD_ERRORS = ('C:\b\slave\chrome-release-snappy\build\chrome\browser'19'\navigation_controller_unittest.cc:381: Failure' + """20Value of: -121Expected: contents->controller()->GetPendingEntryIndex()22Which is: 023""")24SPDY_ERRORS = ('C:\b\slave\chrome-release-snappy\build\chrome\browser'25'\navigation_controller_unittest.cc:439: Failure' + """26Value of: -127Expected: contents->controller()->GetPendingEntryIndex()28Which is: 029""")30SWITCH_ERRORS = ('C:\b\slave\chrome-release-snappy\build\chrome\browser'31'\navigation_controller_unittest.cc:615: Failure' + """32Value of: -133Expected: contents->controller()->GetPendingEntryIndex()34Which is: 035""" + 'C:\b\slave\chrome-release-snappy\build\chrome\browser'36'\navigation_controller_unittest.cc:617: Failure' + """37Value of: contents->controller()->GetPendingEntry()38 Actual: true39Expected: false40""")41TIMEOUT_ERRORS = ('[61613:263:0531/042613:2887943745568888:ERROR:/b/slave'42'/chromium-rel-mac-builder/build/src/chrome/browser/extensions'43'/extension_error_reporter.cc(56)] Extension error: Could not load extension '44'from \'extensions/api_test/geolocation/no_permission\'. Manifest file is '45'missing or unreadable.')46MOREBAD_ERRORS = """47Value of: entry->page_type()48 Actual: 249Expected: NavigationEntry::NORMAL_PAGE50"""51TEST_DATA = ("""52[==========] Running 7 tests from 3 test cases.53[----------] Global test environment set-up.54[----------] 1 test from HunspellTest55[ RUN ] HunspellTest.All56[ OK ] HunspellTest.All (62 ms)57[----------] 1 test from HunspellTest (62 ms total)58[----------] 4 tests from NavigationControllerTest59[ RUN ] NavigationControllerTest.Defaults60[ OK ] NavigationControllerTest.Defaults (48 ms)61[ RUN ] NavigationControllerTest.Reload62%(reload_errors)s63[ FAILED ] NavigationControllerTest.Reload (2 ms)64[ RUN ] NavigationControllerTest.Reload_GeneratesNewPage65[ OK ] NavigationControllerTest.Reload_GeneratesNewPage (22 ms)66[ RUN ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/067%(spdy_errors)s68[ FAILED ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0 (2 ms)69[----------] 4 tests from NavigationControllerTest (74 ms total)70 YOU HAVE 2 FLAKY TESTS71[----------] 1 test from BadTest72[ RUN ] BadTest.TimesOut73%(timeout_errors)s74""" % {'reload_errors': RELOAD_ERRORS,75 'spdy_errors': SPDY_ERRORS,76 'timeout_errors': TIMEOUT_ERRORS} +77'[0531/042642:ERROR:/b/slave/chromium-rel-mac-builder/build/src/chrome'78'/test/test_launcher/out_of_proc_test_runner.cc(79)] Test timeout (30000 ms) '79'exceeded for BadTest.TimesOut' + """80Handling SIGTERM.81Successfully wrote to shutdown pipe, resetting signal handler.82""" +83'[61613:19971:0531/042642:2887973024284693:INFO:/b/slave/chromium-rel-mac-'84'builder/build/src/chrome/browser/browser_main.cc(285)] Handling shutdown for '85'signal 15.' + """86[----------] 1 test from MoreBadTest87[ RUN ] MoreBadTest.TimesOutAndFails88%(morebad_errors)s89""" % {'morebad_errors': MOREBAD_ERRORS} +90'[0531/042642:ERROR:/b/slave/chromium-rel-mac-builder/build/src/chrome/test'91'/test_launcher/out_of_proc_test_runner.cc(79)] Test timeout (30000 ms) '92'exceeded for MoreBadTest.TimesOutAndFails' + """93Handling SIGTERM.94Successfully wrote to shutdown pipe, resetting signal handler.95[ FAILED ] MoreBadTest.TimesOutAndFails (31000 ms)96[----------] 5 tests from SomeOtherTest97[ RUN ] SomeOtherTest.SwitchTypes98%(switch_errors)s99[ FAILED ] SomeOtherTest.SwitchTypes (40 ms)100[ RUN ] SomeOtherTest.Foo101[ OK ] SomeOtherTest.Foo (20 ms)102[ RUN ] SomeOtherTest.FAILS_Bar103Some error message for a failing test.104[ FAILED ] SomeOtherTest.FAILS_Bar (40 ms)105[ RUN ] SomeOtherTest.FAILS_ThisTestTimesOut106""" % {'switch_errors' : SWITCH_ERRORS} +107'[0521/041343:ERROR:test_launcher.cc(384)] Test timeout (5000 ms) '108'exceeded for SomeOtherTest.FAILS_ThisTestTimesOut' + """109[ RUN ] SomeOtherTest.FLAKY_Baz110Some error message for a flaky test.111[ FAILED ] SomeOtherTest.FLAKY_Baz (40 ms)112[----------] 2 tests from SomeOtherTest (60 ms total)113[----------] Global test environment tear-down114[==========] 8 tests from 3 test cases ran. (3750 ms total)115[ PASSED ] 4 tests.116[ FAILED ] 4 tests, listed below:117[ FAILED ] NavigationControllerTest.Reload118[ FAILED ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0119[ FAILED ] SomeOtherTest.SwitchTypes120[ FAILED ] SomeOtherTest.FAILS_ThisTestTimesOut121 1 FAILED TEST122 YOU HAVE 10 DISABLED TESTS123 YOU HAVE 2 FLAKY TESTS124program finished with exit code 1125""")126TEST_DATA_CRASH = """127[==========] Running 7 tests from 3 test cases.128[----------] Global test environment set-up.129[----------] 1 test from HunspellTest130[ RUN ] HunspellTest.Crashes131Oops, this test crashed!132"""133VALGRIND_HASH = 'B254345E4D3B6A00'134VALGRIND_SUPPRESSION = """Suppression (error hash=#%(hash)s#):135{136 <insert_a_suppression_name_here>137 Memcheck:Leak138 fun:_Znw*139 fun:_ZN31NavigationControllerTest_Reload8TestBodyEv140}""" % {'hash' : VALGRIND_HASH}141TEST_DATA_VALGRIND = """142[==========] Running 5 tests from 2 test cases.143[----------] Global test environment set-up.144[----------] 1 test from HunspellTest145[ RUN ] HunspellTest.All146[ OK ] HunspellTest.All (62 ms)147[----------] 1 test from HunspellTest (62 ms total)148[----------] 4 tests from NavigationControllerTest149[ RUN ] NavigationControllerTest.Defaults150[ OK ] NavigationControllerTest.Defaults (48 ms)151[ RUN ] NavigationControllerTest.Reload152[ OK ] NavigationControllerTest.Reload (2 ms)153[ RUN ] NavigationControllerTest.Reload_GeneratesNewPage154[ OK ] NavigationControllerTest.Reload_GeneratesNewPage (22 ms)155[ RUN ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0156[ OK ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0 (2 ms)157[----------] 4 tests from NavigationControllerTest (74 ms total)158[----------] Global test environment tear-down159[==========] 5 tests from 1 test cases ran. (136 ms total)160[ PASSED ] 5 tests.161%(suppression)s162program finished with exit code 255163""" % {'suppression': VALGRIND_SUPPRESSION}164FAILING_TESTS_OUTPUT = """165Failing tests:166ChromeRenderViewTest.FAILS_AllowDOMStorage167PrerenderBrowserTest.PrerenderHTML5VideoJs168"""169FAILING_TESTS_EXPECTED = ['ChromeRenderViewTest.FAILS_AllowDOMStorage',170 'PrerenderBrowserTest.PrerenderHTML5VideoJs']171TEST_DATA_SHARD_0 = ("""Note: This is test shard 1 of 30.172[==========] Running 6 tests from 3 test cases.173[----------] Global test environment set-up.174[----------] 1 test from HunspellTest175[ RUN ] HunspellTest.All176[ OK ] HunspellTest.All (62 ms)177[----------] 1 test from HunspellTest (62 ms total)178[----------] 1 test from BadTest179[ RUN ] BadTest.TimesOut180%(timeout_errors)s181""" % {'timeout_errors': TIMEOUT_ERRORS} +182'[0531/042642:ERROR:/b/slave/chromium-rel-mac-builder/build/src/chrome/test'183'/test_launcher/out_of_proc_test_runner.cc(79)] Test timeout (30000 ms) '184'exceeded for BadTest.TimesOut' + """185Handling SIGTERM.186Successfully wrote to shutdown pipe, resetting signal handler.187""" +188'[61613:19971:0531/042642:2887973024284693:INFO:/b/slave/chromium-rel-mac-'189'builder/build/src/chrome/browser/browser_main.cc(285)] Handling shutdown for '190'signal 15.' + """191[----------] 4 tests from SomeOtherTest192[ RUN ] SomeOtherTest.SwitchTypes193%(switch_errors)s194[ FAILED ] SomeOtherTest.SwitchTypes (40 ms)195[ RUN ] SomeOtherTest.Foo196[ OK ] SomeOtherTest.Foo (20 ms)197[ RUN ] SomeOtherTest.FAILS_Bar198Some error message for a failing test.199[ FAILED ] SomeOtherTest.FAILS_Bar (40 ms)200[ RUN ] SomeOtherTest.FAILS_ThisTestTimesOut201""" % {'switch_errors' : SWITCH_ERRORS} +202'[0521/041343:ERROR:test_launcher.cc(384)] Test timeout (5000 ms) exceeded '203'for SomeOtherTest.FAILS_ThisTestTimesOut' + """204[ RUN ] SomeOtherTest.FLAKY_Baz205Some error message for a flaky test.206[ FAILED ] SomeOtherTest.FLAKY_Baz (40 ms)207[----------] 2 tests from SomeOtherTest (60 ms total)208[----------] Global test environment tear-down209[==========] 7 tests from 3 test cases ran. (3750 ms total)210[ PASSED ] 5 tests.211[ FAILED ] 2 test, listed below:212[ FAILED ] SomeOtherTest.SwitchTypes213[ FAILED ] SomeOtherTest.FAILS_ThisTestTimesOut214 1 FAILED TEST215 YOU HAVE 10 DISABLED TESTS216 YOU HAVE 2 FLAKY TESTS217""")218TEST_DATA_SHARD_1 = ("""Note: This is test shard 13 of 30.219[==========] Running 5 tests from 2 test cases.220[----------] Global test environment set-up.221[----------] 4 tests from NavigationControllerTest222[ RUN ] NavigationControllerTest.Defaults223[ OK ] NavigationControllerTest.Defaults (48 ms)224[ RUN ] NavigationControllerTest.Reload225%(reload_errors)s226[ FAILED ] NavigationControllerTest.Reload (2 ms)227[ RUN ] NavigationControllerTest.Reload_GeneratesNewPage228[ OK ] NavigationControllerTest.Reload_GeneratesNewPage (22 ms)229[ RUN ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0230%(spdy_errors)s231""" % {'reload_errors' : RELOAD_ERRORS,232 'spdy_errors' : SPDY_ERRORS} +233'[ FAILED ] NavigationControllerTest/SpdyNetworkTransTest.Constructor'234'/0 (2 ms)' + """235[----------] 4 tests from NavigationControllerTest (74 ms total)236 YOU HAVE 2 FLAKY TESTS237[----------] 1 test from MoreBadTest238[ RUN ] MoreBadTest.TimesOutAndFails239%(morebad_errors)s240""" % {'morebad_errors': MOREBAD_ERRORS} +241'[0531/042642:ERROR:/b/slave/chromium-rel-mac-builder/build/src/chrome/test'242'/test_launcher/out_of_proc_test_runner.cc(79)] Test timeout (30000 ms) '243'exceeded for MoreBadTest.TimesOutAndFails' + """244Handling SIGTERM.245Successfully wrote to shutdown pipe, resetting signal handler.246[ FAILED ] MoreBadTest.TimesOutAndFails (31000 ms)247[----------] Global test environment tear-down248[==========] 5 tests from 2 test cases ran. (3750 ms total)249[ PASSED ] 3 tests.250[ FAILED ] 2 tests, listed below:251[ FAILED ] NavigationControllerTest.Reload252[ FAILED ] NavigationControllerTest/SpdyNetworkTransTest.Constructor/0253 1 FAILED TEST254 YOU HAVE 10 DISABLED TESTS255 YOU HAVE 2 FLAKY TESTS256""")257TEST_DATA_SHARD_EXIT = 'program finished with exit code '258TEST_DATA_CRASH_SHARD = """Note: This is test shard 5 of 5.259[==========] Running 7 tests from 3 test cases.260[----------] Global test environment set-up.261[----------] 1 test from HunspellTest262[ RUN ] HunspellTest.Crashes263Oops, this test crashed!"""264class TestGTestLogParserTests(unittest.TestCase):265 def testGTestLogParserNoSharing(self):266 # Tests for log parsing without sharding.267 parser = gtest_utils.GTestLogParser()268 for line in TEST_DATA.splitlines():269 parser.ProcessLine(line)270 self.assertEqual(0, len(parser.ParsingErrors()))271 self.assertFalse(parser.RunningTests())272 self.assertEqual(sorted(FAILURES), sorted(parser.FailedTests()))273 self.assertEqual(sorted(FAILURES + FAILS_FAILURES),274 sorted(parser.FailedTests(include_fails=True)))275 self.assertEqual(sorted(FAILURES + FLAKY_FAILURES),276 sorted(parser.FailedTests(include_flaky=True)))277 self.assertEqual(sorted(FAILURES + FAILS_FAILURES + FLAKY_FAILURES),278 sorted(parser.FailedTests(include_fails=True, include_flaky=True)))279 self.assertEqual(10, parser.DisabledTests())280 self.assertEqual(2, parser.FlakyTests())281 test_name = 'NavigationControllerTest.Reload'282 self.assertEqual('\n'.join(['%s: ' % test_name, RELOAD_ERRORS]),283 '\n'.join(parser.FailureDescription(test_name)))284 test_name = 'NavigationControllerTest/SpdyNetworkTransTest.Constructor/0'285 self.assertEqual('\n'.join(['%s: ' % test_name, SPDY_ERRORS]),286 '\n'.join(parser.FailureDescription(test_name)))287 test_name = 'SomeOtherTest.SwitchTypes'288 self.assertEqual('\n'.join(['%s: ' % test_name, SWITCH_ERRORS]),289 '\n'.join(parser.FailureDescription(test_name)))290 test_name = 'BadTest.TimesOut'291 self.assertEqual('\n'.join(['%s: ' % test_name,292 TIMEOUT_ERRORS, TIMEOUT_MESSAGE]),293 '\n'.join(parser.FailureDescription(test_name)))294 test_name = 'MoreBadTest.TimesOutAndFails'295 self.assertEqual('\n'.join(['%s: ' % test_name,296 MOREBAD_ERRORS, TIMEOUT_MESSAGE]),297 '\n'.join(parser.FailureDescription(test_name)))298 parser = gtest_utils.GTestLogParser()299 for line in TEST_DATA_CRASH.splitlines():300 parser.ProcessLine(line)301 self.assertEqual(0, len(parser.ParsingErrors()))302 self.assertTrue(parser.RunningTests())303 self.assertEqual(['HunspellTest.Crashes'], parser.FailedTests())304 self.assertEqual(0, parser.DisabledTests())305 self.assertEqual(0, parser.FlakyTests())306 test_name = 'HunspellTest.Crashes'307 self.assertEqual('\n'.join(['%s: ' % test_name, 'Did not complete.']),308 '\n'.join(parser.FailureDescription(test_name)))309 def testGTestLogParserSharing(self):310 # Same tests for log parsing with sharding_supervisor.311 parser = gtest_utils.GTestLogParser()312 test_data_shard = TEST_DATA_SHARD_0 + TEST_DATA_SHARD_1313 for line in test_data_shard.splitlines():314 parser.ProcessLine(line)315 parser.ProcessLine(TEST_DATA_SHARD_EXIT + '2')316 self.assertEqual(0, len(parser.ParsingErrors()))317 self.assertFalse(parser.RunningTests())318 self.assertEqual(sorted(FAILURES), sorted(parser.FailedTests()))319 self.assertEqual(sorted(FAILURES + FAILS_FAILURES),320 sorted(parser.FailedTests(include_fails=True)))321 self.assertEqual(sorted(FAILURES + FLAKY_FAILURES),322 sorted(parser.FailedTests(include_flaky=True)))323 self.assertEqual(sorted(324 FAILURES + FAILS_FAILURES + FLAKY_FAILURES),325 sorted(parser.FailedTests(include_fails=True, include_flaky=True)))326 self.assertEqual(10, parser.DisabledTests())327 self.assertEqual(2, parser.FlakyTests())328 test_name = 'NavigationControllerTest.Reload'329 self.assertEqual('\n'.join(['%s: ' % test_name, RELOAD_ERRORS]),330 '\n'.join(parser.FailureDescription(test_name)))331 test_name = (332 'NavigationControllerTest/SpdyNetworkTransTest.Constructor/0')333 self.assertEqual('\n'.join(['%s: ' % test_name, SPDY_ERRORS]),334 '\n'.join(parser.FailureDescription(test_name)))335 test_name = 'SomeOtherTest.SwitchTypes'336 self.assertEqual('\n'.join(['%s: ' % test_name, SWITCH_ERRORS]),337 '\n'.join(parser.FailureDescription(test_name)))338 test_name = 'BadTest.TimesOut'339 self.assertEqual(340 '\n'.join(['%s: ' % test_name,341 TIMEOUT_ERRORS, TIMEOUT_MESSAGE]),342 '\n'.join(parser.FailureDescription(test_name)))343 test_name = 'MoreBadTest.TimesOutAndFails'344 self.assertEqual(345 '\n'.join(['%s: ' % test_name,346 MOREBAD_ERRORS, TIMEOUT_MESSAGE]),347 '\n'.join(parser.FailureDescription(test_name)))348 parser = gtest_utils.GTestLogParser()349 for line in TEST_DATA_CRASH.splitlines():350 parser.ProcessLine(line)351 self.assertEqual(0, len(parser.ParsingErrors()))352 self.assertTrue(parser.RunningTests())353 self.assertEqual(['HunspellTest.Crashes'], parser.FailedTests())354 self.assertEqual(0, parser.DisabledTests())355 self.assertEqual(0, parser.FlakyTests())356 test_name = 'HunspellTest.Crashes'357 self.assertEqual('\n'.join(['%s: ' % test_name, 'Did not complete.']),358 '\n'.join(parser.FailureDescription(test_name)))359 def testGTestLogParserValgrind(self):360 parser = gtest_utils.GTestLogParser()361 for line in TEST_DATA_VALGRIND.splitlines():362 parser.ProcessLine(line)363 self.assertEqual(0, len(parser.ParsingErrors()))364 self.assertFalse(parser.RunningTests())365 self.assertFalse(parser.FailedTests())366 self.assertEqual([VALGRIND_HASH], parser.SuppressionHashes())367 self.assertEqual(VALGRIND_SUPPRESSION,368 '\n'.join(parser.Suppression(VALGRIND_HASH)))369 parser = gtest_utils.GTestLogParser()370 for line in FAILING_TESTS_OUTPUT.splitlines():371 parser.ProcessLine(line)372 self.assertEqual(FAILING_TESTS_EXPECTED,373 parser.FailedTests(True, True))374if __name__ == '__main__':...
Automation Testing Tutorials
Learn to execute automation testing from scratch with LambdaTest Learning Hub. Right from setting up the prerequisites to run your first automation test, to following best practices and diving deeper into advanced test scenarios. LambdaTest Learning Hubs compile a list of step-by-step guides to help you be proficient with different test automation frameworks i.e. Selenium, Cypress, TestNG etc.
LambdaTest Learning Hubs:
- JUnit Tutorial
- TestNG Tutorial
- Webdriver Tutorial
- WebDriverIO Tutorial
- Protractor Tutorial
- Selenium 4 Tutorial
- Jenkins Tutorial
- NUnit Tutorial
- Jest Tutorial
- Playwright Tutorial
- Cypress Tutorial
- PyTest Tutorial
YouTube
You could also refer to video tutorials over LambdaTest YouTube channel to get step by step demonstration from industry experts.
Run Lemoncheesecake automation tests on LambdaTest cloud grid
Perform automation testing on 3000+ real desktop and mobile devices online.
Try LambdaTest Now !!
Get 100 minutes of automation test minutes FREE!!
Was this article helpful?
