Test execution basics: passed, failed, crashed

|

In Junit based testing framework there are few possible test outcomes: passed, failed and crashed. In this short note we will dive into that mechanisms.

Test will pass, if no exception is thrown. Empty test should always pass:

class TestFlowShowcase : StringSpec({
    "it should pass"{
        // nothing here
    }
})

Or test with assertion that passes:

class TestFlowShowcase : StringSpec({
    "1 == 1"{
        (1 == 1) shouldBe true
    }
})

Test will fail, if AssertionException (or descendant) would be thrown:

class TestFlowShowcase : StringSpec({
   "it should fail"{
        throw AssertionError("fail")
    }
})

Test will crash, if other exception would be thrown:

class TestFlowShowcase : StringSpec({
    "it should crash"{
        throw Exception("it crashed")
    }
})

How to treat different test outcomes?

PassedNo error was thrown
FailedAssertionError was thrown in test
CrashedOther exception was thrown

“Green” test means only that no error was thrown during test execution – it may also mean that no assertion was performed.

Usually test failed means that something is wrong with system under test – it doesn’t meet expectations defined in assertions.

Test crash indicate that something is wrong with test itself – it does not handle exception thrown in system under test, or its mocks are misconfigured.


Join AndroidPro newsletter & get roadmap for Android devs

👉 Architecture
👉 CI/CD
👉 Testing
👉 Compose
👉 projects to learn from
… in a single document


Read more:

  • #kotlinDevChallenge – 10

    #kotlinDevChallenge – 10

    Implement a function that sums the first 5 even numbers from each of three given integer Flows and returns their total sum. Properly handle…

  • #kotlinDevChallenge – 9

    #kotlinDevChallenge – 9

    Write a function that takes a list of lists of integers and returns a single list containing all unique elements. Ensure that the resulting…

  • #kotlinDevChallenge – 8

    #kotlinDevChallenge – 8

    data class A is given. We create an instance of this class and then call the .copy() function. What will be printed out? Mark…