Monday, April 27, 2015

What should we Unit Test

Unit tests should be done wisely

It is easy to fall into the unit test biggest pitfall - dumb unit tests.

What is the recipe for a popular unit-test pitfall ?
  • Assume unit tests are very important
  • Test everything
  • verify every method call
  • Get 100% coverage
Yes, the above is bad.

The most obvious problem you will find yourself struggling with will be failing tests regularily, and changing the tests to pass those failure points.

That is the direct opposite target we want to achieve from unit tests.
Upgrading of a unit test happens, but it shouldn't happen too much unless you specifically changed code in order to change the business logic.

When a test fails, it should point to a problem with the code not with the test.

Why do we fall to this pitfall ?

Because we:
  • Create unfocused unit tests 
  • We test all the easy things instead of testing the important things
  • We test implementation instead of testing the business logic
Let me elaborate a little on the third point, when we test a method we should test the expected response and not the specific implementation, thus when the implementation changes the unit test will still pass.
Example: If in a method we call list.clear() three times, we shouldn't test (using a mock spy initiating the verify method) that the list.clear() was called three times because that is an implementation detail, and maybe in the future I will optimize that method and will use list.clear() only twice !? - the test shouldn't fail as it is running the exact same business logic...

Unit Test Guidelines (JUnit)

  • Every unit test file should test a single source file
  • Unit test file should be in the same package as the src file, but it's root shuold be "test" instead of "src"
  • The class under test shuold be declared at the head of the unit test
  • It should be constructed in the "@Before setUp()" method
  • Every test method should test a single target only (usually in a single method)
  • Use mocks as much as needed as dummies or stubs, don't use too many spies - those generally test implementation thus should be avoided (not always)
  • Test target should be a single assertion
  • Test targets should cover (some of) the following: 
    • Happy Path (the business logic of this method)
    • Failure cases (Specific cases which failed so we want to create a unit test for this scenario)
    • Edge cases (What happens if we send null or 0 etc)
  • Tests shouldn't be dependant on the conclusions of other test - they should be able to run individually (The only common method should be the setUp() & tearDown() )
  • As a thumbrule: Methods returning a response (other than void) should be asserted, methods returning nothing (void) might need to be verified - but focus your target in mind and don't verify implementation.


  • Methods which return a repsponse should be tested using a single assrtion
  • Methods returning void might need to verify a spy
  • Each test should focus on a single goal only (and mention it in the test name)
  • Test goals can be one of the following: Happy path, Failure case, Edge case.
  • Beware of testing implementation code, beware of using too many spies.

No comments: