Unit Tests

Goals

Concepts

Library

Dependencies

Build Plugins

Lesson

“Contract programming” is the idea that the its declared constants and methods of a class, along with their documentation, can together be considered an agreement between a class and those who use it regarding what data the class supports and how it will behave. In our first look at contract programming you saw how individual methods can use tools such a preconditions to ensure that the caller is using a method correctly. Now you will look at the other side of the coin: how we can test to ensure that a class and its methods will behave correctly when presented with valid (and invalid) data.

Unit Tests

The fundamental unit of design in Java is the class, and in object-oriented design classes make up units of functionality. Contract programming extends beyond individual methods, because these methods many times have to work together with the larger class unit, which may have different states at different times, causing the methods to behave differently as well. The class documentation therefore creates a “contract” about the use of the class as a unit. To ensure that the class adheres to its contract, we can create unit tests.

Just as preconditions are a way to test that information going into a method are valid as required by the method contract, unit tests are a way to ensure that each method of a class returns the correct information its contract promises. A unit test will normally be made up of individual tests, each of them providing input to a method and ensuring that the correct values are returned. It is typical to test the same method several times with different input data.

Here is how we might test the java.lang.Math.abs(int) utility method in Java, which returns the absolute value of any given integer.

Testing Math.abs(int).
//test with 5
assert Math.abs(5) == 5 : "Test failed for 5.";
//test with -5
assert Math.abs(-5) == 5 : "Test failed for -5.";
//test with 0
assert Math.abs(0) == 0 : "Test failed for 0.";
//test with Integer.MIN_VALUE
assert Math.abs(Integer.MIN_VALUE) == Integer.MIN_VALUE : "Test failed for most negative int value.";

Things get more interesting if we test how a unit behaves over time based upon given inputs. The java.lang.StringBuilder class provides an efficient way to construct a string from smaller strings. Rather than using the + operator to concatenate strings, the StringBuilder class allows the string to be “built” from various other strings and characters, using the same “builder” instance. Only at the end of the process will you call the StringBuilder.toString() method to produce the string you have been “building”. We can create a test to ensure that a StringBuilder will reliably create a string after we feed it the various components.

Testing StringBuilder.append(…).
final StringBuilder stringBuilder = new StringBuilder();
stringBuilder.append("Hello");
stringBuilder.append(',').append(' ');
stringBuilder.append("world").append('!');
assert stringBuilder.toString().equals("Hello, world!") : "The append() method isn't working.";

JUnit

At this point you may be wondering just where we should put all these tests, and if assert is really the best way to test each condition. (Remember additionally that assert only works if you turn on assertions when you invoke the JVM.) Nowadays there exist frameworks to help with creating unit tests. One of the first popular frameworks, which we will use here, is JUnit and it integrates directly into the Maven life cycle.

JUnit introduces several annotations, most importantly the annotation org.junit.jupiter.api.Test. It also provides through its org.junit.jupiter.api.Assertions class many static assertXXX(…) methods. These methods are “check methods” as you've seen before, which test some expression and throw an exception if the expression evaluates to false. Here are some of the most common JUnit assertion methods you'll use. You'll see them in use later on in this lesson.

Assertions.assertNotNull​(Object actual)
Asserts that the given argument is not null.
Assertions.assertEquals(long expected, long actual)
Asserts that the two values are equal.
Assertions.assertEquals(double expected, double actual, double delta)
Asserts that the two floating point values are approximately equal within some range.
Assertions.assertEquals​(Object expected, Object actual)
Asserts that the two given objects are equal using Object.equals(Object).
Assertions.assertFalse(boolean condition)
Asserts that the given condition is false.
Assertions.assertTrue(boolean condition)
Asserts that the given condition is true.

To use JUnit, you will need to include it in your Maven dependencies.

Declaring JUnit as a dependency in the Maven POM.
…
  <dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.3.2</version>
    <scope>test</scope>
  </dependency>
…

You do not run JUnit tests as part of your normal program. Instead, your tests will be invoked by a test harness, a toolkit of classes that work together to run your tests in an automated fashion. JUnit provides an “engine” that functions as a test harness. The Maven Surefire Plugin is responsible for locating your tests and invoking them with the JUnit test harness.

Although Maven already includes Surefire as part of the build, the version Maven currently by default does not fully support JUnit 5. Therefore you must indicate a recent version of the Maven Surefire Plugin in your POM.

Specifying a newer version of Maven Surefire Plugin.
 …
  <build>
    <plugins>
      …
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.22.1</version>
      </plugin>
    </plugins>
  </build>
…

Maven as you remember values “convention over configuration”. In order for the Maven Surefire Plugin to find your tests, your test class should in most cases end with …Test, reflecting the name of the class you are testing. If you want to test the FooBar class, you should create a class named FooBarTest, and it should be in the same package as FooBar. But it should not be in the same directory! Even though the package directory sequence will be the same, the Maven Surefire Plugin expects to find test files in a separate directory tree, under src/test/java/. Let's revisit the tree structure that Maven expects, adding in the test hierarchy:

pom.xml Maven POM
src/main/java/

Main source code root directory.

e.g. src/main/java/com/example/Point.java

src/main/resources/ Resources used by the main source code.
src/test/java/

Test source code root directory.

e.g. src/test/java/com/example/PointTest.java

src/test/resources/ Resources used by the test code.

For each test class it finds, the Maven Surefire Plugin will start the JUnit engine, which will in turn find all methods that are marked as with the @Test annotation. For each of these test methods, JUnit will create an instance of your test class and invoke that method. If any of the assertions fail, JUnit will throw an exception, which will be caught by the test harness so that it can report the error back to you. Then JUnit will run the other tests methods in turn.

Let's take another look at the class representing a geometrical point. We want to test that this unit functions is working correctly.

src/main/java/com/example/Point.java
package com.example;

public class Point
{

  private final int x;
  private final int y;

  public Point(final int x, final int y)
  {
    this.x = x;
    this.y = y;
  }

  public int getX()
  {
    return x;
  }

  public int getY()
  {
    return y;
  }

}

Now let's create a simple test of creating a Point instance and making sure it has the correct values. A single test class may contain various tests, but this first test class will contain only one test in a method named testConstructor().

src/test/java/com/example/PointTest.java
package com.example;

import static org.junit.jupiter.api.Assertions.*;

import org.junit.jupiter.api.Test;

public class PointTest
{

  /** Test the {@link Point} constructor. */
  @Test
  public void testConstructor()
  {
    final Point point = new Point(3, 4);
    assertNotNull(point); //null test
    assertEquals(3, point.getX()); //equality test
    assertTrue(point.getY() > point.getX()); //expression test

    final Point point2 = new Point(3, 4);
    assertFalse(point == point2);   //check identity
  }
}

Asserting Failure

The test above shows how to test class functions correctly and that its operations do not fail. But what if you want to test that an operation will fail? For instance if you pass an invalid value to a method, you expect the method to fail fast and throw an error, such as an IllegalArgumentException. In this case you can turn the normal programming logic upside-down by catching the exception and considering the result a success, and failing the test if the operation succeeds (that is, if the exception is not thrown). From past lessons you know that you can fail an assertion by using assert false or simply throwing an AssertionError, but JUnit provides a method especially for this purpose: Assertions.fail() and its variations.

Testing that an operation will fail.
package com.example;

import static org.junit.jupiter.api.Assertions.*;
import org.junit.junit.api.Test;

public class FooBarTest
{

  /** Test that the {@link FooBar} constructor fails fast. */
  @Test
  public void testConstructorFailFast()
  {
    try
    {
      new FooBar(null);  //no need to store the reference; we're just testing the constructor
      fail("FooBar does not accept null in its constructor");
    }
    catch(final NullPointerException nullPointerException)
    {
      //We consider the exception a test success;
      //it is what we except to happen, so there is nothing to do.
    }
  }

}

Disabling Tests

If for some reason you want to temporarily disable a test, you can use the org.junit.jupiter.api.Disabled annotation. Simply add @Disabled to a test method, and JUnit will skip that test.

Temporarily disabling tests using @Disabled.
package com.example;

import org.junit.jupiter.api.Ignore;
import org.junit.jupiter.api.Test;

public class FooBarTest
{

  @Disabled
  @Test  //test will not be performed because of @Disabled
  public void testFoo()
  {
    …
  }
}

Hamcrest

JUnit by itself provides an API and a testing engine, but third party libraries can integrate with JUnit to allow more expressive assertions. One of the most popular libraries named Hamcrest provides an alternate interface to JUnit's Assertions class with its own org.hamcrest.MatcherAssert utility class. Static methods such as MatcherAssert.assertThat(T actual, Matcher<? super T> matcher) accepts the value being tested, followed by one or more nested matcher instances that explain what sort of tests to perform on the value. These utility methods and matchers make up a fluent interface; that is, they can be strung together to make a readable assertion statement that reads almost like English.

Most matchers are available via static methods in the org.hamcrest.Matchers class. Because of its fluent interface, once you see the name of the matcher you'll often know immediately what sort of test it performs. Here are some of the most common matchers you will use day-to-day:

Matchers.closeTo(double operand, double error)
Tests whether the actual value is more or less equal to some operand, within some error range.
Matchers.equalTo(T operand)
Tests whether the actual value is equal to the given operand using Object.equals(Object).
Matchers.greaterThan(T value)
Tests whether the actual value is greater than the given value.
Matchers.instanceOf(java.lang.Class<?> type)
Tests whether the actual value an instance of the given class.
Matchers.is(Matcher<T> matcher)
Wraps another matcher to make the expression read more fluently, without changing the result of the other matcher.
Matchers.is(T value)
Tests whether the actual value is equal to the given operand. Equivalent to is(equalTo(value)).
Matchers.lessThan(T value)
Tests whether the actual value is less than the given value.
Matchers.not(Matcher<T> matcher)
Wraps another matcher to make the expression read more fluently, and also inverting the logic of the other matcher's result.
Matchers.not(T value)
Tests whether the actual value is not equal to the given operand. Equivalent to not(equalTo(value)).
Matchers.nullValue()
Tests whether the actual value is null. Equivalent to is(equalTo(null)) or simply is(null).
Matchers.sameInstance(T target)
Tests whether the actual value is the same instance of the given target. Equivalent to testing value == target for the actual value.

You must include the Hamcrest library as a dependency alongside JUnit.

Declaring Hamcrest as a dependency in the Maven POM.
…
  <dependency>
    <groupId>org.hamcrest</groupId>
    <artifactId>hamcrest</artifactId>
    <version>2.1</version>
    <scope>test</scope>
  </dependency>
…

We'll revamp the PointTest class so that it uses Hamcrest assertions alongside the traditional JUnit assertions. You should not include both in your code; the equivalent JUnit and Hamcrest versions are provided for comparison purposes.

src/test/java/com/example/PointTest.java using Hamcrest.
package com.example;

import static org.junit.jupiter.api.Assertions.*;  //JUnit
import static org.hamcrest.MatcherAssert.*;  //Hamcrest
import static org.hamcrest.Matchers.*;  //Hamcrest

import org.junit.Test;

public class PointTest
{

  /** Test the {@link Point} constructor. */
  @Test
  public void testConstructor()
  {
    final Point point = new Point(3, 4);

    assertNotNull(point);  //JUnit
    assertThat(point, is(not(equalTo(nullValue()))));  //Hamcrest

    assertEquals(3, point.getX());  //JUnit
    assertThat(point.getX(), is(equalTo(3)));  //Hamcrest

    assertTrue(point.getY() > point.getX());  //JUnit
    assertThat(point.getY(), greaterThan(point.getX()));  //Hamcrest

    final Point point2 = new Point(3, 4);
    assertFalse(point == point2);  //JUnit
    assertThat(point, is(not(sameInstance(point2))));  //Hamcrest
  }
}

You can see how the Hamcrest methods are more expressive, not to mention readable. For example, testing that “point.getY() > point.getX()” simply yields true or false; if that test fails, the test harness has no way to know what part of the expression didn't match. But testing that “point.getY(), greaterThan(point.getX())” provides more information to the test harness about the actual comparison being involved, meaning that more information can be reported to the developer if the test fails. For these reasons Hamcrest assertions are preferred over the traditional simpler JUnit assertions.

Maven

With Maven you don't have to worry about how to point JUnit to your tests classes and invoke the JUnit test harness. As long as your tests adhere to the convention outlined above, and that you have included a recent version of the Maven Surefire Plugin, Maven will automatically run your tests as part of the build cycle using the Surefire. Recall the major phases of the Maven life cycle:

  1. validate
  2. initialize
  3. compile
  4. test
  5. package

By default all JUnit tests are executed in the test phase. Testing occurs after the compile phase, as the tests themselves are classes that need compiled. Similarly testing occurs before the package phase, as a project should not be bundled for distribution if it is failing its tests; if one or more tests fail, Maven aborts the build.

You should now already have an idea of how to invoke tests from Maven. As with all life cycle phases, indicate to Maven the test phase or any phase after the test phase, as Maven first performs all phases that come before an indicated phase. The simplest approach is to tell Maven to perform the test phase:

mvn test

Review

Basic outline of a JUnit/Hamcrest test src/test/java/com/example/FooBarTest.java.
package com.example;

import static org.hamcrest.MatcherAssert.*;
import static org.hamcrest.Matchers.*;
import org.junit.jupiter.api.*;

public class FooBarTest
{

  @Test
  public void testFoo()
  {
    …
  }

}

Summary

Gotchas

In the Real World

Mixing JUnit 4 and JUnit 5

In the real world you may encounter a project that has a significant number of JUnit 4 tests in place. Ideally you would convert the tests to JUnit 5, which is not too difficult: for the most part it requires adding dependencies, changing JUnit imports, and modifying a few JUnit annotation. Nevertheless JUnit 5 makes it easy to run older versions of JUnit such as Junit 3 and Junit 4 in the same project as newer JUnit 5 tests. To enable this capability you will need to include the latest org.junit.vintage:junit-vintage-engine dependency, which should be given a scope of test the same as the JUnit engine itself. Note that this dependency brings in junit:junit:4.12 automatically.

Enabling “vintage” JUnit versions in the Maven POM.
…
  <dependency>
    <groupId>org.junit.vintage</groupId>
    <artifactId>junit-vintage-engine</artifactId>
    <version>5.3.2</version>
    <scope>test</scope>
  </dependency>
…

For further information see Maven Surefire Plugin: Using JUnit 5 Platform.

Using Hamcrest with JUnit 4

As mentioned earlier in this lesson, JUnit 4 included a transitive dependency to a subset of Hamcrest. The Hamcrest library at one time comprised several modules, and the last released JUnit 4 dependency junit:junit:4.12 included the org.hamcrest:hamcrest-core:jar:1.3 module. Thus if the core Hamcrest module provided sufficient capabilities for a project, no further dependencies were need. However to include the full Hamcrest 1.3 library, containing the full set of matchers, one would have to include the additional org.hamcrest:hamcrest-library module separately.

Adding full Hamcrest 1.3 support to JUnit 4.
…
  <dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <scope>test</scope>
  </dependency>

  <dependency>
    <groupId>org.hamcrest</groupId>
    <artifactId>hamcrest-library</artifactId>
    <version>1.3</version>
    <scope>test</scope>
  </dependency>
…

With the release of Hamcrest 2.1, the various modules have been combined into the single org.hamcrest:hamcrest:2.1 dependency used in this lesson. However because junit:junit:4.12 will continue to transitively include an outdated org.hamcrest:hamcrest-core:jar:1.3 module, Hamcrest ships a separate org.hamcrest:hamcrest-core:jar:2.1 dependency which does nothing more than transitively include org.hamcrest:hamcrest:jar:2.1. Thus to include Hamcrest 2.1 with JUnit 4 with no outdated dependencies, one needs to explicitly specify the org.hamcrest:hamcrest-core:jar:2.1 dependency. This will override the outdated version of org.hamcrest:hamcrest-core while also bringing in org.hamcrest:hamcrest.

Adding Hamcrest 2.1 support to JUnit 4.
…
  <dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.12</version>
    <scope>test</scope>
  </dependency>

  <dependency>
    <groupId>org.hamcrest</groupId>
    <artifactId>hamcrest-core</artifactId>
    <version>2.1</version>
    <scope>test</scope>
  </dependency>
…

For more insight beind the reasoning behind the Hamcrest bundling decisions, read the discussion at Hamcrest Issue #224 on GitHub.

Think About It

Self Evaluation

Task

Add all appropriate unit tests for the Book class.

See Also

References

Resources