Unit Tests
Goals
- Understand unit tests.
- Create a unit test using JUnit.
- Integrate JUnit tests into the Maven life cycle.
Concepts
- builder design pattern
- continuous integration (CI)
- edge case
- fluent interface
- Hamcrest
- happy path
- identity
- integration test
- JUnit
- matcher
- method chaining
- scope
- test-driven development (TDD)
- test harness
- unit
- unit test
Library
java.lang.Math.abs(int)
java.lang.StringBuilder
org.hamcrest.MatcherAssert
org.hamcrest.MatcherAssert.assertThat(T actual, Matcher<? super T> matcher)
org.hamcrest.Matchers
org.hamcrest.Matchers.closeTo(double operand, double error)
org.hamcrest.Matchers.equalTo(T operand)
org.hamcrest.Matchers.greaterThan(T value)
org.hamcrest.Matchers.instanceOf(java.lang.Class<?> type)
org.hamcrest.Matchers.is(Matcher<T> matcher)
org.hamcrest.Matchers.is(T value)
org.hamcrest.Matchers.lessThan(T value)
org.hamcrest.Matchers.not(Matcher<T> matcher)
org.hamcrest.Matchers.not(T value)
org.hamcrest.Matchers.nullValue()
org.hamcrest.Matchers.sameInstance(T target)
org.junit.jupiter.api
org.junit.jupiter.api.Assertions
org.junit.jupiter.api.Assertions.assertNotNull(Object actual)
org.junit.jupiter.api.Assertions.assertEquals(double expected, double actual, double delta)
org.junit.jupiter.api.Assertions.assertEquals(long expected, long actual)
org.junit.jupiter.api.Assertions.assertEquals(Object expected, Object actual)
org.junit.jupiter.api.Assertions.assertFalse(boolean condition)
org.junit.jupiter.api.Assertions.assertThrows(Class<T> expectedType, Executable executable)
org.junit.jupiter.api.Assertions.assertTrue(boolean condition)
org.junit.jupiter.api.Assertions.fail()
org.junit.jupiter.api.Disabled
org.junit.jupiter.api.Test
Dependencies
org.junit.jupiter:junit-jupiter-engine:5.3.2
(scope:test
)org.hamcrest:hamcrest:2.1
(scope:test
)
Build Plugins
Lesson
“Contract programming” is the idea that the its declared constants and methods of a class, along with their documentation, can together be considered an agreement between a class and those who use it regarding what data the class supports and how it will behave. In our first look at contract programming you saw how individual methods can use tools such a preconditions to ensure that the caller is using a method correctly. Now you will look at the other side of the coin: how we can test to ensure that a class and its methods will behave correctly when presented with valid (and invalid) data.
Unit Tests
The fundamental unit of design in Java is the class, and in object-oriented design classes make up units of functionality. Contract programming extends beyond individual methods, because these methods many times have to work together with the larger class unit, which may have different states at different times, causing the methods to behave differently as well. The class documentation therefore creates a “contract” about the use of the class as a unit. To ensure that the class adheres to its contract, we can create unit tests.
Just as preconditions are a way to test that information going into a method are valid as required by the method contract, unit tests are a way to ensure that each method of a class returns the correct information its contract promises. A unit test will normally be made up of individual tests, each of them providing input to a method and ensuring that the correct values are returned. It is typical to test the same method several times with different input data.
Here is how we might test the java.lang.Math.abs(int)
utility method in Java, which returns the absolute value of any given integer.
Things get more interesting if we test how a unit behaves over time based upon given inputs. The java.lang.StringBuilder
class provides an efficient way to construct a string from smaller strings. Rather than using the +
operator to concatenate strings, the StringBuilder
class allows the string to be “built” from various other strings and characters, using the same “builder” instance. Only at the end of the process will you call the StringBuilder.toString()
method to produce the string you have been “building”. We can create a test to ensure that a StringBuilder
will reliably create a string after we feed it the various components.
JUnit
At this point you may be wondering just where we should put all these tests, and if assert
is really the best way to test each condition. (Remember additionally that assert
only works if you turn on assertions when you invoke the JVM.) Nowadays there exist frameworks to help with creating unit tests. One of the first popular frameworks, which we will use here, is JUnit and it integrates directly into the Maven life cycle.
JUnit introduces several annotations, most importantly the annotation org.junit.jupiter.api.Test
. It also provides through its org.junit.jupiter.api.Assertions
class many static assertXXX(…)
methods. These methods are “check methods” as you've seen before, which test some expression and throw an exception if the expression evaluates to false
. Here are some of the most common JUnit assertion methods you'll use. You'll see them in use later on in this lesson.
Assertions.assertNotNull(Object actual)
- Asserts that the given argument is not
null
. Assertions.assertEquals(long expected, long actual)
- Asserts that the two values are equal.
Assertions.assertEquals(double expected, double actual, double delta)
- Asserts that the two floating point values are approximately equal within some range.
Assertions.assertEquals(Object expected, Object actual)
- Asserts that the two given objects are equal using
Object.equals(Object)
. Assertions.assertFalse(boolean condition)
- Asserts that the given condition is
false
. Assertions.assertTrue(boolean condition)
- Asserts that the given condition is
true
.
To use JUnit, you will need to include it in your Maven dependencies.
You do not run JUnit tests as part of your normal program. Instead, your tests will be invoked by a test harness, a toolkit of classes that work together to run your tests in an automated fashion. JUnit provides an “engine” that functions as a test harness. The Maven Surefire Plugin is responsible for locating your tests and invoking them with the JUnit test harness.
Although Maven already includes Surefire as part of the build, the version Maven currently by default does not fully support JUnit 5. Therefore you must indicate a recent version of the Maven Surefire Plugin in your POM.
Maven as you remember values “convention over configuration”. In order for the Maven Surefire Plugin to find your tests, your test class should in most cases end with …Test
, reflecting the name of the class you are testing. If you want to test the FooBar
class, you should create a class named FooBarTest
, and it should be in the same package as FooBar
. But it should not be in the same directory! Even though the package directory sequence will be the same, the Maven Surefire Plugin expects to find test files in a separate directory tree, under src/test/java/
. Let's revisit the tree structure that Maven expects, adding in the test hierarchy:
pom.xml | Maven POM |
---|---|
src/main/java/ |
Main source code root directory. e.g. |
src/main/resources/ | Resources used by the main source code. |
src/test/java/ |
Test source code root directory. e.g. |
src/test/resources/ | Resources used by the test code. |
For each test class it finds, the Maven Surefire Plugin will start the JUnit engine, which will in turn find all methods that are marked as with the @Test
annotation. For each of these test methods, JUnit will create an instance of your test class and invoke that method. If any of the assertions fail, JUnit will throw an exception, which will be caught by the test harness so that it can report the error back to you. Then JUnit will run the other tests methods in turn.
Let's take another look at the class representing a geometrical point. We want to test that this unit functions is working correctly.
Now let's create a simple test of creating a Point
instance and making sure it has the correct values. A single test class may contain various tests, but this first test class will contain only one test in a method named testConstructor()
.
Asserting Failure
The test above shows how to test class functions correctly and that its operations do not fail. But what if you want to test that an operation will fail? For instance if you pass an invalid value to a method, you expect the method to fail fast and throw an error, such as an IllegalArgumentException
. In this case you can turn the normal programming logic upside-down by catching the exception and considering the result a success, and failing the test if the operation succeeds (that is, if the exception is not thrown). From past lessons you know that you can fail an assertion by using assert false or simply throwing an AssertionError
, but JUnit provides a method especially for this purpose: Assertions.fail()
and its variations.
Disabling Tests
If for some reason you want to temporarily disable a test, you can use the org.junit.jupiter.api.Disabled
annotation. Simply add @Disabled
to a test method, and JUnit will skip that test.
Hamcrest
JUnit by itself provides an API and a testing engine, but third party libraries can integrate with JUnit to allow more expressive assertions. One of the most popular libraries named Hamcrest provides an alternate interface to JUnit's Assertions
class with its own org.hamcrest.MatcherAssert
utility class. Static methods such as MatcherAssert.assertThat(T actual, Matcher<? super T> matcher)
accepts the value being tested, followed by one or more nested matcher instances that explain what sort of tests to perform on the value. These utility methods and matchers make up a fluent interface; that is, they can be strung together to make a readable assertion statement that reads almost like English.
Most matchers are available via static methods in the org.hamcrest.Matchers
class. Because of its fluent interface, once you see the name of the matcher you'll often know immediately what sort of test it performs. Here are some of the most common matchers you will use day-to-day:
Matchers.closeTo(double operand, double error)
- Tests whether the actual value is more or less equal to some operand, within some error range.
Matchers.equalTo(T operand)
- Tests whether the actual value is equal to the given operand using
Object.equals(Object)
. Matchers.greaterThan(T value)
- Tests whether the actual value is greater than the given value.
Matchers.instanceOf(java.lang.Class<?> type)
- Tests whether the actual value an instance of the given class.
Matchers.is(Matcher<T> matcher)
- Wraps another matcher to make the expression read more fluently, without changing the result of the other matcher.
Matchers.is(T value)
- Tests whether the actual value is equal to the given operand. Equivalent to
is(equalTo(value))
. Matchers.lessThan(T value)
- Tests whether the actual value is less than the given value.
Matchers.not(Matcher<T> matcher)
- Wraps another matcher to make the expression read more fluently, and also inverting the logic of the other matcher's result.
Matchers.not(T value)
- Tests whether the actual value is not equal to the given operand. Equivalent to
not(equalTo(value))
. Matchers.nullValue()
- Tests whether the actual value is
null
. Equivalent tois(equalTo(null))
or simplyis(null)
. Matchers.sameInstance(T target)
- Tests whether the actual value is the same instance of the given target. Equivalent to testing
value == target
for the actual value.
You must include the Hamcrest library as a dependency alongside JUnit.
We'll revamp the PointTest
class so that it uses Hamcrest assertions alongside the traditional JUnit assertions. You should not include both in your code; the equivalent JUnit and Hamcrest versions are provided for comparison purposes.
You can see how the Hamcrest methods are more expressive, not to mention readable. For example, testing that “point.getY() > point.getX()
” simply yields true
or false
; if that test fails, the test harness has no way to know what part of the expression didn't match. But testing that “point.getY(), greaterThan(point.getX())
” provides more information to the test harness about the actual comparison being involved, meaning that more information can be reported to the developer if the test fails. For these reasons Hamcrest assertions are preferred over the traditional simpler JUnit assertions.
Maven
With Maven you don't have to worry about how to point JUnit to your tests classes and invoke the JUnit test harness. As long as your tests adhere to the convention outlined above, and that you have included a recent version of the Maven Surefire Plugin, Maven will automatically run your tests as part of the build cycle using the Surefire. Recall the major phases of the Maven life cycle:
validate
initialize
compile
test
package
- …
By default all JUnit tests are executed in the test
phase. Testing occurs after the compile
phase, as the tests themselves are classes that need compiled. Similarly testing occurs before the package
phase, as a project should not be bundled for distribution if it is failing its tests; if one or more tests fail, Maven aborts the build.
You should now already have an idea of how to invoke tests from Maven. As with all life cycle phases, indicate to Maven the test
phase or any phase after the test
phase, as Maven first performs all phases that come before an indicated phase. The simplest approach is to tell Maven to perform the test
phase:
Review
Summary
- You must place tests in the same package as the class being tested, but in the
src/test/java/
directory tree rather than thesrc/main/java/
directory tree. - You should give tests the same name as the class being tested, with a
...Test
suffix. - Add a
@Test
annotation to test methods.
Gotchas
- Remember to declare your JUnit dependency as having
<scope>test</scope>
. There is no reason to ship JUnit out with your application in production; it is only for internally testing your program. - Don't forget to add the
@Test
annotation to your test methods, or JUnit will not run them. - Floating-point testing is tricky! Use the assertion methods made specially for floating point numbers such as
double
; do not simply assert their equality. Assertions.assertEquals(long expected, long actual)
andMatchersAssert.assertThat(T actual, org.hamcrest.Matcher<T> matcher)
use opposite orders of expected and actual values; don't get confused and reverse the order of the arguments you pass.- Continually concatenating string parts into the same variable is very inefficient. If you need to “build” a string, use a
StringBuilder
instead.
In the Real World
- Don't litter all your classes with
main(String[])
methods withSystem.out.println(…)
statements as a way to do quick-and-dirty tests. These methods will become out-of-date because there is no framework to run them on a regular basis and make sure they continue to function—or even to test their results. Take the time to create a proper unit test; don't irritate your teammates with clutter. - The old JUnit
assertEquals(...)
and related assertions have fallen out of favor because they are less readable and provide less specific feedback on error conditions. - Prefer the new Hamcrest
assertThat(..., is(...))
methods over the old JUnit assertions.
Mixing JUnit 4 and JUnit 5
In the real world you may encounter a project that has a significant number of JUnit 4 tests in place. Ideally you would convert the tests to JUnit 5, which is not too difficult: for the most part it requires adding dependencies, changing JUnit imports, and modifying a few JUnit annotation. Nevertheless JUnit 5 makes it easy to run older versions of JUnit such as Junit 3 and Junit 4 in the same project as newer JUnit 5 tests. To enable this capability you will need to include the latest org.junit.vintage:junit-vintage-engine
dependency, which should be given a scope of test
the same as the JUnit engine itself. Note that this dependency brings in junit:junit:4.12
automatically.
For further information see Maven Surefire Plugin: Using JUnit 5 Platform.
Using Hamcrest with JUnit 4
As mentioned earlier in this lesson, JUnit 4 included a transitive dependency to a subset of Hamcrest. The Hamcrest library at one time comprised several modules, and the last released JUnit 4 dependency junit:junit:4.12
included the org.hamcrest:hamcrest-core:jar:1.3
module. Thus if the core Hamcrest module provided sufficient capabilities for a project, no further dependencies were need. However to include the full Hamcrest 1.3 library, containing the full set of matchers, one would have to include the additional org.hamcrest:hamcrest-library
module separately.
With the release of Hamcrest 2.1, the various modules have been combined into the single org.hamcrest:hamcrest:2.1
dependency used in this lesson. However because junit:junit:4.12
will continue to transitively include an outdated org.hamcrest:hamcrest-core:jar:1.3
module, Hamcrest ships a separate org.hamcrest:hamcrest-core:jar:2.1
dependency which does nothing more than transitively include org.hamcrest:hamcrest:jar:2.1
. Thus to include Hamcrest 2.1 with JUnit 4 with no outdated dependencies, one needs to explicitly specify the org.hamcrest:hamcrest-core:jar:2.1
dependency. This will override the outdated version of org.hamcrest:hamcrest-core
while also bringing in org.hamcrest:hamcrest
.
For more insight beind the reasoning behind the Hamcrest bundling decisions, read the discussion at Hamcrest Issue #224 on GitHub.
Think About It
- Does my unit test check a single behavior, or is it checking several behaviors? Ideally each test method will test a single behavior, using the fewest number of assertions possible.
Self Evaluation
- What is the fundamental unit of work in Java?
- How do you use method chaining?
- What package should test classes be in? In what directory tree should they be stored?
- Ideally how many assertions should appear in each test?
Task
Add all appropriate unit tests for the Book
class.
- Ensure that a properly constructed
Book
will return the correct values via its “getter” methods. - Ensure that the
Book
constructors properly fail fast and throw appropriate exceptions if they are attempted to be invoked with invalid values.
See Also
- Unit Testing with JUnit (vogella)
- A Guide to JUnit 5 (Baeldung)
- Using JUnit 5 Platform (Maven Surefire Plugin)
- Migrating from JUnit 4 to JUnit 5 (Baeldung)
- Using Hamcrest for testing (vogella)
- The Hamcrest Tutorial (Google Code Archive)