Optionals

Goals

Recognize the drawbacks of null references.
Learn how to use Optional<T> in APIs to mitigate the risks of null references.

Concepts

null
optional-return idiom

Language

null

Library

Dependencies

org.github.npathai:hamcrest-optional:1.0 (scope: test)

Preview

final Car car = …;
car.getTrunk()
    .flatMap(Trunk::getSpareTire)
    .ifPresent(car::changeFlat);

Lesson

One of the most pesky issues to tackle in Java, as well as in many programming languages, is the the appearance of null values. (You have no doubt already encountered many bugs related to null in your homework during this course.) The first difficulty null exhibits is remembering whether a method's return value needs to be checked for null. The @Nullable annotation from JSR-305 mitigates this problem somewhat, but after recognizing that null may be returned the next difficulty is how to handle null in an elegant manner.

But even after identifying and handling them, null values make your program less understandable and less safe. A null value has no type, so there is sometimes no way to know what an object reference should have been, had it not been null. And in fact null in itself doesn't have clear semantics; does it mean that a value is missing, that it is unavailable, or that some error occurred?

`Optional<T>`

Java now provides a construct that not only makes it clear when a value may not be present, but also provides semantically rich methods for dealing with null—at the same time being especially friendly to functional expressions, and the lambdas and method references used with them. The java.util.Optional<T> class represent a value that is optional—a value object that may be empty and not represent any value.

To see how Optional<T> can help the null problem, let's see a typical case where the null problem might arise. Consider a car that develops a flat tire, and you wish to change the flat using the spare tire in the trunk (also called a boot in the United Kingdom). You may have an API with Car, Trunk, and Tire interfaces.

Two problems immediately present themselves: not all cars have trunks, and not every trunk contains a spare tire! Your API might reflect these possibilities using null:

Modeling unavailable values in an API using null.

public interface Car {

  @Nullable String getRegistrationPlate();

  @Nullable Trunk getTrunk();

  void changeFlat(@Nonnull Tire spare);

  …
}

public interface Trunk {

  @Nullable Tire getSpareTire();

  …
}

final Car car = …;
final String registration = car.getRegistrationPlate();
logVehicleRegistration(registration);  //may throw NullPointerException!
final Trunk trunk = car.getTrunk();
final Tire spare = trunk.getSpareTire();  //may throw NullPointerException!
car.changeFlat(spare);  //may throw NullPointerException!

The problem is that there are two chances to get a null reference: when calling Car.getTrunk() and when calling Trunk.getSpareTire(). You must account for the possibility of null by checking for the presence of a value before using each nullable reference. This is simple to do (if you remember), but soon gets verbose and cumbersome.

Checking for null to verify the presence of a value.

final Car car = …;
final String registration = car.getRegistrationPlate();
logVehicleRegistration(registration != null ? registration : "unregistered");
final Trunk trunk = car.getTrunk();
if(trunk != null) {
  final Tire spare = trunk.getSpareTire();
  if(spare != null) {
    car.changeFlat(spare);
  }
}

Replacing both of these return values with Optional<T> provides even better documentation than @Nullable ever could that these methods might not return a value.

Modeling unavailable values in an API using Optional<T>.

public interface Car {

  Optional<String> getRegistrationPlate();

  Optional<Trunk> getTrunk();

  void changeFlat(@Nonnull Tire spare);

  …
}

public interface Trunk {

  Optional<Tire> getSpareTire();

  …
}

final Car car = …;
final Optional<String> registration = car.getRegistrationPlate();
logVehicleRegistration(registration.get());  //may throw NoSuchElementException!
final Optional<Trunk> trunk = car.getTrunk();
final Optional<Tire> spare = trunk.get().getSpareTire();  //may throw NoSuchElementException!
car.changeFlat(spare.get());  //may throw NoSuchElementException!

You can retrieve the value contained in an Optional<T> by using the Optional.get() method, as shown in the above example. If the Optional<T> is empty (that is, if no value was returned), Optional.get() will throw a java.util.NoSuchElementException. But now we seem to be back where we started. Have we simply traded the possibility of a NullPointerException for a NoSuchElementException?

Keep in mind the value of Optional<T> as documentation, indicating the possibility of a missing value. But documentation is not the only benefit of Optional<T>, as you'll see in the following sections. Indeed it is only the start.

Don't use Optional.get() if you can help it. As you can see here, it brings back some of the dangers of null references. In this lesson you'll learn about more elegant, functional alternatives.

Don't we still have the possibility that the Optional<T> itself could be null? It is true that nothing in Java prevents the returned Optional<T> itself from being null, but that would not be appropriate: the whole point of Optional<T> is to obviate the need for null. You can safely assume that no reference to an Optional<T> is itself null.

Never use a null value for a reference to an Optional<T>; this would defeat the purpose of Optional<T>.

As with the standard functional interfaces, Java provides primitive Optional<T> versions: OptionalInt, OptionalLong, and OptionalDouble. Unfortunately these versions do not have all the capabilities of the normal Optional<T> class. Because only a single value is involved, the performance issues associated with autoboxing should be diminished, and it is recommended that you generally use the object Optional<T> class instead of the primitive versions.

Checking Value Presence

`Optional.isPresent()`

When using Optional<T> you can avoid a NoSuchElementException as you would avoid a NullPointerException: by first checking if a value is present before continuing processing. The Optional.isPresent() method indicates whether the Optional<T> instance contains a value, and can be used wherever you would normally place a null check.

Consider the car's registration plate, which may not be available if the car is new has not yet been sold. Above a null check is performed using the ternary operator: registration != null ? registration : "unregistered". If the registration were returned as an Optional<String>, the check could be done in a similar manner using Optional.isPresent():

Using Optional.isPresent() to verify the presence of a value.

final Car car = …;
final Optional<String> registration = car.getRegistrationPlate();
logVehicleRegistration(registration.isPresent() ? registration.get() : "unregistered");

Being forced to use Optional.isPresent() combined with Optional.get() is perhaps more cumbersome than null checks, but the use of Optional<T> alone does have one benefit: it makes it difficult to forget that the value is optional and that checks are needed. Optional<T> however provides additional methods that parallel many standard use cases for null handling, and these capabilities combine to make the use of Optional<T> more elegant than standard null checks.

`Optional.orElse(T)`

The use of the ternary operator for null checking is a common one. Its goal is to return the value if present, or an alternate or default value if the original value is not present, as you saw with registration.isPresent() ? registration.get() : "unregistered" above. Optional<T> provides a convenient method Optional.orElse(T other) which performs exactly this functionality.

Using Optional.orElse(T) to return a value or an alternate value.

final Car car = …;
final Optional<String> registration = car.getRegistrationPlate();
logVehicleRegistration(registration.orElse("unregistered"));

In fact with the convenience of Optional.orElse(T), there is no longer any need to temporarily store the registration using a separate variable, as the value retrieval logic is now encapsulated in Optional<String>.

final Car car = …;
logVehicleRegistration(car.getRegistrationPlate().orElse("unregistered"));

`Optional.orElseThrow(…)`

Depending on the requirements, the absence of a valid vehicle registration may be considered an error condition. Perhaps in real life the service dispatcher should have checked for the presence of a registration plate before sending a service technician. During the changing of the flat, if the registration number is not present it represents an invalid state, and an InvalidStateException should be thrown.

The Optional.orElseThrow(Supplier<? extends X> exceptionSupplier) allows a custom exception to be thrown if the value is not present. Rather than wastefully creating an exception instance each time, this method allows a java.util.function.Supplier<T> to be provided that can supply a new Throwable when needed. As Supplier<T> is a functional interface, you should recognize that Optional.orElseThrow(…) is a prime candidate for a lambda or a method reference.

Using Optional.orElseThrow(…) to return a value or throw an exception.

final Car car = …;
logVehicleRegistration(car.getRegistrationPlate().orElseThrow(() -> new IllegalStateException("Missing registration."));

If the message used in the IllegalStateException were not important, the a constructor reference would make the program logic even more concise.

final Car car = …;
logVehicleRegistration(car.getRegistrationPlate().orElseThrow(IllegalStateException::new));

`Optional.ifPresent(…)`

One other Optional<T> method for checking for the presence of a value is Optional.ifPresent(Consumer<? super T> consumer). This method takes a java.util.function.Consumer<T> that will be invoked if the value is present; otherwise, no action will be performed.

At first glance it may seem that Optional.ifPresent(…) provides little value over an explicit check using if(optional.isPresent()) { … } following by the action to perform. That Consumer<T> is a functional interface, however, brings some other possibilities. Going back to the original example above, one could provide a method reference to Car.changeFlat(Tire) rather than calling it explicitly.

Using Optional.ifPresent(…) to conditionally process an optional value.

final Car car = …;
final Trunk trunk = …;
trunk.getSpareTire().ifPresent(car::changeFlat);

Processing Values

Mapping

`Optional.map(…)`

Often getting the optional value is not your final goal—it is only a means for retrieving some other value from the first one. Let's say that you (possibly) have a spare tire, and you want to make sure the valve stem is closed. This would involve asking the Tire for its ValveStem as illustrated below. Note that when this API was first created, it was assumed that every tires had a valve stem, and thus Tire.getValveStem() returns a reference to the ValveStem itself, not an Optional<ValveStem>.

Using an optional value to get another value in turn.

public interface Tire {

  @Nullable ValveStem getValveStem();

  …
}

public interface ValveStem {

  void close();

  …
}

final Car car = …;
final Optional<Trunk> trunk = car.getTrunk();
if(trunk.isPresent()) {
  final Optional<Tire> spare = trunk.get().getSpareTire();
  final ValveStem valveStem = spare.get().getValveStem();  //may throw NoSuchElementException!
  valveStem.close();  //may throw NullPointerException!
}

This is getting complicated! Because Trunk.getSpareTire() returns an Optional<Tire>, we would need need to add something like if(spare.isPresent()) before calling Tire.getValveStem(). And because the returned ValveStem may be null, we would additionally need to add if(valveStem != null) before calling valveStem.close().

What we are really wanting to do is to map the the Tire to its ValveStem so that we can close it. You will note that the method Tire.getValveStem() is invoked on a Tire instance and produces a ValveStem instance. If we have an instance of Tire, in other words, we can produce a ValveStem (which may be null) for that Tire—just as we could do with a Map<Tire, ValveStem>, if we had such a map available.

The method Optional.map(Function<? super T, ? extends U> mapper) allows us to map an optional value to another value in just such a way. The output will itself be an Optional<T>, for two reasons. First, if the original value is not present, it cannot be mapped to another value. Second, the mapping itself may produce null; whatever value is produced is therefore wrapped in an Optional<T>. The mapper parameter is a java.util.function.Function<T, R> representing the mapping function between the two values. Here the method used for mapping is Tire.getValveStem(), and you can use a method reference to meet the requirements of Function<Tire, ValveStem>, as shown below.

This mapper function necessary for mapping an Optional<T> is really no different than the keyExtractor function you used in a previous lesson to create a Comparator<T> using Comparator.comparing(Function<? super T, ? extends U> keyExtractor). With Comparator.comparing(…) you used a method reference such as Point::getX to map a Point to its X coordinate. The same principle applies here.

Mapping an optional value to another value using Optional.map(…).

final Car car = …;
final Optional<Trunk> trunk = car.getTrunk();
if(trunk.isPresent()) {
  final Optional<Tire> spare = trunk.get().getSpareTire();
  final Optional<ValveStem> valveStem = spare.map(Tire::getValveStem); //wraps output with Optional<>
  valveStem.ifPresent(ValveStem::close);  //note method reference
}

In Optional<T> mapping operations there is often no longer a need for the intermediate variable, as there are no longer any null checks (or even optional value presence checks) to perform—the Optional<T> class takes care of all the intermediate checks for you.

final Car car = …;
final Optional<Trunk> trunk = car.getTrunk();
car.getTrunk().ifPresent(trunk -> {
  trunk.getSpareTire()
      .map(Tire::getValveStem)
      .ifPresent(ValveStem::close);
});

Do not fail to notice that Optional.map(…) will wrap the output of the mapper function with another Optional<T>.

By recognizing that we were mapping from an optional value to one of its properties; and by leveraging Optional.map(…), Optional.ifPresent(…), and method references; what was originally a precarious set of decisions has now been turned into an a safe, concise set of functions that together read almost like a spoken language sentence.

`Optional.flatMap(…)`

Your first impulse when analyzing the above code may be to replace the if(trunk.isPresent()) test with a similar use of Optional.map(…) from Trunk to Tire using the Trunk.getSpareTire() method as the mapping function.

Incorrectly mapping an optional value to another optional value using Optional.map(…).

final Car car = …;
final Optional<Trunk> trunk = car.getTrunk();
final Optional<Tire> spare = trunk.map(Trunk::getSpareTire); //ERROR; returns Optional<Optional<Tire>>

You might have forgotten that Trunk.getSpareTire() already returns an Optional<Tire>, and because Optional.map(…) wraps the result of the mapper with an Optional<T>, the result is an Optional<Optional<Trunk>>. This is more layers of indirection than we need!

The solution is Optional.flatMap(Function<? super T, Optional<U>> mapper), which functions identically to Optional.map(…) except that Optional.flatMap(…) returns any resulting Optional<T> as-is without wrapping it in an additional Optional<T>.

Correctly mapping an optional value to another optional value using Optional.flatMap(…).

final Car car = …;
final Optional<Trunk> trunk = car.getTrunk();
final Optional<Tire> spare = trunk.flatMap(Trunk::getSpareTire);

A combination of Optional.map(…), Optional.flatMap(…), Optional.ifPresent(…), and method references tidies up the code nicely. Now we can make sure the valve stem is closed on the spare tire without worrying if there is a trunk or whether the spare tire (if any) has a valve stem—all without a single if(…) statement!

final Car car = …;
car.getTrunk()
  .flatMap(Trunk::getSpareTire)
  .map(Tire::getValveStem)
  .ifPresent(ValveStem::close);

And coming full circle to our original example, changing a flat tire (but only if the car has a trunk with a spare tire inside) becomes simply:

final Car car = …;
car.getTrunk()
  .flatMap(Trunk::getSpareTire)
  .ifPresent(car::changeFlat);

Filtering

When studying lambdas we discussed how that you can remove items from a collection by calling Collection.removeIf(Predicate<? super E> filter) and passing a java.util.function.Predicate<T> to test each element individually and determine if it should be removed. In an even earlier lesson you saw that Google's Guava library has filtering utility methods such as Iterators.filter(Iterator<T> unfiltered, Predicate<? super T> predicate), for which the given Guava predicate encapsulates the test for filtering individual elements of the Iterator<T>.

The Optional.filter(Predicate<? super T> predicate) method provides a way to filter an optional value. The using a Predicate<T> parameter indicates the test to apply to the value. If the value passes the test, the returned Optional<T> will likely be the same Optional<T> you started with. If the value does not pass the test (or there is no value to begin with), the returned Optional<T> will be empty.

To understand how Optional.filter(…) works, you can think of Optional<T> as an Iterator<T> that can return at most a single object. In this light, Optional.isPresent() is analogous to Iterator.hasNext(), and Optional.get() is analogous to Iterator.next().

Let's suppose that a service technician is only authorized to change a flat tire on a car with a registration plate that starts with foo. If Optional<T> were not used, the service technician would first have to check the registration plate (which may be null) and then compare its beginning characters with the string "foo".

Manually filtering a nullable value.

final Car car = …;
final String registration = car.getRegistrationPlate();
if(registration == null) {
  throw new IllegalStateException("Missing registration plate.");
}
if(!registration.startsWith("foo")) {
  throw new IllegalStateException("Unsupported registration.");
}
logVehicleRegistration(registration);

Filtering with Optional<T> becomes easier, safer, and more readable:

Filtering an Optional<T> value.

final Car car = …;
logVehicleRegistration(car.getRegistrationPlate()
    .filter(registration -> registration.startsWith("foo"))
    .orElseThrow(IllegalStateException::new));

If the specifications for your application require you throw different IllegalStateExceptions with a different messages to distinguish between error conditions, you would need to do that in separate steps, using Guava's Preconditions.checkState(…) for example:

final Car car = …;
final String registration = car.getRegistrationPlate()
    .orElseThrow(() -> throw new IllegalStateException("Missing registration plate.");
checkState(registration.startsWith("foo"), "Unsupported registration.");
logVehicleRegistration(registration);

Producing an `Optional<T>`

You can wrap any instance of an object with Optional<T> using the static factory method Optional.of(T value), as you should expect. What you might not have expected is that Optional.of(…) will throw a NullPointerException if the given value is null. If you want an instance of Optional<T> that contains no value, you can use Optional.empty().

To see how this works, let's create a method for a Point to indicate in which quadrant it lies, taking into consideration that a Point on one of the axes (that is, with a X and/or Y coordinate of zero) is not considered to be in any quadrant. We'll create an enum to represent the four quadrant values.

Implementing Point.getQuadrant() returning Optional<Quadrant> using Optional.of(…) and Optional.empty().

public enum Quadrant {I, II, III, IV}

public class Point {

…

  public Optional<Quadrant> getQuadrant() {
    if(getX() > 0) {
      if(getY() > 0) {
        return Optional.of(Quadrant.I);  //quadrant 1
      } else if(getY() < 0) {
        return Optional.of(Quadrant.IV);  //quadrant 4
      }
    } else if(getX() < 0) {
      if(getY() > 0) {
        return Optional.of(Quadrant.II);  //quadrant 2
      } else if(getY() < 0) {
        return Optional.of(Quadrant.III);  //quadrant 3
      }
    }
    return Optional.empty();  //no quadrant
  }
}

If you already have a reference may may be null, you can create an Optional<T> instance using Optional.ofNullable(T value). This is especially useful if you are working with a legacy API (such as the Java Collections Framework) that does not support Optional<T>. Let us revisit the VehicleRepository from a previous lesson and have lookup based upon registration number return an Optional<Vehicle> instead of @Nullable Vehicle. A typical implementation might use a Map<String, Vehicle> to associate vehicles with registration numbers, which are unique identifiers. As Map.get() returns a nullable reference, it is a simple step to return an Optional<Vehicle> by using Optional.ofNullable(…).

Converting a nullable vehicle reference to Optional<Vehicle> using Optional.ofNullable(…).

public class FakeVehicleRepository implements VehicleRepository {
…

  /** The map of vehicles, mapped to their registration numbers. */
  private final Map<String, Vehicle> registeredVehicles;

  …

  @Override
  public Optional<Vehicle> findVehicleByRegistrationNumber(@Nonnull final String registrationNumber)
      throws IOException;
    checkNotNull(registrationNumber); //preconditions are still a good idea
    return Optional.ofNullable(registeredVehicles.get(registrationNumber));
  }
}

Drawbacks

The Optional<T> class brings a lot of benefits, but it isn't a perfect tool, as some of the articles in the See Also section demonstrate. Sometimes it makes code safer and more readable. Sometimes it makes simple logic more complicated. And Optional<T> still is no better than null for indicating why a value is missing.

The main purpose of Optional<T> was to make it easier for an API to indicate that a return value is optional. Some go so far as to advocate that Optional<T> should never be used as input parameters of methods or in constructor parameters. In your own code try to relegate the use of Optional<T> as an optional-return idiom as intended unless the benefit of using it for another purpose is significant.

Try not to store references to Optional<T> in your business objects. As you will learn in lessons ahead, Optional<T> was not designed to work with the default Java approach for storing and retrieving objects from a data store.

Methods that return Optional<T> present an extra layer of difficulty in unit tests. Rather than simply comparing a return value, you must first ensure that the value is not empty, and then extract the actual returned value to make the comparison. The Hamcrest Optional project provides matchers that assist with matching Optional<T> results, including isEmpty() and hasValue(Object). You can even mix and match them with existing matchers, such as collection matchers or hasValue(instanceOf(Foo.class)). There exists a feature request to add matchers for Optional<T> to the core Hamcrest library at some point; see Add matchers for JDK 8 Optionals #82.

Review

Gotchas

The output of Optional.map(…) is an Optional<T>, so if the mapping function also produces an Optional<T>, this would produce an Optional<Optional<T>>. You would need to use Optional.flatMap(…) instead in this case.
Optional.of(…) will throw a NullPointerException if the given value is null. If you don't know whether a given reference is null (such as when converting from legacy APIs), use Optional.ofNullable(…).
Avoid storing references to Optional<T> in your business objects, as this can cause problems using certain object storage mechanisms.

In the Real World

The primitive counterparts of Optional<T> (OptionalInt, OptionalLong, and OptionalDouble) have features removed. In most cases stick with Optional<T> instead of the primitive versions.
Retrieving an optional value using Optional.get() can be almost as dangerous as accessing a nullable reference; use a more functional alternate if you can.
When mapping several layers deep into an Optional<T>, if one of the mapping methods itself returns an Optional<T> use Optional.flatMap(…) instead of Optional.map(…).

Think About It

Filtering an optional value makes more sense if you think of an Optional<T> as analogous to a Iterator<T> that can return at most a single object.

Self Evaluation

What part specifically of an API was the Optional<T> class meant to improve?
What are some of the drawbacks of Optional<T>?

Task

Upgrade your PublicationRepository API to use Optional<T> for single-publication lookup by name.

Improve the command-line interface of Booker so that it accepts a --name (-n) parameter to the list command, indicating the name of a single publication to list. If no --name is indicated, list all the publications as you do now. Use a getter method returning Optional<String> in your command line options class to indicate whether a name parameter was passed, trimming the string of beginning and trailing whitespace making use of Optional<T> methods. In the presentation layer operations such as trimming whitespace may be considered normalization when converting from user data to business data. You may need to use quotes when testing from the command line.

booker list [--name <name>]
booker -h | --help

Option	Alias	Description
`list`		Lists all available publications.
`--help`	`-h`	Prints out a help summary of available switches.
`--name`	`-n`	Indicates a filter by name for the `list` command.

References

The Design of Optional (Nocolai Parlog)

Resources

Hamcrest Optional

Optionals

Goals

Concepts

Language

Library

Dependencies

Preview

Lesson

Optional<T>

Checking Value Presence

Optional.isPresent()

Optional.orElse(T)

Optional.orElseThrow(…)

Optional.ifPresent(…)

Processing Values

Mapping

Optional.map(…)

Optional.flatMap(…)

Filtering

Producing an Optional<T>

Drawbacks

Review

Gotchas

In the Real World

Think About It

Self Evaluation

Task

See Also

References

Resources

`Optional<T>`

`Optional.isPresent()`

`Optional.orElse(T)`

`Optional.orElseThrow(…)`

`Optional.ifPresent(…)`

`Optional.map(…)`

`Optional.flatMap(…)`

Producing an `Optional<T>`