3. Method Specifications and Testing

Most likely, the code that you have written in the past has been somewhat small in scope, containing only a few code files. You likely developed this code yourself, understood how all the parts worked, and understood how they worked together to achieve the goal of your program. Having a “complete picture” of the code becomes impractical or even impossible in larger commercial software, which may contain millions (or billions) of lines of code spread across thousands of functions in thousands of separate code files. In these large projects, each developer or development team is responsible for writing and maintaining a small part of the application. Often, this segment is not isolated but instead interacts with parts of the application written by other developers or teams. To make this work, software engineers need to adopt good development practices to facilitate seamless interactions across the code base. One foundational aspect of this is developing clear and thorough specifications for their code. Another is carefully testing the code that they write to ensure that it meets its specifications. This lecture centers on these strategies.

Method Specifications

Code specifications (or documentation) are a textual description of the intended behavior of a piece of code.

In today’s lecture, we’ll focus on method specifications. Typically, our specifications take the form of comments within our code. Java provides different types of comments.

1. Inline comments begin with a double slash, // and turn the rest of that line into a comment (i.e., text that does not affect the execution of the code). They are useful for clarifying the behavior of that code line. In particular, inline comments help future maintainers of your code (who may be other developers or you when the code is not fresh in your mind) to follow your reasoning. Inline comments are not required on every line of code; doing this would make the code less readable. Instead, they should be used sparingly where clarification is warranted.

2. Multi-line comments are enclosed in /* ... */ punctuation. They are useful for documentation within a method that spans more than one line. As an example, we will often use multi-line comments to document loop invariants, which are the subject of our next lecture.

3. JavaDoc comments are another type of multi-line comment that are distinguished by different opening punctuation: /** ... */ (note the two asterisks). JavaDoc is a tool that automatically generates documentation webpages from the contents of these comments. For example, the main Java language specification was created using the JavaDoc tool. We will write JavaDoc comments above each method we write to describe its specifications.

To talk about specifications, we need to introduce two roles that describe individuals who may interact with a piece of code, the implementer and the client.

For example, in the previous lecture, we acted as clients of Java’s String class when we constructed String objects and invoked methods on them. The implementers of the String class were developers of the Java language. In our first lecture, where we developed a method daysToMinutes() to convert units, we were both the implementer of this method (when filling in its definition) and also its client (when we called it from our main() method).

An implementer is responsible for providing specifications for the code that they write, and the client is responsible for reading these specifications to ensure that they use the code correctly. As such, the specifications can be thought of as a contract between the implementer and the client. Next, we’ll talk about what information belongs in this contract.

Pre-conditions and Post-conditions

What information does a client need to know to use a method?

1. They need to know the method’s name.
2. They need to know how many parameters the method has and what types each of these parameters has.
3. They need to know what type the method returns.

These first three things are all conveyed by the method’s signature. A related notion that we will discuss in more detail soon is that the client should be aware of the possibility of exceptional behavior that can arise within the method. Beyond knowing the types of the inputs and outputs of a method, the client should know how to interpret these values.

4. The client needs to understand how each of the parameters will be interpreted by the method, and whether there are extra requirements on the values of the parameters beyond their types.
5. The client needs to understand what properties they can expect of the return value, such as how it relates to the parameters that were passed in.

Finally, beyond the return value, a method may have side-effects, meaning it may change the state of some objects stored on the heap (possibly separate from the object referenced by the return value). Since these changes may not be directly visible by inspecting the return value, it is important that the client is made aware of such side-effects in the method specifications.

6. The client needs to understand any method side-effects.

Stepping back, we can arrange these pieces of the specifications into two broad categories, pre-conditions and post-conditions.

All the pre-conditions and post-conditions of a method must be described in its specifications. When this is done, the method outlines a contract of the form “If you (the client) satisfy all the pre-conditions when you call this method, then I (the implementer) will satisfy all the post-conditions when I return from the method.”

Let’s look at some examples of writing method specifications. First, let’s consider the following method:

1

static boolean uppercaseAt(String str, int i) { ... }

1

static boolean uppercaseAt(String str, int i) { ... }

This method should return true if the character in str at position i is uppercase, otherwise it should return false. For this behavior to make sense, there is an implicit relationship between the values of (the String object referenced by) str and i: i must be at least 0 (the leftmost index in str) and at most str.length() - 1 (the rightmost index in str). This relationship is a pre-condition that we must document in the specifications. The post-condition of the method consists only of its return value; there are no side-effects. Thus, we can document the method as follows:

1
2
3
4
5

/** 
 * Returns `true` if the character in `str` at index `i` is uppercase, otherwise 
 * returns `false`. Requires that `0 <= i < str.length()`.
 */
static boolean uppercaseAt(String str, int i) { ... }

1
2
3
4
5

/** 
 * Returns `true` if the character in `str` at index `i` is uppercase, otherwise 
 * returns `false`. Requires that `0 <= i < str.length()`.
 */
static boolean uppercaseAt(String str, int i) { ... }

Notice that the specifications are written as a series of complete English clauses. Most of the time the clauses begin with a verb (i.e. “Requires”, “Returns”) that describes some aspect of the method. “Returns” clauses document the return value. “Requires” clauses document pre-conditions. Additionally, the specifications provide us with the information that we need to correctly interpret each parameter. str is the String that we will query at one index, and i is that index.

Dealing with Pre-condition Violations

What should this method do when an illegal value of i is passed in? The short answer is “whatever the implementer wants;” by violating the pre-condition, the client opened the door for undefined behavior. However, there are some common approaches.

1. The implementer can do nothing to account for the possibility of a pre-condition violation and simply write the method definition under the assumption that the pre-condition is met:

1
2
3
4
5
6
7

/** 
 * Returns `true` if the character in `str` at index `i` is uppercase, otherwise 
 * returns `false`. Requires that `0 <= i < str.length()`.
 */
static boolean uppercaseAt(String str, int i) {
  return Character.isUpperCase(str.charAt(i));
}

1
2
3
4
5
6
7

/** 
 * Returns `true` if the character in `str` at index `i` is uppercase, otherwise 
 * returns `false`. Requires that `0 <= i < str.length()`.
 */
static boolean uppercaseAt(String str, int i) {
  return Character.isUpperCase(str.charAt(i));
}

If we dig into the charAt() documentation we see that an IndexOutOfBoundsException will be thrown if the pre-conditions are violated.

2. The implementer can use a Java assert statement to detect the pre-condition violation.

1
2
3
4
5
6
7
8

/** 
 * Returns `true` if the character in `str` at index `i` is uppercase, otherwise 
 * returns `false`. Requires that `0 <= i < str.length()`.
 */
static boolean uppercaseAt(String str, int i) {
  assert 0 <= i && i < str.length();
  return Character.isUpperCase(str.charAt(i));
}

1
2
3
4
5
6
7
8

/** 
 * Returns `true` if the character in `str` at index `i` is uppercase, otherwise 
 * returns `false`. Requires that `0 <= i < str.length()`.
 */
static boolean uppercaseAt(String str, int i) {
  assert 0 <= i && i < str.length();
  return Character.isUpperCase(str.charAt(i));
}

The semantics of the assert keyword specify that the expression following it must have type boolean. If this expression evaluates to true then nothing happens; the code proceeds as normal. If this expression evaluates to false, then Java throws an AssertionError, which should crash the program.

We call this approach “defensive programming”, since the implementer is defending against pre-condition violations with the heavy-handed approach of crashing the program when one is detected.

This is our preferred approach for handling pre-condition violations in CS 2110.

3. The implementer can throw a “more appropriate” exception.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

/** 
 * Returns `true` if the character in `str` at index `i` is uppercase, otherwise 
 * returns `false`. Requires that `0 <= i < str.length()`.
 */
static boolean uppercaseAt(String str, int i) {
  if (i < 0 || i >= str.length()) {
    throw new IllegalArgumentException("`i` must be between 0 and `str.length()-1`.");
  }
  return Character.isUpperCase(str.charAt(i));
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

/** 
 * Returns `true` if the character in `str` at index `i` is uppercase, otherwise 
 * returns `false`. Requires that `0 <= i < str.length()`.
 */
static boolean uppercaseAt(String str, int i) {
  if (i < 0 || i >= str.length()) {
    throw new IllegalArgumentException("`i` must be between 0 and `str.length()-1`.");
  }
  return Character.isUpperCase(str.charAt(i));
}

Doing this is a bad programming practice. If we are going out of our way to throw an exception, it is better to expose this to the user in the method specifications (in a throws clause). We will discuss exceptions more in an upcoming lecture.

4. The implementer can write some “hacky error-preventing code” that allows the program to continue.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

/** 
 * Returns `true` if the character in `str` at index `i` is uppercase, otherwise 
 * returns `false`. Requires that `0 <= i < str.length()`.
 */
static boolean uppercaseAt(String str, int i) {
  if (i < 0) {
    i = 0;
  } else if (i >= str.length()) {
    i = str.length() - 1;
  }
  return Character.isUpperCase(str.charAt(i));
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

/** 
 * Returns `true` if the character in `str` at index `i` is uppercase, otherwise 
 * returns `false`. Requires that `0 <= i < str.length()`.
 */
static boolean uppercaseAt(String str, int i) {
  if (i < 0) {
    i = 0;
  } else if (i >= str.length()) {
    i = str.length() - 1;
  }
  return Character.isUpperCase(str.charAt(i));
}

Doing this is a very dangerous programming habit, so we strongly recommend against it. This code essentially removes the pre-condition from the method without revealing this in the specifications. Bugs in the client code (perhaps the client intended to meet the pre-condition but had an arithmetic off-by-one error) can go unnoticed since the program will continue to execute. This may lead to more confusing unintended behaviors downstream in the program from this method call.

Specifying Side-Effects

Let’s take a look at a second example of a method’s specifications, this time for a method with a side-effect.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

/**
 * Zeroes-out all entries before and including the first instance of `key` in 
 * `arr` and returns the index where `key` was found. If `key` is not present in 
 * `arr`, then all indices of `arr` are zeroed out, and `arr.length` is returned.
 */
 static int zeroThrough(int[] arr, int key) {
  int i = 0;
  int val;
  while(i < arr.length) {
    val = arr[i];
    arr[i] = 0;
    if (val == key) {
      return i;
    }
    i++;
  }
  return i;
 }

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

/**
 * Zeroes-out all entries before and including the first instance of `key` in 
 * `arr` and returns the index where `key` was found. If `key` is not present in 
 * `arr`, then all indices of `arr` are zeroed out, and `arr.length` is returned.
 */
 static int zeroThrough(int[] arr, int key) {
  int i = 0;
  int val;
  while(i < arr.length) {
    val = arr[i];
    arr[i] = 0;
    if (val == key) {
      return i;
    }
    i++;
  }
  return i;
 }

This method includes both a return value (the index of the first instance of key) and a side-effect (zeroing out entries of arr), which are documented as separate clauses in the specifications. A corner-case of the method is the possibility that key is not present in arr, and this behavior is called out in a second sentence of the specifications. There are no pre-conditions of this method. Any int[] array and any int key may be passed in.

Let’s trace through the execution of this code on a smallish input. This offers some good review of objects in memory diagrams that we discussed last lecture as well as tracing through loops that we will discuss next lecture.

1
2
3
4

void example() {
  int[] nums = {4, 3, 5, 6, 1};
  int loc = zeroThrough(nums, 5);
}

1
2
3
4

void example() {
  int[] nums = {4, 3, 5, 6, 1};
  int loc = zeroThrough(nums, 5);
}

Unit Testing

As we are developing a program, we’d like a way to check that it is working correctly (i.e., it conforms to its specifications). We can imagine doing this by running the program to verify that it has the correct behavior; however, there are issues with this approach. Early in the development, we may not have a runnable program. We might not have written a main() method, or this method may rely on other pieces of the code that do not work yet (imagine the “real-world” scenario where we are writing one file out of thousands in the final application). Furthermore, if we are able to run the program and find that it doesn’t have the expected behavior, how can we know whether this is the fault of the code that we wrote versus code that was written by other developers? This situation motivates a need for unit testing.

To perform this isolated testing, we will need a new way to run pieces of our code without the need to cumbersomely assemble a full application. A Java testing framework called JUnit affords us this ability.

JUnit

JUnit is a flexible unit testing framework that has built-in support in many integrated development environments (IDEs) such as IntelliJ. JUnit tests are written in separate files under the green “tests” folder of your IntelliJ project. We often name these files “<class name>Test.java” where “<class name>.java” is the file containing the code we are writing tests against. For example, if we suppose that our uppercaseAt() and zeroThrough() methods were written in a Specs class in the Specs.java file, we’d write their unit tests in the SpecsTest class in a SpecsTest.java file located in the “tests” folder. JUnit is driven through Java annotations, a new-to-us Java feature that allows us to decorate entities such as class and method declarations. When we use JUnit, we will annotate methods to signify that they are unit tests. Below you will find one complete example of a JUnit test for our uppercaseAt() method.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

import static org.junit.jupiter.api.Assertions.*;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

public class SpecsTest {

    @DisplayName("WHEN we call `uppercaseAt()` with a String with an uppercase " 
                 + "character at position i, THEN it should return true.")
    @Test
    void testUppercaseAtTrue() {
        assertTrue(Specs.uppercaseAt("Hello", 0));
    }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

import static org.junit.jupiter.api.Assertions.*;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

public class SpecsTest {

    @DisplayName("WHEN we call `uppercaseAt()` with a String with an uppercase " 
                 + "character at position i, THEN it should return true.")
    @Test
    void testUppercaseAtTrue() {
        assertTrue(Specs.uppercaseAt("Hello", 0));
    }
}

Let’s break down the different components of this code.

• The import statements at the top of the file (outside of the SpecsTest) class are the mechanism for using code from another source (in this case, the JUnit library) within our Java file. In this case, the top import statement allows us to call assertTrue() within our test method, and the other two import statements allow us to use the @DisplayName and @Test annotations.

• The testUppercaseAtTrue() method is decorated with the @Test annotation. This tells JUnit that this method should be treated as a unit test during its execution. JUnit will run this test and report back a status (either a passed test, a failed test, or a runtime error) in its test results output.

• The testUppercaseAtTrue() method is also decorated with the @DisplayName annotation. This provides a description of what the test is checking and often takes the form “WHEN <set up> THEN <expected result>.” The display name is also used by IntelliJ to format the test results output.

• The testUppercaseAtTrue() method has the void return type. This is required by JUnit and also makes sense intuitively. A test case is run for its side-effect (of verifying that another piece of code is working as expected), not for any return value.

• The test method includes an assertTrue() statement. JUnit uses assert...() methods to determine whether tests pass or fail. In the case of assertTrue(), JUnit will evaluate the boolean expression passed in as an argument. If it is true, the method returns successfully. If it is false, then JUnit produces an exception that marks the test as failed. When all assert...() methods called within an @Test method return successfully, the test “passes”.

• Note that within the assertTrue() call, we reference the uppercaseAt() method with a new syntax, Specs.uppercaseAt(). This is the way that we reference a static method that is located within another file: <class name>.<static method name>(<arguments>).

When we develop a unit test, we start with the display name, coming up with a description for one test that we’d like to perform against our code unit. Typically, it is good practice to make the behavior that we are testing as simple as possible and write many tests that each cover one simple behavior. This allows us to use the passage/failure of each test to diagnose fine-grained issues in our code. Once we have written the display name for a test, we think about assertions that we can write to confirm the behavior we are testing.

JUnit Assertions

An assertion typically consists of an expression derived from the unit we are testing (the “actual” value) and the value that we expect the expression to evaluate to (the “expected” value). Critically, the expected value should be determined based on specifications, not by tracing through the code we are testing. Remember, unit tests verify that code conforms to its spec, so we focus on the spec when we write them. Typically, the expected value is hard-coded into the test method.

Oftentimes, it will be appropriate to include multiple, related assertions in the same unit tests. For example, we can flesh out the above unit test to:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

@DisplayName("WHEN we call `uppercaseAt()` with a String with an uppercase " 
             + "character at position i, THEN it should return true.")
@Test
void testUppercaseAtTrue() {
    assertTrue(Specs.uppercaseAt("Hello", 0));
    assertTrue(Specs.uppercaseAt("APPLE", 1));
    assertTrue(Specs.uppercaseAt("APPLE", 4));
    assertTrue(Specs.uppercaseAt("baNana", 2));
    assertTrue(Specs.uppercaseAt("1d5nal#fRa3", 8));
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

@DisplayName("WHEN we call `uppercaseAt()` with a String with an uppercase " 
             + "character at position i, THEN it should return true.")
@Test
void testUppercaseAtTrue() {
    assertTrue(Specs.uppercaseAt("Hello", 0));
    assertTrue(Specs.uppercaseAt("APPLE", 1));
    assertTrue(Specs.uppercaseAt("APPLE", 4));
    assertTrue(Specs.uppercaseAt("baNana", 2));
    assertTrue(Specs.uppercaseAt("1d5nal#fRa3", 8));
}

We can also write a similar test that covers the case where uppercaseAt() returns false.

1
2
3
4
5
6
7
8
9

@DisplayName("WHEN we call `uppercaseAt()` with a String with a non-uppercase " 
             + "character at position i, THEN it should return false.")
@Test
void testUppercaseAtFalse() {
    assertFalse(Specs.uppercaseAt("Hello", 1));
    assertFalse(Specs.uppercaseAt("apple", 0));
    assertFalse(Specs.uppercaseAt("L8R", 1));
    assertFalse(Specs.uppercaseAt("mWe75^H", 5));
}

1
2
3
4
5
6
7
8
9

@DisplayName("WHEN we call `uppercaseAt()` with a String with a non-uppercase " 
             + "character at position i, THEN it should return false.")
@Test
void testUppercaseAtFalse() {
    assertFalse(Specs.uppercaseAt("Hello", 1));
    assertFalse(Specs.uppercaseAt("apple", 0));
    assertFalse(Specs.uppercaseAt("L8R", 1));
    assertFalse(Specs.uppercaseAt("mWe75^H", 5));
}

JUnit provides a variety of assertion methods that allow us to write comprehensive unit tests. We’ll continue to introduce more as they become relevant. For now, the most important assertion method(s) will be assertEquals(). Most assertEquals() methods take two arguments. The first argument is the “expected” value, and the second argument is the “actual” value. It is crucial that you order these arguments correctly so that the testing output you receive is accurate: expected first, actual second.

Below, we give examples of some unit tests for the zeroThrough() method that cover various aspects of its specifications.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

@DisplayName("WHEN we call `zeroThrough()` and the first entry of the array is "
             + "the key, THEN it should return 0.")
@Test
void testZeroThroughFirstReturned() {
    assertEquals(0, Specs.zeroThrough(new int[]{1, 2, 3, 4, 5}, 1));
}

@DisplayName("WHEN we call `zeroThrough()` and the last entry of the array is "
             + "the key, THEN it should return the last index.")
@Test
void testZeroThroughLastReturned() {
    assertEquals(4, Specs.zeroThrough(new int[]{1, 2, 3, 4, 5}, 5));
}

@DisplayName("WHEN we call `zeroThrough()` and the key is not present in the "
             + "array, THEN the array length should be returned.")
@Test
void testZeroThroughUnfoundReturn() {
    assertEquals(5, Specs.zeroThrough(new int[]{1, 2, 3, 4, 5}, 6));
}

@DisplayName("WHEN we call `zeroThrough()` and the key is present multiple " 
             + "times, THEN the first index should be returned.")
@Test
void testZeroThroughMultipleKey() {
    assertEquals(1, Specs.zeroThrough(new int[]{1, 2, 3, 2, 1}, 2));
}

@DisplayName("WHEN we call `zeroThrough()` and the key appears once " 
             + ", THEN the key entries and all entries to its left are "
             + "zeroed out, and no entries to its right are zeroed out.")
@Test
void testZeroThroughZeroesCorrectly() {
    int[] nums = {1, 2, 3, 4, 5}; // save reference in local variable for later
    int loc = Specs.zeroThrough(nums,3);
    for (int i = 0; i <= loc; i++) {
      assertEquals(0, nums[i]);
    }
    for (int j = loc + 1; j < nums.length; j++) {
      assertNotEquals(0, nums[j]);
    }
}

@DisplayName("WHEN we call `zeroThrough()` and the key does not appear " 
             + "in the array, THEN the entire array is zeroed out.")
@Test
void testZeroThroughZeroesAll() {
    int[] nums = {1, 2, 3, 4, 5}; // save reference in local variable for later
    Specs.zeroThrough(nums,6); // don't need to store return value
    for (int i = 0; i < nums.length; i++) {
      assertEquals(0, nums[i]);
    }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

@DisplayName("WHEN we call `zeroThrough()` and the first entry of the array is "
             + "the key, THEN it should return 0.")
@Test
void testZeroThroughFirstReturned() {
    assertEquals(0, Specs.zeroThrough(new int[]{1, 2, 3, 4, 5}, 1));
}

@DisplayName("WHEN we call `zeroThrough()` and the last entry of the array is "
             + "the key, THEN it should return the last index.")
@Test
void testZeroThroughLastReturned() {
    assertEquals(4, Specs.zeroThrough(new int[]{1, 2, 3, 4, 5}, 5));
}

@DisplayName("WHEN we call `zeroThrough()` and the key is not present in the "
             + "array, THEN the array length should be returned.")
@Test
void testZeroThroughUnfoundReturn() {
    assertEquals(5, Specs.zeroThrough(new int[]{1, 2, 3, 4, 5}, 6));
}

@DisplayName("WHEN we call `zeroThrough()` and the key is present multiple " 
             + "times, THEN the first index should be returned.")
@Test
void testZeroThroughMultipleKey() {
    assertEquals(1, Specs.zeroThrough(new int[]{1, 2, 3, 2, 1}, 2));
}

@DisplayName("WHEN we call `zeroThrough()` and the key appears once " 
             + ", THEN the key entries and all entries to its left are "
             + "zeroed out, and no entries to its right are zeroed out.")
@Test
void testZeroThroughZeroesCorrectly() {
    int[] nums = {1, 2, 3, 4, 5}; // save reference in local variable for later
    int loc = Specs.zeroThrough(nums,3);
    for (int i = 0; i <= loc; i++) {
      assertEquals(0, nums[i]);
    }
    for (int j = loc + 1; j < nums.length; j++) {
      assertNotEquals(0, nums[j]);
    }
}

@DisplayName("WHEN we call `zeroThrough()` and the key does not appear " 
             + "in the array, THEN the entire array is zeroed out.")
@Test
void testZeroThroughZeroesAll() {
    int[] nums = {1, 2, 3, 4, 5}; // save reference in local variable for later
    Specs.zeroThrough(nums,6); // don't need to store return value
    for (int i = 0; i < nums.length; i++) {
      assertEquals(0, nums[i]);
    }
}

Note that we include tests that cover multiple possible scenarios for the parameters (the key is present once, present multiple times, or not present) and make assertions about both the return value and side-effects of the method. In short, we have written tests that cover all aspects of the specifications. As we proceed in the course, we will talk more about designing tests that provide adequate coverage.

1
2
3
4
5
6

@Test
void testZeroThroughZeroesCorrectly() {
  int[] nums = {1, 2, 3, 4, 5};
  Specs.zeroThrough(nums,3);
  assertArrayEquals(new int[]{0, 0, 0, 4, 5}, nums);
}

1
2
3
4
5
6

@Test
void testZeroThroughZeroesCorrectly() {
  int[] nums = {1, 2, 3, 4, 5};
  Specs.zeroThrough(nums,3);
  assertArrayEquals(new int[]{0, 0, 0, 4, 5}, nums);
}

Unit Testing and Pre-conditions

We just noted the importance of writing tests that cover all aspects of a method’s specifications. This calls into question the way that we should handle pre-conditions. Should we write tests that cover the possibility of precondition violations? For example:

1
2
3
4
5

@Test
void testInvalidIndex() {
  boolean b = Specs.uppercaseAt("Apple",5);
  // some assertion about b
}

1
2
3
4
5

@Test
void testInvalidIndex() {
  boolean b = Specs.uppercaseAt("Apple",5);
  // some assertion about b
}

No! Remember, the contract of a method is “If you (the client) satisfy all the pre-conditions when you call this method, then I (the implementer) will satisfy all the post-conditions when I return from the method.” Violating pre-conditions opens the door for undefined behavior. The method can do whatever it wants and still satisfy the spec. Since there is no “expected” behavior outlined by the spec, there are no valid assertions that we can make about the post-conditions of this function call. Unit tests must always satisfy pre-conditions.

Underspecification

Sometimes, the specifications of a method may allow for multiple different satisfactory definitions. In the case of this ambiguity, we may say that the method is underspecified. For example, consider the following specification:

1
2
3
4
5
6

/**
 * Returns an index `i` with `0 < i < a.length-1` that corresponds to a 
 * local maximum of array `a`, meaning `a[i] > a[i-1]` and `a[i] > a[i+1]`.
 * Returns 0 if `a` does not contain a local maximum.
 */
static int findLocalMax(int[] a) { ... }

1
2
3
4
5
6

/**
 * Returns an index `i` with `0 < i < a.length-1` that corresponds to a 
 * local maximum of array `a`, meaning `a[i] > a[i-1]` and `a[i] > a[i+1]`.
 * Returns 0 if `a` does not contain a local maximum.
 */
static int findLocalMax(int[] a) { ... }

Ambiguity arises because the specifications do not pin down which local maximum index will be returned. As an example, in the array {1, 2, 4, 3, 6, 5, 1} both indices 2 and 4 correspond to local maxima. Which will be returned? Will it be the smallest one? The largest one? The one corresponding to the largest value of a? Any of these conforms to the spec, which only requires that a local maximum is returned. Since unit tests check that the spec is satisfied, any tests that we write for this method must be robust to both possible return values.

How do we accomplish this? We cannot use two separate assertEquals() calls, since these will execute sequentially. If the first one fails, an error will be thrown that will halt execution of the test method and mark the test as a failure. Moreover, the return type of JUnit’s assert...() methods is void, so it’s unclear how we would combine these calls to determine the success of the test.

A second potential strategy is to use boolean operations to construct an expression involving both possibilities that we can pass into assertTrue().

1
2
3
4
5
6
7
8
9

@DisplayName("WHEN an array with multiple local maxima is "
             + "passed into `findLocalMax()`, THEN the index of one "
             + "of these maxima is returned.")
@Test
void testMultipleMaxima() {
  int[] a = {1, 2, 4, 3, 6, 5, 1};
  int i = findLocalMax(a);
  assertTrue(i == 2 || i == 4);
}

1
2
3
4
5
6
7
8
9

@DisplayName("WHEN an array with multiple local maxima is "
             + "passed into `findLocalMax()`, THEN the index of one "
             + "of these maxima is returned.")
@Test
void testMultipleMaxima() {
  int[] a = {1, 2, 4, 3, 6, 5, 1};
  int i = findLocalMax(a);
  assertTrue(i == 2 || i == 4);
}

The findLocalMax() spec suggests an alternate strategy. It provides us criteria that the return value i will satisfy, a[i] > a[i-1] and a[i] > a[i+1]. We can check these criteria directly within our unit test to verify that the return value conforms to the spec.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

@DisplayName("WHEN an array with multiple local maxima is "
             + "passed into `findLocalMax()`, THEN the index of one "
             + "of these maxima is returned.")
@Test
void testMultipleMaxima() {
  int[] a = {1, 2, 4, 3, 6, 5, 1};
  int i = findLocalMax(a);
  assertTrue(a[i] > a[i-1]);
  assertTrue(a[i] > a[i+1]);
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

@DisplayName("WHEN an array with multiple local maxima is "
             + "passed into `findLocalMax()`, THEN the index of one "
             + "of these maxima is returned.")
@Test
void testMultipleMaxima() {
  int[] a = {1, 2, 4, 3, 6, 5, 1};
  int i = findLocalMax(a);
  assertTrue(a[i] > a[i-1]);
  assertTrue(a[i] > a[i+1]);
}

In settings with more significant ambiguity (What if there were 100 local maxima in the array?), this is a much more elegant approach.

Underspecified methods underscore the distinction between specifications and implementation. When we develop tests, we focus on the specifications. These outline properties of our code guarantees to the client and set the contract for future developers who may update the code we have implemented. As an implementer, we are free to add or leverage additional properties (that don’t contradict the spec) to improve the understanding of our code, but such properties cannot be assumed in our tests. To conclude today’s lecture, we’ll discuss a technique for maintaining this separation during the development process.

Test-Driven Development

To write unit tests for a method, we need only the method’s specifications, not its definition (the code contained in the method’s body). This suggests the possibility of the following development process:

1. Identify the need for a new method in your code.
2. Write out a method signature identifying the name, return value, and parameter names/types for the method.
3. Write the method specifications that describes what the method will do (its pre-conditions and post-conditions).
4. Develop a set of unit tests that will confirm that the method definition conforms to its specifications.
5. Starting with a “trivial” method definition (just a return statement), run the tests and use failing tests to identify a bit of missing functionality (i.e., deviation from the spec) in the method. Update the code to address this missing functionality and rerun the test to confirm that it now passes.
6. Repeat step 5, iteratively building up the method definition. Once all the tests pass (and assuming you have a comprehensive set of tests), you will be confident that your code is working correctly.

This process is referred to as test-driven development.

Test-driven development (TDD) offers us many benefits over writing code first and testing it later.

• By using TDD, we focus our efforts on satisfying the specifications, rather than writing code “blindly” first and then going back and trying to shoehorn it into the specifications later.
• TDD forces us to think about corner cases and different scenarios upfront (as we are developing the test cases), which often lets us write more elegant code. Discovering corner cases after the “main” code is written can lead to hacky patches and busy code.
• TDD helps facilitate a separation of responsibilities. Some people can focus on developing specifications and unit tests and hand off the responsibility to other developers who focus on implementation.
• TDD ensures that the tests we write are against the specifications of a method and not its implementation details. Early in a programmer’s career, it is often tempting to write tests that essentially “mirror” the logic of the source code, leading to tests that pass but do not actually verify what we want. Writing the tests first, before the source code exists, prevents this possibility.

This last point is an important one and connects to some other testing terminology.

At this point, we are primarily concerned with black box testing, but we will see some use cases for glass box testing later in the course.

Exercises

1
2
3

static int factorial(int n) {
  return n < 2 : 1 ? n * factorial(n-1);
}

1
2
3

static int factorial(int n) {
  return n < 2 : 1 ? n * factorial(n-1);
}

1
2
3
4
5
6
7
8
9

static int firstNum(String s) {
  for (int i = 0; i < s.length(); i++) {
    char c = s.charAt(i);
    if (c >= '0' && c <= '9') {
      return c - '0';
    }
  }
  return -1;
}

1
2
3
4
5
6
7
8
9

static int firstNum(String s) {
  for (int i = 0; i < s.length(); i++) {
    char c = s.charAt(i);
    if (c >= '0' && c <= '9') {
      return c - '0';
    }
  }
  return -1;
}

1
2
3
4
5
6
7

static void prefixMax(int[] a) {
  for (int i = 1; i < a.length; i++) {
    if (a[i] < a[i - 1]) {
      a[i] = a[i - 1];
    }
  }
}

1
2
3
4
5
6
7

static void prefixMax(int[] a) {
  for (int i = 1; i < a.length; i++) {
    if (a[i] < a[i - 1]) {
      a[i] = a[i - 1];
    }
  }
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

static int cleanData(double[] data) {
  int numCorrections = 0;
  for (int i = 0; i < data.length; i++) {
    if (data[i] < 0) {
      data[i] = 0;
      numCorrections++;
    }
  }
  return numCorrections;
}

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

static int cleanData(double[] data) {
  int numCorrections = 0;
  for (int i = 0; i < data.length; i++) {
    if (data[i] < 0) {
      data[i] = 0;
      numCorrections++;
    }
  }
  return numCorrections;
}

1
2
3
4
5

/**
 * Returns the entry with the largest absolute value contained in array `arr`.
 * Requires that `arr.length >= 1` (i.e., `arr` is non-empty).
 */
int maximumAbsoluteValue(int[] arr) { ... }

1
2
3
4
5

/**
 * Returns the entry with the largest absolute value contained in array `arr`.
 * Requires that `arr.length >= 1` (i.e., `arr` is non-empty).
 */
int maximumAbsoluteValue(int[] arr) { ... }

1
2
3
4
5
6

@Test
  void testMaximumIsNegative() {
      int[] arr = {0, 5, -6};
      assertEquals(-6, maximumAbsoluteValue(arr));
  }
  

1
2
3
4
5
6

@Test
  void testMaximumIsNegative() {
      int[] arr = {0, 5, -6};
      assertEquals(-6, maximumAbsoluteValue(arr));
  }
  

1

System.out.println(0.1 + 0.1 + 0.1);

1

System.out.println(0.1 + 0.1 + 0.1);

  0.30000000000000004
  

1
2
3
4
5

/**
 * Returns an array `output` such that `output` consists of the distinct 
 * elements of `input` in some order.
 */
static int[] removeDuplicates(int[] input) { ... }

1
2
3
4
5

/**
 * Returns an array `output` such that `output` consists of the distinct 
 * elements of `input` in some order.
 */
static int[] removeDuplicates(int[] input) { ... }

1

static boolean isValidOutput(int[] input, int[] output) { ... }

1

static boolean isValidOutput(int[] input, int[] output) { ... }

1
2
3
4
5
6
7

/**
 * Returns the longest String `p` that is a prefix of both given Strings `s` 
 * and `t`. A String `p` is a prefix of a String `s` if `p = s.substring(0,i)`
 * for some `i`. For example, `longestCommonPrefix("cactus", "cabbage") = "ca"`.
 * Requires that `s.length() > 0` and `t.length() > 0`.
 */
public String longestCommonPrefix(String s, String t) { ... }

1
2
3
4
5
6
7

/**
 * Returns the longest String `p` that is a prefix of both given Strings `s` 
 * and `t`. A String `p` is a prefix of a String `s` if `p = s.substring(0,i)`
 * for some `i`. For example, `longestCommonPrefix("cactus", "cabbage") = "ca"`.
 * Requires that `s.length() > 0` and `t.length() > 0`.
 */
public String longestCommonPrefix(String s, String t) { ... }

1
2
3
4
5
6
7
8

/**
 * Shifts the entries of `a` to the left to fill in the spaces previously 
 * occupied by 0s. For example, the array {1,0,2,0,0,3} would become 
 * {1,2,3,x,x,x} with any elements occupying its last three indices. 
 * Returns the number of non-zero entries originally in `a`.
 * Requires that `a.length > 0`.
 */
public int condense(int[] a) { ... }

1
2
3
4
5
6
7
8

/**
 * Shifts the entries of `a` to the left to fill in the spaces previously 
 * occupied by 0s. For example, the array {1,0,2,0,0,3} would become 
 * {1,2,3,x,x,x} with any elements occupying its last three indices. 
 * Returns the number of non-zero entries originally in `a`.
 * Requires that `a.length > 0`.
 */
public int condense(int[] a) { ... }

1
2
3
4
5
6
7

/**
 * Locates the first instance of the smallest entry of `a` and moves this to 
 * the first index, rotating the other entries one index to the right to make 
 * space. For example, {2,3,4,1,5,6,1} would become {1,2,3,4,5,6,1}.
 * Requires that `a.length > 0`.
 */
public int selectSmallest(int[] a) { ... }

1
2
3
4
5
6
7

/**
 * Locates the first instance of the smallest entry of `a` and moves this to 
 * the first index, rotating the other entries one index to the right to make 
 * space. For example, {2,3,4,1,5,6,1} would become {1,2,3,4,5,6,1}.
 * Requires that `a.length > 0`.
 */
public int selectSmallest(int[] a) { ... }

1
2
3
4
5
6

/**
 * Returns the character with the greatest number of occurrences in the given 
 * String `s`. If there are multiple such characters, returns the one that 
 * appears first in `s`. Requires that `s.length() > 0`. 
 */
public char mostFrequent(String s) { ... }

1
2
3
4
5
6

/**
 * Returns the character with the greatest number of occurrences in the given 
 * String `s`. If there are multiple such characters, returns the one that 
 * appears first in `s`. Requires that `s.length() > 0`. 
 */
public char mostFrequent(String s) { ... }