4. Loop Invariants

In the previous lecture, we talked about specifications, documenting comments that describe the behaviors of different units of code. As we proceed in the course, these behaviors will become more complicated and often require one or more intricate loops to achieve. Therefore, it will be helpful to have techniques for reasoning about the behavior of our code within a “loopy” method. Loop invariants allow us to do this. More generally, invariants are properties that we can assert throughout our code’s execution.

By stipulating that an invariant must hold at these “checkpoints”, we force our code into a structure that is easier to reason about, document, and maintain. We’ll see that thinking about “loopy” procedures in terms of invariants will simplify the process of writing loops, allowing us to track multiple variables more easily and avoid off-by-one errors that are pervasive in hastily written code.

Prerequisites

Before we can talk about loop invariants, we’ll standardize some terminology for loops and briefly review Java’s loop syntax. We’ll also introduce some new notation and diagrams to help us talk about ranges of indices in arrays.

Loop Anatomy

Consider the following simple for loop, which adds one to each entry of an int[] array, nums.

We call the code within the loop’s scope (i.e., the code that is between the curly braces delimiting the loop) its body. The code that precedes the body is the loop’s declaration. In a for loop, the declaration includes three pieces, separated by semicolons.

The first piece, “int i = 0”, declares a loop variable, i, and initializes its value to 0.

The second piece, “i < nums.length”, is a boolean expression that tells us when we proceed with another iteration of the loop. We call this condition the loop guard since it “guards” access to the loop body.

The third piece, “i++”, is code that updates the loop variable. It is executed at the end of the loop body.

These same pieces, loop variables, initialization, guard, body, and increment, can also be found in a while loop, just arranged slightly differently.

Rewriting our example code from above gives:

Since we are able to directly translate a for loop into a while loop, we will focus on while loops for the rest of this lecture. In addition, since while loops place the increment step within the loop body, we won’t distinguish it as a separate piece. Rather, we’ll refer to the entire inner scope of the loop (including the increment) as its body.

Range Notation

Many of the loops that we will discuss today and throughout the course will act on arrays or array-like (i.e., linearly ordered) sequences. When talking about these sequences, we will often want to refer to ranges, or contiguous subsequences, of their elements. We’ll use a standard notation for these ranges that borrows from mathematical interval notation. For example, the notation a[2..4] refers to the three-element subsequence consisting of a[2], a[3], and a[4]. Here, the .. notation denotes the “interval” of all indices between 2 and 4, and the surrounding square brackets indicate that this interval includes both endpoints. We use round brackets when we want to exclude an endpoint, so a[2..4) consists of only a[2] and a[3] (it excludes the endpoint a[4]), whereas a(2..4) consists of only a[3] (it excludes both endpoints).

We will sometimes use some shorthand when an endpoint falls at the boundary of the array. When we omit the left number in range notation, this means “starting from the first index”, so a[..3] would include a[0], a[1], a[2], and a[3]. Similarly, a(..3) would include a[1] and a[2] but exclude the endpoints a[0] and a[3]. When we omit the right number in range notation, this means “ending at the last index”, so a[4..] would consist of a[4] through and including a[a.length-1].

It is possible to use range notation to refer to an empty range, which is signified by any interval whose first included index is greater than its last included index. Some such notation is a[..0), a(2..2), or a[3..2]. A lot of these rules may seem a bit strange or arbitrary now, but as you start using them to document the code that you write, they should become more natural.

Array Diagrams

It will also be useful to draw pictures that visualize properties of various ranges of an array. We do this using array diagrams. These diagrams are different from the memory diagrams we have seen so far. They extract away a lot of lower-level details (such as the objects, types, runtime stack, memory heap, etc.) and leave only a visualization of the array itself. We use vertical lines to delineate ranges of the array and use labels at the top of these lines to clarify these delineations. The ranges (boxes) of the array are labeled with properties.

For example, if we have an array a with capacity 8 in which a[..3) are even and a[3..] are odd, we can visualize this with the array diagram:

As we will see later, we’ll often have loop variables labeling array diagrams that represent the state of the array after some iteration of the loop. For example, the diagram

will appear in our analysis of the binary search algorithm and indicates that the entries in a[..l) are all less than v, the entries in a[r..] are all greater than or equal to v, and we don’t know anything about the entries in a[l..r).

Writing “Loopy” Code

Now that we have introduced some vocabulary and notation to reason about loops, let’s put it into practice to develop the following method from the perspective of loop invariants. While this method may seem simple, and while the procedure that we follow will likely feel excessive, it is important to practice techniques on small examples that we can then extend to more complicated examples that require them. Consider the following method specification:

To compute the frequency of key, we will need to compare it to each element of a, which will require iteration over a. To start developing our loop, we should think about what local variables we will need to declare. In other words, what will we need to keep track of as we proceed with the iteration? In this case, we need to keep track of which entries of a we have checked and which entries we have not. If we choose to scan the entries in order, from left to right, we can accomplish this with an int variable i that represents the next index that we will inspect. We’ll also need to keep a running count of how many times we have seen key during our scan. We can do this with another int variable count.

Visualizing Loop Behavior

The next thing that we’ll do is use array diagrams to visualize the state of the array over the course of the loop. It’s usually easiest to start by drawing the state at the very beginning of the loop (just before we evaluate the loop guard for the first time) and at the very end of the loop (at the point where the loop guard evaluates to false).

For our frequencyOf() example, the “Pre” diagram is pretty basic. Since we haven’t looked at any entries of a, and since our method spec doesn’t assert any pre-conditions about a, we know nothing.

At the end of the loop, we expect that our count variable will hold the number of occurrences of key in a.

The last picture that we’ll draw will visualize the state of the array and local variables of the method at the start of an arbitrary loop iteration. In this case, since the loop variable i is keeping track of our progress, we’ll consider the state of the program when i has a certain value. Since we will scan over the array from left to right, we will have checked the array cells with indices 0 through i-1, in our range notation, a[..i), but will not yet have looked at a[i..]. In this way, i delineates a boundary between two array ranges, and this boundary will move to the right as time advances. We’ll visualize by drawing a rightward arrow below the boundary line in our diagram.

Given a particular value of i at some point during the loop’s execution, how should we interpret the value of count at this point? Since we have scanned the entries to the left of the i boundary, count is the number of times that key occurs among the elements we’ve checked, a[..i). Visually,

This picture illustrates the main property that is maintained (i.e., invariant) throughout the loop: whenever we evaluate the loop guard, count will equal the number of occurrences of key in a[..i). This property is the invariant of this loop.

From Array Diagrams to Code

Let’s use our loop invariant and its diagram to write the frequencyOf() method. We’ll start by placing the loop invariant in a multi-line comment at the top of a while loop stub.

It is good programming practice to document your loop invariants, as this will give other developers (or even you in the future, when your loop code is not fresh in your mind) an easy way to understand the design of the loop. Now, we can use the loop invariant to help write each part of the loop: the initialization, loop guard, and loop body (plus increment).

Initialization: The loop invariant must be true when we enter the loop

Before we enter the loop, we must write initialization statements for all of the variables involved in the invariant (i and count). We must choose their initial values so that the loop invariant is true the first time that the loop guard is evaluated. In our diagram, we can visualize this by sliding the boundary (or multiple boundaries in more complicated examples we’ll see later) in the opposite direction of their arrows (i.e., backwards in time).

When we do this, we obtain the “Pre” array diagram. Moreover, we see that the boundary i overlaps with array index 0, meaning i should be initialized to 0 when we enter the loop. This makes sense. Since we haven’t looked at any entries of a yet and we plan to scan a from left to right, the first index we will inspect is i = 0.

Now, we can use the loop invariant to figure out how to initialize the local variable count. Plugging in i=0 to the loop invariant, we must initialize count to the number of occurrences of key in a[..0). Since this is notation for the empty range, which cannot contain any occurrences of key, we should set count = 0. Hooray! The loop invariant is true at the start.

Loop Guard: If the loop invariant is true when we exit the loop, we have computed what we wanted to.

We’ll exit the loop after we have inspected all of the entries of a. This corresponds with pushing our array boundary as far as we can in the direction of its arrow (i.e., forward in time). At this point, the boundary coincides with the right edge of the array, and we recover the “Post” diagram.

We see that the loop variable i overlaps with a.length, so the loop should end once i == a.length. Recall that we exit the loop the first time that the loop guard evaluates to false, meaning we should continue to loop as long as i < a.length (or i != a.length). This makes sense. The loop guard will remain true as long as i, the next index we want to inspect, i, is a valid array index.

Now, we can use the loop invariant to figure out what our return value should be after we exit the loop. Plugging in i=a.length to the loop invariant, we see that count will be equal to the number of occurrences of key in a[..a.length) when we exit the loop. This is range notation for the entire array a, meaning count will hold the exact number we wanted to compute. Thus, we can return count after breaking out of the loop and satisfy the method post-condition.

Loop Body: Our loop makes progress toward termination in each iteration and re-establishes the loop invariant before re-evaluating the loop guard.

We’ve gotten to the main portion of the loop, the body. We enter the body immediately after evaluating the loop guard, so the loop invariant will be true. During one iteration of the loop body, we need to accomplish two things. First, we need to make some measurable progress toward our goal. This will help guarantee that our loop will eventually terminate. Then, we also need to make sure that we re-establish the loop invariant. In our frequencyOf() example, we make progress by inspecting one more element of a, a[i], which allows us to increment i at the end of the body (the next element we’ll check moves one cell to the right). However, when we update the value of i, we may also need to update the value of count to preserve the loop invariant. How do we do this? If a[i] == key, we’ve found another occurrence, so we should increment count by 1. Otherwise, if a[i] != key, we don’t need to do anything; the value of count remains correct even after incrementing i. This suggests the following loop body.

Recapping This Example

The above example illustrates the benefits of developing loop invariants and array diagrams; once you set things up correctly, everything you need to write the loop perfectly on the first try (no off-by-one indexing errors) is there for you, as long as you know where to look. In particular, you can use the following steps to write your loopy code.

1. Think about how you’ll process the data to carry out the desired behavior of the loop. As the loop proceeds, what will you know about different ranges of the array? Use this to begin sketching the “Inv” array diagram.
2. Identify the local variables that you’ll need to track your progress during the loop. There will be one loop variable for each boundary between array ranges (which should be written on one side of the boundary). Sometimes, additional variables will be needed to model the properties of the array ranges. Use this information to finish drawing the “Inv” diagram, which should include all of these variables.
3. Use the “Inv” array diagram to write the loop invariant, documenting it as a comment at the start of your loop skeleton.
4. Slide the “Inv” diagram boundaries in the opposite direction of their arrows. Confirm that the resulting diagram accurately depicts the “Pre” state (it may help to draw a separate “Pre” diagram as we did above) of the loop. Use the overlaps in the “Pre” diagram to initialize the loop variables, and use the loop invariant to initialize any other local variables.
5. Slide the “Inv” diagram boundaries in the direction of their arrows. Confirm that the resulting diagram accurately depicts the “Post” state. Use the overlaps in the “Post” diagram to derive the loop termination condition, and negate this to obtain the loop guard. Add additional code after the loop to satisfy the method post-conditions.
6. Develop the loop body so that it makes progress toward the post-condition. Make sure that it re-establishes the loop invariant by the end of the iteration.

For the rest of the lecture, we’ll look at two more examples of developing “loopy” code with this process. We will continue to make extensive use of loop invariants throughout the course; hopefully, you will be convinced that they are a useful tool that you will adopt throughout your CS career.

More Examples

Argmin

Let’s define the following method according to its specification.

First, let’s think about the local variables that we’ll need in this method. Again, we’ll need to scan over the contents of array a in search of the smallest element, and we’ll use a loop variable i to keep track of the next index to visit. We’ll also need to keep track of the smallest value we’ve seen, since we can use this to identify the minimum element. We can do this with a double variable min. While it may seem like this is sufficient, we’ll need an additional variable if we’d like to compute the argmin using only one pass over the array. The method returns the index of the minimum element, not its value, so we’ll use an int variable loc to keep track of the location of the smallest value that we’ve seen.

This results in the following invariant diagram.

Let’s take a second to think through the condition in the left array range. We want our local variable min to model the smallest value that we’ve seen so far. How can we express this as a property of an array range, which is what we write in a loop invariant diagram? Well, no value that we’ve seen could be smaller than min (otherwise, we would have updated min to that smaller value). Said differently, every value that we’ve seen, that is, the entire range a[..i), must be $\geq$ min.

The invariant on loc is that it always stores an array index where min occurs (i.e., a[loc] = min). This is not a property of an array range, so we wrote this part of the invariant beneath the diagram.

Now, let’s think about the “Pre” diagram and the array initialization. As we slide the diagram boundary to the left, it shrinks the left range. However, we can’t push it all the way to the left edge (i = 0). We must initialize min to a value in the array, so we need to read one array entry before entering the loop. It is most natural to read a[0]. Then, min = a[0] will be the smallest value we’ve seen and loc = 0 is the index where min can be found. This coincides with sliding the diagram to i = 1, which makes sense. Index 1 will be the first one inspected when we enter the loop since we already inspected index 0 during the initialization.

Next, let’s think about the “Post” diagram. This is much easier. Just like in our frequencyOf() example, we’ll exit the loop once we’ve inspected the entire array and slid the boundary all the way to the right (we should guard the loop on the condition i < a.length). At this point, min is the true minimum array value, meaning loc stores the argmin and is the return value.

Step through the following animation to see how we can develop the method definition using the information from these diagrams, following the steps outlined in the previous section.

Partitioning

Let’s define the following method according to its specification.

This method partitions, or splits, the elements of the array into two distinct ranges: a range of even numbers and a range of odd numbers. How can we accomplish this task? Over the course of the method, we will need to inspect each element of the array to check its parity. We can again accomplish this by scanning the elements from left to right, using an int variable i to keep track of the next index to inspect. To carry out the partitioning, we can imagine “growing” two ranges from the edges of the array: a range of all the even numbers starting from the left and a range of all the odd numbers starting from the right.

The variable i can represent the “frontier” of this left range a[..i), and we’ll need a second variable j to track the “frontier” of the right range a[j..]. Within our paritySplit() method, we will be rearranging the entries of a, which we’ll do as a series of element swaps. To keep the body of this method focused, let’s extract this swapping code out into a separate helper method. The easiest way to swap the contents of two variables is to introduce a third temporary variable to hold one of the values while we swap over the other.

Step through the animation below to walk through the development of this loop.

Exercises