An Iterative Change-Counting Procedure

… the process is really iterative: Its state is captured completely
by its three state variables, and an interpreter need keep track of
only three variables in order to execute the program.

– SICP (1st ed.), p.33

After much head-scratching and experimentation, I have implemented the following iterative change-counting procedure, which I call `cc-linear` (See also previously, previously):

```(define (cc-linear amt kinds)
(define (cc-linear-aux amount kinds-of-coins count stack)
(cond ((or (= amount 0)
(= kinds-of-coins 1))
(if (null? stack)
(+ count 1)
(let* ((new-amt-pair (car stack))
(new-stack (cdr stack)))
(cc-linear-aux (car new-amt-pair) (cdr new-amt-pair) (+ 1 count) new-stack))))
((or (< amount 0)
(= kinds-of-coins 0))
(if (null? stack)
count
(let ((new-amt-pair (car stack))
(new-stack (cdr stack)))
(cc-linear-aux (car new-amt-pair) (cdr new-amt-pair) count new-stack))))
(else (cc-linear-aux amount
(- kinds-of-coins 1)
count
(cons (cons (- amount (first-denomination kinds-of-coins))
kinds-of-coins)
stack)))))
(cc-linear-aux amt kinds 0 '()))
```

Note that the definition of the `first-denomination` procedure is given in the text, as well as being downloadable from the SICP website.

This procedure returns the correct answer. There are a few issues with it, however:

• It uses `cons`, `let*`, `car`, `cdr`, and `null?`, none of which are available to the student at this point in the text.
• It simulates a stack by passing the list of delayed computations as an argument to subsequent invocations of itself. This means that it doesn’t generate a “linear iterative” process, since the amount of space used is not constant.

As for the use of `car`, `cdr`, and friends, I’m willing to let it pass since I still learned a lot by forcing myself to think in terms of iterating via recursive procedure calls.

As far as the shape of the process being generated is concerned, it can be described as “resembling linear recursion”. In the course of implementing this solution I wrote out the following simulation by hand of changing \$0.12 using only two kinds of coins, a nickel and a penny.

Pre-visualizing the process to be generated by `cc-linear’

(12 2) 0 ()
( 2 2) 0 ((12 1))
(-3 2) 0 ((2 1) (12 1))
( 2 1) 0 ((12 1))
( 1 1) 0 ((2 0) (12 1))
( 0 1) 0 ((2 0) (12 1))
( 2 0) 1 ((12 1))
(12 1) 1 ()
(11 1) 1 ((12 0))
(10 1) 1 ((11 0) (12 0))
( 9 1) 1 ((10 0) (11 0) (12 0))
( 8 1) 1 (( 9 0) (10 0) (11 0) (12 0))
( 7 1) 1 (( 8 0) ( 9 0) (10 0) (11 0) (12 0))
( 6 1) 1 (( 7 0) ( 8 0) ( 9 0) (10 0) (11 0) (12 0))
( 5 1) 1 (( 6 0) ( 7 0) ( 8 0) ( 9 0) (10 0) (11 0) (12 0))
( 4 1) 1 (( 5 0) ( 6 0) ( 7 0) ( 8 0) ( 9 0) (10 0) (11 0) (12 0))
( 3 1) 1 (( 4 0) ( 5 0) ( 6 0) ( 7 0) ( 8 0) ( 9 0) (10 0) (11 0) (12 0))
( 2 1) 1 (( 3 0) ( 4 0) ( 5 0) ( 6 0) ( 7 0) ( 8 0) ( 9 0) (10 0) (11 0) (12 0))
( 1 1) 1 (( 2 0) ( 3 0) ( 4 0) ( 5 0) ( 6 0) ( 7 0) ( 8 0) ( 9 0) (10 0) (11 0) (12 0))
( 0 1) 1 (( 1 0) ( 2 0) ( 3 0) ( 4 0) ( 5 0) ( 6 0) ( 7 0) ( 8 0) ( 9 0) (10 0) (11 0) (12 0))
( 1 0) 2 (( 2 0) ( 3 0) ( 4 0) ( 5 0) ( 6 0) ( 7 0) ( 8 0) ( 9 0) (10 0) (11 0) (12 0))

Et cetera…

As you can see, the stack continues to grow, and passing that huge association list in as a procedure argument is not without cost, as the following performance analysis shows (for the definitions of `with-timing-output` and `cc-faster`, see here.):

Performance analysis of `cc’ vs. `cc-faster’ vs. `cc-linear’ – `cc-linear’ spends a fair amount of time in garbage collection, presumably due to all of the list manipulation.

(with-timing-output (cc 292 5))
Run time: 3.42
GC time: .03
Actual time: 3.478
;Value: 8526

(with-timing-output (cc-faster 292 5))
Run time: .05
GC time: 0.
Actual time: .047
;Value: 8526

(with-timing-output (cc-linear 292 5))
Run time: .11
GC time: .04
Actual time: .15
;Value: 8526

Attempting to Count Change, Iteratively

I have discovered what I believe is a nice optimization for the tree-recursive change-counting procedure, `cc`, mentioned in a previous post. (In the first edition, it’s Exercise 1.9.)

Unfortunately I’m still struggling with the real problem: how to implement this problem to evolve an iterative process, as requested by the book. My attempts to recast the computation using only variables passed as procedure arguments are failing. These attempts always need more “registers” than I have available, and I start thinking in terms of some kind of “storage” where I can stash the intermediate problems that I can’t work on yet.

Of course, these “registers” are arguments to the function, and this “storage” is also known as the stack. And growing the stack is what we’re trying to avoid.

Draw the tree illustrating the process generated by the (`cc`) procedure … making change for 11 cents.

Here’s what appeared in my notebook – Note that the procedure calls surrounded by boxes are the ones that return a value of 1. In this case, the return value of `(cc 11 3)` is 4, shown by the 4 boxed procedure calls:

While drawing out this tree, I noticed that once the procedure tries to make change for an `amount` using only pennies, it makes `(* 2 amount)` unnecessary procedure calls. It’s easy to see that they just bang down the list from `amount` to 0 and finally return 1. My idea was to reformulate the procedure so that we perform this check first, like so:

```(define (cc-faster amount kinds-of-coins)
(cond ((= kinds-of-coins 1) 1) ; Are we making change with pennies? If so, there's only one way to do it!
((= amount 0) 1)
((or (< amount 0)
(= kinds-of-coins 0))
0)
(else (+
(cc-faster amount (- kinds-of-coins 1))
(cc-faster (- amount (first-denomination kinds-of-coins))
kinds-of-coins)))))
```

Calling this procedure results in the following tree of procedure calls:

Note that instead of 55 procedure calls, we now make only 15! And the discrepancy only grows wider as we count larger amounts of change, prompting me to make the following measurements using MIT Scheme – the `with-timing-output` macro is described at the bottom:

```(with-timing-output (cc 700 5))
; Run time:     75.81
; GC time:      .34
; Actual time:  76.641
;Value: 207530

(with-timing-output (cc-faster 700 5))
; Run time:     .56
; GC time:      .02
; Actual time:  .608
;Value: 207530
```

This reduces running time from over a minute to less than a second, which is kind of exciting! It’s still not an iterative process, though.

Just for completeness, here’s how long the same procedure takes using `m-cc`, the memoizing version of `cc` described previously. Obviously looking stuff up in memory is faster than computing it over and over again:

```(with-timing-output (m-cc 700 5))
; Run time:     0.
; GC time:      0.
; Actual time:  0.
;Value: 207530
```

And here’s the definition of the `with-timing-output` macro. It’s just sugar on top of the `with-timings` measurement procedure built into MIT Scheme:

```(define-syntax with-timing-output
(syntax-rules ()
((with-timing-output body)
(with-timings
(lambda () body)
(lambda (run-time gc-time real-time)
(display "Run time:\t")
(write (internal-time/ticks->seconds run-time))
(newline)
(display "GC time:\t")
(write (internal-time/ticks->seconds gc-time))
(newline)
(display "Actual time:\t")
(write (internal-time/ticks->seconds real-time))
(newline))))))
```

Next step: ITERATION!

Working through SICP for the first time. Found a used first edition (hardcover, no less) on eBay. Some misguided person had marked each of the covers up with a giant X in black marker.

The first edition differs from the second in a few ways. For example, at the end of the tree-recursive change-counting problem on pages 39-40 of the second edition, the authors say “it is not obvious how to design a better algorithm for computing the result, and we leave this problem as a challenge”. They also note that memoization can be used to reduce the number of redundant computations that occur.

In other words, we can trade space for time.

In the first edition, Exercise 1.9 asks you to design a procedure that evolves an iterative process for solving the change-counting problem. I’m working on a solution to that problem; in the meantime, I implemented a procedure that I call `memoizing-apply` which does the following:

1. See if an answer has already been computed and stored in a table `*mr*` (short for `memoized-results’).
2. If the answer hasn’t already been computed, compute it. Then stick the answer in the table `*mr*`.

This process is known as memoization or tabulation, and `*mr*` serves as a lookup table.

We define a table `*mr*`, and a procedure `memoized?` that checks if the procedure has already been called with the current arguments. This assumes that the procedure always returns the same answer when you give it the same arguments.

```(define *mr* '())

(define (memoized? args)
(let ((result (assoc args *mr*)))
(if result
(cdr result)
#f)))

```

`Memoizing-apply` does what we described above: it checks if the answer has already been computed (is already in the lookup table), and if so it just returns the answer. If not, it computes the answer and sticks it in the table.

```(define (memoizing-apply f xs)
(or (memoized? xs)
(let ((result (apply f xs)))
(begin (push! (cons xs result) *mr*)
result))))

```

Note that `push!` is not a standard Scheme procedure; here’s the definition:

```(define-syntax push!
(syntax-rules ()
((push! item seq)
(set! seq (cons item seq)))))

```

Let’s try it out. (Note that you can get the definitions for the procedures `cc` and `first-denomination` from a freely available copy of the book available at the SICP website.)

Here’s my modified version of `cc`, the change-counting procedure. I call it `m-cc`, and it uses the `memoizing-apply` procedure above to avoid doing redundant work:

```(define (m-cc amount kinds-of-coins)
(cond ((= amount 0) 1)
((or (< amount 0) (= kinds-of-coins 0)) 0)
(else (+
(memoizing-apply m-cc (list amount (- kinds-of-coins 1)))
(memoizing-apply m-cc (list (- amount (first-denomination kinds-of-coins))
kinds-of-coins))))))

```

The above code is different from the original in that it uses `memoizing-apply` to make the recursive call to itself. In my testing this is much faster than the equivalent `cc` version.

The memoization technique works very nicely, but it’s only appropriate in situations where memory is freely available. On most computers, most of the time, it’s probably the best way to go, since the tree-recursive implementation is readable and easy to understand. The exercise that asks you to solve the problem using an iterative process will likely be more complicated, but I’m not sure yet.