Monthly Archives: May 2015

Reasons to Recommend Common Lisp

On Reddit, /u/rhabarba asks: Should I learn (Common) Lisp? Here’s my reply:

I am a technical writer and “hobbyist” programmer who does some unofficial programming at work to automate documentation things. To give you a sense of my language background, I enjoy programming in Perl, Python, Scheme, and Common Lisp.

Reasons to recommend Common Lisp include:

  • In many ways, it’s a superset of the other languages I mentioned: you can “script” with regular expression and filesystem support, and you can also write larger programs using built-in OO that’s still more advanced than the built-in Perl or Python stuff.
  • Multi-paradigm: you can write code in whatever style you want to solve your problem. You only use OO if you want to, you’re not forced by the language. Functional style is there for you, especially via libraries. You can also be imperative and bash as much state as you want. Almost any type of code you might want to write can be (probably has been?) written in Common Lisp. There’s a lot of room to grow as you learn more.
  • Dynamic, interactive, and inspectable: redefine almost anything on the fly at the REPL, watch those changes get picked up by the rest of your system as it runs. This is true for running web servers, whatever. The debugger and inspector available via SLIME in Emacs is also better than anything else I’ve used in other languages.
  • Multiple good implementations to choose from; there are several fast “native” compilers, several portable interpreter-based implementations. If you write portable code you can run on many different implementations, so you can mix and match depending on what’s easiest to install or use for your needs.
  • Quicklisp lets you install any of 1200* libraries, very easily. You’ve got all the basics: HTTP clients and servers, JSON, XML, and Markdown parsers, and lots more advanced stuff too.
  • 25 year old code that does advanced, weird, or just cool things still works fine. It just runs way faster now.
  • Good books: ‘Paradigms of Artificial Intelligence Programming’ by Peter Norvig, ‘Object-Oriented Programming in Common Lisp’ by Sonya Keene, and many others.

Finally, I have been able to write programs to do “harder” things in Lisp than I have in other languages, even though I’m not really a programmer or “engineer” or whatever title you prefer. I think the simple list data structure is a great way to bootstrap a prototype of your program, and it encourages data-structure thinking rather than “stringy” thinking as some languages do. That said, I use a lot of languages to get my work done, so it’s not a religion or anything.

Good luck!

The Sentinel File Pattern

In this short essay I’ll describe the “sentinel file” pattern, which I recently used when writing a command-line tool to use at $WORK for interacting with our web API.

The essence of the sentinel file pattern is that you use a certain file’s last-modified time as a record against which you compare other time-based values.

It is useful in many contexts, such as software builds; in the context of web APIs, it can be used to track whether you will need to reauthenticate with the API before you fire off a bunch of API calls.

The recipe is essentially this:

  • Update a sentinel file F‘s timestamp at time T.
  • When you are about to take an action such as make an API call, see if the current time, T’, is greater than the timeout value of your web API, V, plus the sentinel file’s existing timestamp T.

We can translate this into Scheme as follows (this is scsh, to be exact), where:

;; F = sentinel-file
;; T = (file-last-mod sentinel-file)
;; T' = (time)
;; V = api-timeout-value

(define (sentinel-expired? sentinel-file)
   (> (time)
      (+ (file-last-mod sentinel-file) api-timeout-value)))

Note that TIME and FILE-LAST-MOD are part of the scsh POSIX API.

This pattern is much more efficient than storing some kind of “am I logged in?” value in a JSON/YAML/XML/s-expression config file that has to be read in and parsed on every invocation and written out from time to time.

I debated whether to write about this simple technique at all because it seems like an old trick that many people know. However, I’m going to assume that I am not unique, and that there are lots of people out there who could benefit from using this technique when the right situation arises.

The Debugger is a Notebook

penrose-tiling-based-modular-origami.jpg

My style of programming has changed since the ODB. I now write insanely fast, making numerous mistakes. This gives me something to search for with the ODB. It’s fun.

– Bil Lewis, creator of the Omniscient Debugger

I. Programmers and Poets

In this short essay I will explore some similarities between the way (some) programmers work when doing exploratory programming that can be fruitfully compared to the way poets write poems. I will also sprinkle in some attempts to make the case that a good debugger is core to the experience of composing a new program that you don’t understand very well yet, and compare that with the experience of writing a poem. I know a lot about the former because I am not a very good programmer, so many programs that explore computer science concepts are “exploratory” for me; I know a lot about the latter because I am a reasonably good poet who has written many poems (which is to say, that I have edited many poems, which is really more important).

This work is largely inspired by:

  • The experience of working an programs for SICP exercises and getting popped into the MIT Scheme debugger a lot! 1
  • Using the Scheme 48/scsh inspector a bunch while working on geiser-scsh
  • Writing a lot of poems

II. Generating {Program,Poem} Text

Computer program texts are compressed descriptions of computational processes designed to be experienced by computers and humans. Similarly, poems are compressed descriptions of cognitive and emotional processes designed to be experienced by humans.

Both artifacts strive to encapsulate something that was understood by the author(s) at one point in time and convey that understanding to a reader at another point in time (human or machine). In poetry world, there are a number of different ways to work. There are ostensibly some writers who think really hard for a moment and write a fully-formed sentence. Then they think for a few moments more and write down another fully-formed sentence. And so on.

In reality, there are very few poets who work this way. Most people work using an approximation of what Sussman beautifully describes as “problem-solving by debugging almost-right plans”. 2 This is actually how human beings create new things! As my professor told our writing workshop, “I can’t teach you how to write. I can only teach you how to edit your own work”. Few people write well, and fewer edit well. But in the end, writing and editing are actually the same thing. When you first edit a poem, you may correct minor errors in the text. The more important work is “running the program” of the poem in your head, reading it over and over, reciting it aloud, testing whether it achieves the aesthetic purpose you have set for it. You will add a pronoun in one place, and replace an adjective in another. You might remove the last line, or add another stanza entirely. Do this for long enough, and you may find the poem has changed substantially over the course of having been “debugged”. It may also achieve a goal that you didn’t know existed when you began writing it. I suspect there is something very similar at work when people are doing exploratory programming sessions.

III. Debugger as Crutch/Enabler

Debuggers are seen by some as a crutch. I agree that debuggers are a crutch. There’s a reason crutches were invented. Because without them, you would have to crawl along, dragging your broken leg behind you in the dirt. And we all have a “broken leg” of sorts when we’re working on a problem we don’t understand very well.

I’d like to propose a better metaphor for debuggers. The debugger is a notebook where you can sketch out different versions of your program. You may correct minor errors in a variable declaration, or change a parameter to a procedure. You might redefine an existing procedure as a higher-order procedure that replaces two or three more verbose ones. And so on. All inside the debugger!

A sufficiently powerful debugger will give you the freedom to sketch out an idea quickly, watch it break, and play around in the environment where the breakage occurred, reading variable bindings, entering new definitions, etc.

I think of this style of programming as “sketching in your notebook” because you don’t write poems by staring at a blank sheet of paper for two minutes and then carefully writing a fully-formed sentence. You have an initial idea or impulse, and then you express it as fast as you can! You write down whatever parts of it you can manage to catch hold of, since your brain is working and associating much faster than your pen can really capture. What you end up with is a pile of things, some of which you will discard, some of which are worth keeping just as they are, and some of which are worth keeping but non-optimal and will need to be rewritten. If you actually have an idea worth expressing, you are in much greater danger of not capturing something than you are of making a mess. You will always start by making a mess and then cleaning it up 3.

I submit that a sufficently powerful, expressive debugging environment is as necessary to the programmer as a pocket notebook to the poet.

Interesting Reads

These essays explore writing, debugging, and thinking in more depth:

(Image courtesy fdecomite under Creative Commons License.)

Footnotes:

1

For more information about how cool the MIT Scheme debugger is, see Joe Marshall’s informative blog post.

2

This technique is mentioned on his CSAIL page here. For more information, see the link to his paper elsewhere on this page.

3

You may enjoy an interesting essay with this title: Make a Mess, Clean it Up!