Macro-writing Macros

John Jacobsen
Wednesday, November 25, 2015

Home All Posts

Earlier post Lazy Physics code clojure

Later post Questions to Ask art

… in which we explore the power of macros, and macro-writing macros, to DRY out repetitive code.


I’ve been writing Clojure code full time for nearly two years now. I have a pretty good feel for the language, its virtues and its faults. Mostly, I appreciate its virtues (though I still wish the REPL started faster).

For me one of the language’s attractions has always been that it’s a Lisp — a “homoiconic” language, i.e., one defined in terms of its own data structures. Homoiconicity has one primary virtue, which is that it makes metaprogramming more powerful and straightforward than it is in non-homoiconic languages (arguably at some cost to readability).

In Lisp, this metaprogramming is accomplished with macros, which are functions that transform your code during a separate stage of compilation. In other words, you write little programs to change your programs before they execute. In effect, you extend the compiler itself.

I run a Clojure study group at work and find that it can be hard to explain the utility (or appeal) of this to newcomers to Lisp. This is partly because macros do things you can’t easily do in other languages, and because the things you want to do tend to relate to abstractions latent in a particular codebase.

While playing around with 3d rendering in Quil, I recently came across a use case that reminded me of the following quote by Paul Graham:

The shape of a program should reflect only the problem it needs to solve. Any other regularity in the code is a sign, to me at least, that I’m using abstractions that aren’t powerful enough— often that I’m generating by hand the expansions of some macro that I need to write1.

In Quil, there are multiple situations in which one needs to create a temporary context to carry out a series of operations, restoring the original state afterwards:

  1. Save current style with push-style; change style and draw stuff; restore previous style with pop-style.
  2. Start shape with begin-shape; draw vertices; end-shape to end.
  3. Save current position/rotation with push-matrix; translate / rotate and draw stuff; restore old position/rotation with pop-matrix.

Here’s an example:

    (fill 255)
    (translate [10 10 10])
      (vertex x1 y1 0)
      (vertex x2 y2 0)
      (vertex x2 y2 h)
      (vertex x1 y1 h)
      (vertex x1 y1 0)

The (try ... (finally ...)) constructions may not be strictly needed for a Quil drawing, but it’s a good habit to guarantee that stateful context changes are undone, even if problems occur.

In a complex Quil drawing the idioms for saving style, translation state, and denoting shapes appear often enough that one hungers for a more compact way of representing each. Here’s one way to do it:

(defmacro with-style [& body]

(defmacro with-matrix [& body]

(defmacro with-shape [& body]

The original code then becomes more compact and easier to read:

    (fill 255)
    (translate [10 10 10])
      (vertex x1 y1 0)
      (vertex x2 y2 0)
      (vertex x2 y2 h)
      (vertex x1 y1 h)
      (vertex x1 y1 0))))

In this example code, the contexts with-matrix, etc. appear so often that the resulting savings in lines of code and mental overhead for the reader is substantial.

However, the astute reader will realize that the macro definitions themselves are pretty repetitive—in fact, they look almost identical except for the setup and teardown details (this kind of “context manager” pattern is common enough that Python has its own language construct for it).

I generally reach for macros when I have a pattern that occurs with obvious repetition that’s not easy to abstract out using just pure functions. Control abstractions such as loops or exception handling are common examples. (I find this situation occurs especially frequently when writing test code).

In any case, the solution for our repetitive macros could be something like:

(defmacro defcontext
  [nom setup teardown]
  `(defmacro ~(symbol (str “with-” nom))
     [~'& body#]

Yikes! I have to admit I had to write a lot of macros, and also refer to this helpful page for reference, before I could write (and grok) this macro.

With defcontext in hand, our repetitive macro code just becomes:

(defcontext style (push-style) (pop-style))
(defcontext shape (begin-shape) (end-shape))
(defcontext matrix (push-matrix) (pop-matrix))

These are exactly equivalent to the three context macros (with-*) defined above.

With a little effort, it’s actually not too hard to construct such a nested macro. It’s largely a matter of writing out the code you want to generate, and then writing the code that generates it, testing with macroexpand-1 at the REPL as you go. This page by A. Malloy has a lot of helpful remarks, including this cautionary note: “Think twice before trying to nest macros: it’s usually the wrong answer.” In this case, I actually think it’s the right answer, because the pattern of a context with setup and teardown is so common that I know I’ll reuse this macro for many other things—we have effectively added one of my favorite Python features to Clojure in just a few lines of code2

There’s a saying in the Clojure community: data > functions > macros. I’m a big believer in this. Clojure’s powerful built-in abstractions for wrangling data in all its forms make it the language I prefer above all others these days. But occasionally that means wrangling the data that is the code itself, thereby reaping the benefits in power, brevity and expressiveness.


Figure 2: Image generated by the Quil code used for this example.



Paul Graham, Revenge of the Nerds.


To be even more like Python’s context managers, defcontext would want to enable the user to bind some local state resulting from the setup phase of execution (“ with x() as y: ” idiom); examples include file descriptors or database connections. This is left as an exercise for the reader.

Earlier post Lazy Physics code clojure

Later post Questions to Ask art

Blog Posts (140)


Reflections on a Year of Daily Memory Drawings art

Repainting art

Daily Memory Drawings art

Questions to Ask art

Lazy Physics code clojure

Fun with Instaparse code clojure

Nucleotide Repetition Lengths code clojure

Updating the Genome Decoder code clojure

Getting Our Hands Dirty (with the Human Genome) code clojure

Validating the Genome Decoder code clojure

A Two Bit Decoder code clojure

Exploratory Genomics with Clojure code clojure

Rosalind Problems in Clojure code clojure

Introduction to Context Managers in Python code python

Processes vs. Threads for Integration Testing code python

Resources for Learning Clojure code clojure

Continuous Testing in Python, Clojure and Blub code clojure python

Programming Languages code

Milvans and Container Malls southpole

Oxygen southpole

Ghost southpole

Turkey, Stuffing, Eclipse southpole

Wind Storm and Moon Dust southpole

Shower Instructions southpole

Fresh Air and Bananas southpole

Traveller and the Human Chain southpole

Reveille southpole

Drifts southpole

Bon Voyage southpole

A Nicer Guy? southpole

The Quiet Earth southpole

Ten southpole

ISO50 southpole art

In Defense of Hobbies misc code art

Closure southpole

Takeoff southpole

Mummification southpole

Eleventh Hour southpole

Diamond southpole

Baby, It’s Cold Outside southpole

Fruition southpole

Settling In southpole

Revolution Number Nine southpole

Red Eye to McMurdo southpole

Faults in Ice and Rock southpole

Bardo southpole

Chasing the Sun southpole

Downhole southpole

Coming Out of Hibernation southpole

Managing the Most Remote Data Center in the World code southpole

Photoshop on a Dime art

Man on Wire misc

Posing Rigs art

Metric art

Cuba southpole

Wickets southpole

Safe southpole

Broken Glasses southpole

End of the Second Act southpole

Pigs and Fish southpole

Last Arrivals southpole

Lily White southpole

In a Dry and Waterless Place southpole

Immortality southpole

Routine southpole

Tourists southpole

Passing Notes southpole

Translation southpole

The Usual Delays southpole

RNZAF southpole

CHC southpole

Wyeth on Another Planet art

Detox southpole

Packing southpole

Nails southpole

Gearing Up southpole

Gouache, and a new system for conquering the world art

Fall 2008 HPAC Studies art

YABP (Yet Another Blog Platform) southpole

A Bath southpole

Green Marathon southpole

Sprung southpole

Outta Here southpole

Lame Duck DAQer southpole

Eclipse Town southpole

One More Week southpole

IceCube Laboratory Video Tour; Midrats Finale southpole

SPIFF, Party, Shift Change southpole

Good things come in threes, or 18s southpole

Sleepless in the Station southpole

Post Deploy southpole

Midrats southpole

IceCube and The Beatles southpole

Video: Flight to South Pole southpole

The Pure Land southpole

Almost There southpole

There are no mice in the Hotel California Bunkroom southpole

Short Timer southpole

Sleepy in MacTown southpole

Superposition of Luggage States southpole

Sir Ed southpole

Shortcut to Toast southpole

Pynchon, Redux southpole

Flights: Round 1 southpole

Packing for the Pole southpole

Goals for Trip southpole

Balaklavas southpole

Tree and Man (Test Post) southpole

Schedule southpole

How to mail stuff to John at the South Pole southpole

Summer and Winter southpole

Homeward Bound southpole

Redeployment southpole

Short-timer southpole

The Cleanest Air in the World southpole

One more day (?) southpole

One more week (?) southpole

Closing Softly southpole

More Photos southpole

Super Bowl Wednesday southpole

Night Owls southpole

First Week southpole

More Ice Pix southpole

Settling In southpole

NPX southpole

Pole Bound southpole

Bad Dirt southpole

The Last Laugh southpole

Nope southpole

First Delay southpole

Batteries and Sheep southpole

All for McNaught southpole

The Big (Really really big…) Picture southpole

t=0 southpole

Giacometti southpole

Descent southpole

Video Tour southpole

How to subscribe to blog updates southpole

What The Blog is For southpole