- Table of contents
- 1. Views on LOOP
- 2. The purpose of ITERATE
- 3. Comparing looping clauses
- 4. Documentation
- 5. Obtaining ITERATE
- 6. Conclusion
Looping is the most common non-trivial construct in imperative programming, so having a domain language for generating iteration code is unquestionably useful. I’ll explore two options within Common Lisp:
- LOOP is a standard macro with an expressive syntax and built-in support for several iterative patterns. It’s acclaimed and criticized in equal portion.
- ITERATE is a second looping macro created outside of the standard as an answer to the problems and limitations of LOOP.
Superficially, the differences between them are few:
(loop for i from 0 for el in '(a b c d e f) collect (cons i el)) => ((0 . A) (1 . B) (2 . C) (3 . D) (4 . E) (5 . F)) (iter (for i from 0) (for el in '(a b c d e f)) (collect (cons i el))) => ((0 . A) (1 . B) (2 . C) (3 . D) (4 . E) (5 . F))
The Common Lisp standard also includes the looping macros DO and DO*. They’re widely used but can be terse and obscure; in any case, I’m leaving them out of the comparison.
1. Views on LOOP
The following are the three most frequent criticisms of LOOP:
It doesn’t look like Lisp. The pathological case is hash table iteration. I need to look this up every time:
(loop for key being the hash-keys in *some-table* using (hash-value val) collect (list key val))
As a general compromise, some developers use keyword syntax for LOOP forms, though it’s not a definite improvement:
(loop :for counter :downfrom 20 :downto 0 :by 2 :when (zerop (mod counter 3)) :collect counter) => (18 12 6 0)
- Editors auto-indent LOOP poorly. I work for the preeminent employer of Lisp hackers, yet no one I know has an Emacs configuration which indents all corners of LOOP correctly. (That is, for my definition of correct; there’s no standard.)
- LOOP can behave unpredictably when
forclauses interact in a certain way. However, in my experience this behaviour only presents itself in examples contrived to show it.
See also these articles:
- Dan Weinreb discusses the tradeoffs in accepting LOOP into Common Lisp
- Joe Marshall’s “Why I am a knee-jerk anti-LOOPist”
- Geoff Wozniak’s “In defense of LOOP”.
2. The purpose of ITERATE
The ITERATE website makes two main claims:
- It’s extensible
- It helps editors auto-indent by having a more Lisp-like syntax
Both are true. ITERATE is as much a mini-language as LOOP, but its clauses look like, and frequently are, regular Lisp forms.
;; A LOOP example (loop for i upto 20 if (oddp i) collect i into odds else collect i into evens finally (return (values evens odds))) => (0 2 4 6 8 10 12 14 16 18 20) (1 3 5 7 9 11 13 15 17 19) ;; The equivalent in ITERATE (iter (for i from 0 to 20) (if (oddp i) (collect i into odds) (collect i into evens)) (finally (return (values evens odds)))) => (0 2 4 6 8 10 12 14 16 18 20) (1 3 5 7 9 11 13 15 17 19)
IF above is not an ITERATE construct — it’s the normal Common Lisp operator. The LOOP
if clause will be too, but only after a couple layers of macroexpansion.
Another example, harder to reproduce in LOOP:
(macrolet ((divisorp (n m) `(zerop (mod ,n ,m)))) (iter (for i from 0 to 30) (cond ((divisorp i 2) (collect i into twos)) ((divisorp i 3) (collect i into threes)) ((divisorp i 5) (collect i into fives))) (finally (return (values twos threes fives))))) => (0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30) (3 9 15 21 27) (5 25)
It’s true that distinguishing an ITERATE form from a regular form is less straightforward than it is with LOOP, but in practice the distinction is rarely important. A benefit of the intermixing is that ITERATE clauses such as
collect may appear arbitrarily deep in the body, in contrast to LOOP where they must fall within the mini-language portion.
Also: for comparison with LOOP’s egregious hash table iteration syntax in (1), here is the cleaner equivalent in ITERATE:
(iter (for (key val) in-hashtable *some-table*) (collecting (list key val)))
ITERATE’s extensibility is discussed in-depth in a second article. Through some means, LOOP can apparently be extended, but it’s not done often or easily. SBCL, at least, provides LOOP extension hooks, but using them ties you to that compiler.
3. Comparing looping clauses
In most cases, ITERATE is a superset of functionality.
appending. ITERATE has these and also
(iter (for el in '(a b c a d b)) (adjoining el)) => (A B C D) (iter (for lst in '((a b c) (d b a) (g d h))) (unioning lst)) => (A B C D G H)
accumulating is an accumulator builder. Here’s how to implement
(iter (for lst in '((a b c) (d b a) (g d h))) (accumulating lst by #'union)) => (A B C G D H)
minimizing. ITERATE also includes
reducing is the generalized reduction builder:
(iter (with dividend = 100) (for divisor in '(10 5 2)) (reducing divisor by #'/ initial-value dividend)) => 1
A simple macro to lessen code noise:
(defmacro dividing (num &keys (initial-value 0)) `(reducing ,num by #'/ initial-value ,initial-value)) (iter (for i in '(10 5 2)) (dividing i :initial-value 100)) => 1
(Obviously the above is better stated
(/ 100 10 5 2), but imagine leveraging this clause in a more complicated loop.)
The ITERATE package provides a macro,
DEFMACRO-CLAUSE, to create new clauses more idiomatically; read more about it in the follow up article.
These are the same in both:
thereis, corresponding to the functions
SOME. ITERATE can specify both
never in the same loop, but not to much purpose.
There is no analogue for
finding in LOOP. The ITERATE website provides a good use case:
(iter (for lst in '((a) (b c d) (e f))) (finding lst maximizing (length lst))) => (B C D) ;; The rough equivalent in LOOP: (loop with max-lst = nil with max-key = 0 for lst in '((a) (b c d) (e f)) for key = (length lst) do (when (> key max-key) (setf max-lst lst max-key key)) finally (return max-lst)) => (B C D)
finding is a pattern for operating on the result of one expression based on the result of another.
next-iteration, which is like
continue in C or
next in Perl. It’s a major inconvenience of LOOP that it doesn’t have this construct. ITERATE also offers the
(if-first-time then else) form and the
first-iteration-p var for conditioning on the initial iteration in cases which aren’t covered by patterns.
for clause, ITERATE and LOOP have the same syntax and same destructuring capabilities. However, ITERATE can also “destructure” multiple-value returns:
(for (values (a . b) c d) = (three-valued-function ...))
The consing equivalent in LOOP:
for ((a . b) c d) = (multiple-value-list (three-valued-function ...))
LOOP can destructure within a
with clause, but ITERATE cannot. The manual cites implementation difficulty.
This is what DO does and DO* doesn’t, and it’s strictly unsupported in ITERATE. LOOP includes this optionally with an
(loop for el in '(a b c d e) and prev-el = nil then el collect (list el prev-el)) => ((A NIL) (B A) (C B) (D C) (E D))
The ITERATE documentation states:
“My view is that if you are depending on the serial/parallel distinction, you are doing something obscure”
I have to agree. Also, the useful case of parallel binding in the LOOP example above can be accomplished with another ITERATE concept, variable backtracking:
(iter (for el in '(a b c d e)) (for prev-el previous el) (collect (list el prev-el))) => ((A NIL) (B A) (C B) (D C) (E D))
There are many resources out there for learning to use LOOP. The LOOP for Black Belts chapter of Practical Common Lisp is my favourite. The Common Lisp Quick Reference is also excellent in its treatment of LOOP.
For ITERATE, there really is only the manual, but it’s thorough. It includes another comparison of LOOP and ITERATE. There’s also a
DISPLAY-ITERATE-CLAUSES function which comes with the package and can provide dynamic assistance:
(display-iterate-clauses) INITIALLY Lisp forms to execute before loop starts AFTER-EACH Lisp forms to execute after each iteration ELSE Lisp forms to execute if loop is not entered FINALLY Lisp forms to execute after loop ends ;; ... (display-iterate-clauses 'multiply) MULTIPLY &OPTIONAL INTO Multiply into a variable
It’d be great to see this integrated into SLIME.
5. Obtaining ITERATE
It’s easiest to do this with ASDF-Install. To get ITERATE working in a repl or scratch buffer:
;; (Example below in SBCL.) CL-USER> (asdf:oos 'asdf:load-op :ASDF-INSTALL) ;; stuff CL-USER> (asdf-install:install "ITERATE") ;; lots of stuff ;; (only needs to be done once.) CL-USER> (defpackage "MY-PACKAGE" (:use "CL" "ITERATE")) #<PACKAGE "MY-PACKAGE"> CL-USER> (in-package "MY-PACKAGE") #<PACKAGE "MY-PACKAGE"> MY-PACKAGE> (iter ...) ;; etc.
In almost every case, ITERATE is more convenient to use and more powerful than LOOP. If you’re in control of your own project and aren’t religiously partial to DO (or functional programming), ITERATE is worth a try.
Thanks to Richard Kreuter, Alec Berryman, and John O’Laughlin for their comments and suggestions.