Surveying Emacs Lisp

Posted in Uncategorized on March 31st, 2009

Emacs is well known for being highly configurable. In large part, this is due to its close tie to some form of Lisp over the majority of its long and varied history. Within all extant implementations of the editor, the Lisp is Emacs Lisp, a language which is hard to love but easy to like.

1. Overview

Emacs Lisp (inconsistently abbreviated to Elisp) is descended from MacLisp and looks a lot like Common Lisp.

(defun fibonacci (n)
  (if (< n 2)
      1
    (+ (fibonacci (- n 1))
       (fibonacci (- n 2)))))

Being a Lisp, it is inherently extensible, and so is well-suited as an embedded language. Actually, a case can be made that Emacs Lisp is more powerful than many general-purpose programming languages. Some of its notable features:

  • lambdas and first-class functions
  • macros
  • byte-code compilation
  • an interactive shell/REPL (through the *scratch* buffer)
  • an integrated debugger and profiler

Considering its complete isolation to a sub-platform, Emacs Lisp is very successful. Tools that have been written in it include several IRC clients, a popular mail/news reader, two different terminal emulators, a web browser, and even a video editor. Here are several excellent programming resources I’ve found helpful:

2. Portability considerations

Emacs users tend to customize their editor to a great extent, so upgrading to a new version can be a dog, and therefore extension writers are quite conservative about portability.

And alas, the situation is doubly complicated. Developers must consider not only compatibility between versions (e.g. 21 to 22), but also compatibility between variants, as GNU Emacs and XEmacs are both popular.[1] My informal survey of ITA Software, an Emacs-heavy development house, seems to show that engineers stick with what they know; those who picked up Emacs during the periods GNU Emacs lagged behind XEmacs continue to use XEmacs, while most of the others use GNU Emacs. It’s split pretty evenly.

It would be useful to identify the actual usage breakdown among the different versions of the editor, but these numbers do not exist; Debian Popularity Contest can at least provide an estimate:

Tracked usage of Emacs packages in Debian
Package Installs Notes
emacsen-common 15890 (required package for all Emacs installs)
emacs21 7522 GNU
emacs22 4738 GNU
xemacs21 2211
emacs-snapshot 427 GNU (at the time of writing, v23.0)

Likewise, Ubuntu Popularity Contest:

Tracked usage of Emacs packages in Ubuntu
Package Installs Notes
emacsen-common 86805 (required package for all Emacs installs)
emacs21 32358 GNU
emacs22 16640 GNU
xemacs21 7394
emacs-snapshot 4764 GNU (at the time of writing, v23.0)

Again, at best these tables should be considered datapoints in a larger census, but they do seem to show GNU Emacs as significantly more popular than XEmacs, and 21 as the widest-used GNU Emacs version. (Can anyone provide additional stats?)

[1] Other variants such as Aquamacs may deserve a mention here as well — I have no information about their install bases.

3. A short case study

Two years ago I wrote a filesystem explorer / buffer switcher plugin for Vim and then one year ago brought it to Emacs. It had been written in Ruby, which has rough language parity with Emacs Lisp. (No macros vs. no closures.)

i. Initial development

Since Emacs Lisp is not really object-oriented, I dropped the original formal inheritance model. Also, several classes were removed, as existing functionality in Emacs — stuff which did not exist in Vim — made them redundant.

A minimal port was finished in a few hours and 200 lines-of-code. The current version sits at 494 lines. For comparison, the Vim plugin (on which development continues in parallel) is 1466 lines.

From the Vim plugin, here is an inefficient Ruby function to convert an array of strings into column_count columns (reading downward, i.e. the way ls works):

def columnize(strings, column_count)
  rows = (strings.length / Float(column_count)).ceil
 
  # Break the array into sub arrays representing columns
  cols = strings.inject([[]]) { |array, e|
           if array.last.size < rows
             array.last << e
           else
             array << [e]
           end
           array
         }
 
  return cols
end

And below, the close equivalent in Emacs Lisp. It uses push+nreverse instead of append (<<) above, which obfuscates the algorithm a little:

(defun columnize (strings column-count)
  "Break the list STRINGS into sublists representing columns."
  (let ((nrows (ceiling (/ (length strings)
                           (float column-count)))))
    (nreverse
     (mapcar 'nreverse
             (reduce (lambda (lst e)
                       (if (< (length (car lst))
                              nrows)
                           (push e (car lst))
                         (push (list e) lst))
                       lst)
                     strings
                     :initial-value (list (list)))))))

There is room for improvement in both cases — 10 pride points to the person who codes up the best rewrite of either function. :-)

So far I’ve had to invest much less development time on the Emacs extension, though in fairness it has benefited from having an original to crib from. Also, it is currently less featureful than the Vim plugin.

ii. Backporting

I had targeted GNU Emacs 23, since that’s the version I use. The port to GNU Emacs 22 was trivial. A port to GNU Emacs 21 is difficult for the lack of several convenience functions, so it’s on the back burner.

I spent about an hour on a tricky port to XEmacs before deciding that for two reasons, I couldn’t justify the effort:

  1. Ugly portability wrappers would complicate the otherwise compact code. A concise assert becomes an awkward block:[2]
    ;; Before
    (assert (minibufferp))
     
    ;; After
    (assert (if (boundp 'xemacsp)
                (eq (window-buffer (minibuffer-window))
                    (current-buffer))
              (minibufferp)))
  2. This package was written for my use, and I haven’t yet had reason to use XEmacs.

Perhaps if I had targeted GNU Emacs 21 as my baseline, portability wouldn’t have been as much of an issue; the library couldn’t have been as easy to write, however, and I believe side-projects especially should optimize for development time.

[2] A colleague — and also a more capable Emacs user — suggests to isolate portability wrappers as a partial solution. This is good practice in general. So, in perhaps a separate file:

(when (not (boundp 'minibufferp))
  (defun minibufferp ()
    (eq (window-buffer (minibuffer-window))
        (current-buffer))))
(provide 'lusty-xemacs)

Then, within the main code:

(when (boundp 'xemacsp)
  (require 'lusty-xemacs))
 
;; ...
 
;; Clean again!
(assert (minibufferp))

This is what I will do when I re-examine the XEmacs port.

4. The Common Lisp compatibility package

The comprehensive cl package provides many great extensions to Emacs Lisp which make the language more approachable, especially to Common Lisp programmers. Some of its nifty additions:

  • defun*, defmacro* (adding implicit blocks and keyword arguments)
  • flet, labels, macrolet
  • loop, do
  • lexical-let
  • multiple-value-bind, destructuring-bind
  • case, ecase, typecase
  • setf, incf/decf, define-modify-macro
  • push, pushnew, pop

Pragmatically, XEmacs loads this package by default. GNU Emacs, however, lightly discourages its use. From the reference manual:

… we have a policy that packages installed in Emacs must not load cl at run time. … If you are writing packages that you plan to distribute and invite widespread use for, you might want to observe the same rule.

And also:

Please don’t require the cl package of Common Lisp extensions at runtime. Use of this package is optional, and it is not part of the standard Emacs namespace.

An imperfect workaround using eval-when-compile is advocated instead. I suggest to disregard the undertone of this note; Emacs Lisp benefits considerably from the inclusion of cl, and it is time to move forward. In his article about Ejacs, Emacs expert Steve Yegge likewise recommends unrestrained use of cl.

5. Conclusion

Under the all-important light of getting things done, Emacs Lisp is not a bad language. Emacs users are fortunate, and they should be thankful — users of other editors are not so lucky.

Thanks to Ron Gut and Chris Burke for their input on a draft of this article.