lisp on The Neo-Babbage Files

Lambda Calculus and Lisp, part 2 (recursion excursion)

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Thu, 20 Feb 2025 01:26:00 -0600

From the previous entry in this series, one of the things of note in discussing the nature of the connections between LISP and (the) lambda calculus was John McCarthy’s concern about recursion and higher-order functions.

A couple of excerpts from previous quotes from McCarthy on the subject to set the stage:

…And so, the way in which to [be able to handle function passing/higher order functions] was to borrow from Church’s Lambda Calculus, to borrow the lambda definition. Now, having borrowed this notation, one the myths concerning LISP that people think up or invent for themselves becomes apparent, and that is that LISP is somehow a realization of the lambda calculus, or that was the intention. The truth is that I didn’t understand the lambda calculus, really. In particular, I didn’t understand that you really could do conditional expressions in recursion in some sense in the pure lambda calculus.…

[McCarthy 1978a:190¹]

…Writing eval required inventing a notation representing LISP functions as LISP data, and such a notation was devised for the purposes of the paper with no thought that it would be used to express LISP programs in practice. Logical completeness required that the notation used to express functions used as functional arguments be extended to provide for recursive functions, and the LABEL notation was invented by Nathaniel Rochester for that purpose. D.M.R. Park pointed out that LABEL was logically unnecessary since the result could be achieved using only LAMBDA — by a construction analogous to Church’s Y-operator, albeit in a more complicated way.…

[McCarthy 1978a:179¹]

Examining Church’s Y Combinator will be something we return to (probably in a number of posts), but I’ll defer discussion of it for the moment.

For now, let’s consider recursion in lisps. We’ll be talking a lot of recursion in Emacs Lisp today in fact.

Self-reference, self-embedding

Recursion is a concept or process depends on a simpler or previous version of itself. It’s ubiquitous, including in the natural world: Wikipedia notes that “Shapes that seem to have been created by recursive processes sometimes appear in plants and animals, such as in branching structures in which one large part branches out into two or more similar smaller parts. One example is Romanesco broccoli.”

Figure 1: Romanesco broccoli (Brassica oleracea), from Wikipedia

Recursion is a thing I deal with a lot in my day job, as it is a feature of natural language, especially syntax and semantics. To provide a quick illustration — though this is not at all how modern generative syntax is done anymore — consider phrase structure grammar and phrase structure rules used by Noam Chomsky and his colleagues in the 1950s.

Sentences (and linguistics objects generally) have formal structure, and it is part of the productive/creative nature of language that we might envision this structure as involving abstract structure rules that can be expanded in different ways (and then have vocabulary filled in).

A phrase structure rule will generally have the form A → [B C], indicating that A may be rewritten or expanded into [B C]. (In the following, I’ll use round brackets ()'s to denote optional constituents.)

So, a subset of these rules for English might include something like (note that some things have multiple rules that can apply to them):

1. S → NP VP
2. NP → (Det) N
3. N → (AdjP)* N
4. VP → V
5. VP → V NP (NP)
6. VP → V CP
7. CP → (Comp) S
...

Code Snippet 1: a snippet of phrase structure grammar rules for English [Nb.: again, not prolog, but maybe the best fontlocking choice here]

(Where S is “sentence”; Det is a determiner (like “the”, “a”); NP is a noun phrase; Nₙ is a noun head; AdjP is an adjective phrase; VP is a verb phrase; V is a verb head; CP is a complementiser phrase; Comp is a complementiser (like “that”).)

So we can rewrite S as NP VP (1) and then rewrite NP VP as Det N VP (2, choosing an optional Det) and then Det N VP as Det N V (4) and then insert lexical items of the appropriate category into the ‘heads’ (the non-P elements). So we might choose “the” for Det and “cat” for N and “purrs” for V, and get the sentence “the cat purrs”.

But note that some of the rules allow for expansion into elements that contain expansions back into themselves. So rule (1) allows an S to exapnd into NP VP and rule (6) allows for a VP to expand into V CP and rule (7) allows for a CP to expand into an S. At which point we can apply rule (1) again to expand the new S into NP VP, and then repeat this process as many times as we like:

S
NP VP
N VP
N V CP
N V Comp S
N V Comp NP VP
N V Comp N VP
N V Comp N V CP
N V Comp N V Comp S
N V Comp N V Comp NP VP
N V Comp N V Comp N VP
N V Comp N V Comp N V CP
N V Comp N V Comp N V Comp S
N V Comp N V Comp N V Comp NP VP
N V Comp N V Comp N V Comp N VP
N V Comp N V Comp N V Comp N V CP
N V Comp N V Comp N V Comp N V Comp S
N V Comp N V Comp N V Comp N V Comp NP VP
...

Code Snippet 2: a recursive English sentence expansion [Nb.: again, not prolog, but maybe the best fontlocking choice here]

And it’s easy to imagine examples of what such a sentence could be like, e.g., “John said that Sita said that Bill said that Mary said that Ram said that Kim said that…". It won’t be infinitely long, but there’s no particular theoretical bound on how long it could be (memory processing and finite human lifespans will impose practical limits, of course).

On the formal semantics side, things are similar: consider that logical languages too (which are often used to formalise natural language semantics) allow for recursion. Propositional logic construction with Boolean operators too have no theoretical upper limits: we can write (t ↔ (p ∧ (q ∨ (r → (s ∧ ¬¬¬¬¬¬t))))),² and there’s nothing which prevents composing this bit of formalism with yet another bit, and so on.

And in mathematics and computer science, recursion is often a thing which suggests itself.

For instance, though there are other ways of calculating it, the Fibonacci sequence³, i.e., a sequence of numbers in which each element is the sum of the two elements that precede it (e.g. 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144…).

A natural way of writing an equation to calculate these is something like:

Fib(0) = 0 as base case 1.

Fib(1) = 1 as base case 2.

For all integers n > 1, Fib(n) = Fib(n − 1) + Fib(n − 2).

Rule 3 makes reference to itself. I.e., in order (by this method) to calculate Fib(6), you have to calculate Fib(5), for which you have to calculate Fib(4)), for which you have to calculate Fib(3), for which you have to calculate Fib(2), which you can then base on rules (1) & (2): you can add 0 and 1 (= Fib(0) and Fib(1)) together to get Fib(2), and then you can calculate Fib(3) by adding Fib(1) and Fib(2) and so on.

Cursed recursion

In early Lisp(s), despite the concern around recursion, writing recursive functions was/is not always pragmatically viable, because it can lead to stack overflows (potential infinities are hard in practice).

Scheme, a Lisp dialect originally created at MIT by Guy L. Steele, Jr. and Gerald Jay Sussman, was the first Lisp to implement tail call optimisation, which is a way of making significantly recursive functions viable, making tail calls similar in memory requirement to their equivalent loops.

Before talking (a little bit more) about tail recursion, let’s look at concrete examples of a tail recursive function and its loop equivalent. We’ve mentioned Fibonacci numbers already, so let’s try to write functions calculate these (in Emacs Lisp).

Following the mathematical abstraction for the Fibonacci sequence above, we could write a function like this:

;; -*- lexical-binding: t; -*-
(defun fib1 (n a b)
  "Calculate the first `n' Fibonacci numbers, recursively."
  (if (< n 1)
      a
    (cons a
          (fib1 (1- n) b (+ a b)))))

Code Snippet 3: a first go at a recursive elisp function for fibonacci numbers

I’ve tried to make this as simple as possible, we could make it nicer (say, with a wrapper function or some some of flet) so that we didn’t have to pass in the initial values. But, keeping it simple (for now): fib1 is a function taking three arguments: n, the quantity of Fibonacci numbers to return; a, the first number to start with; and b, the second number to start with.

Following the schema above, we’re going to pass 0 for a and 1 for b. Let’s get the first ten numbers of the sequence, and pass 10 for n:

(fib1 10 0 1)  ; (0 1 1 2 3 5 8 13 21 34 . 55)

I know, the output is a bit ugly because we’re just cons’ing the results and so it’s not a proper list, but we’re keeping things simple.

Let’s walk through how it works. The function first looks at n, if n is less than 1, it returns a (whatever it is). If n isn’t less than 1, we return the cons of a with the result of calling fib1 itself on “n minus 1” b and “a plus b”.

So if we start with n=3, a=0, b=1, that is, evaluate (fib1 3 0 1), the function would say, well, 3 isn’t less than 1, so I’m going to create a cons with 0 and the result of calling (fib1 2 1 1) (2 = “n minus 1”, because n is currently 3; 1 because b=1; and 1 because a + b = 0 + 1 = 1).

So at this point we have a cons that looks like this:

(0 . (fib1 2 1 1))  ;; [= (cons 0 (fib1 2 1 1))]

When we evaluate (fib1 2 1 1), n is still not less than 1, so we’re going to cons the current value of a (which is 1) with another call to fib1: (fib1 1 1 2) (1 = “n minus 1”, because n is currently 2; 1 because b=1; and 2 because a + b = 1 + 1 = 2).

So now we have:

(0 1 . (fib1 1 1 2))  ;; [= (cons 0 (cons 1 (fib1 1 1 2)))]

Now we have to evaluate (fib1 1 1 2). n is still not less than 1; so we create another cons of 1 (as a is currently 1) with yet another call of fib1: (fib1 0 2 3) (0 = “n minus 1”, because n is currently 1; 2 because b=2; and 3 because a + b = 1 + 2 = 3). And so now:

(0 1 1 . (fib1 0 2 3))  ;; [= (cons 0 (cons 1 (cons 1 (fib1 0 2 3))))]

And, finally, evaluating (fib1 0 2 3), now n is less than one, so we take the first branch of the conditional and just return a, which is 2. So the result of starting with (fib1 3 0 1) is:

(0 1 1 . 2)  ;; [= (cons 0 (cons 1 (cons 1 2)))]

And you can try this with other values of n, e.g., try evaluating (fib1 100 0 1) to get the first 100 members of the sequence.⁴

But, at least for me on Emacs 30.0.93, 529 is the limit. If we try (fib1 520 0 1), the debugger pops up with a 1622 line long error, which begins:

Debugger entered--Lisp error: (excessive-lisp-nesting 1622)
  cl-print--cons-tail(excessive-lisp-nesting (1601) #<buffer  *temp*>)
  #f(compiled-function (object stream) #<bytecode 0x491d55561699a09>)(#0 #<buffer  *temp*>)
  apply(#f(compiled-function (object stream) #<bytecode 0x491d55561699a09>) (#0 #<buffer  *temp*>))
  #f(compiled-function (&rest args) #<bytecode 0x1c31d892b8046a8b>)()
  #f(compiled-function (cl--cnm object stream) #<bytecode 0x7c361f66f109692>)(#f(compiled-function (&rest args) #<bytecode 0x1c31d892b8046a8b>) #0 #<buffer  *temp*>)
  apply(#f(compiled-function (cl--cnm object stream) #<bytecode 0x7c361f66f109692>) #f(compiled-function (&rest args) #<bytecode 0x1c31d892b8046a8b>) (#0 #<buffer  *temp*>))
  #f(compiled-function (object stream) #<bytecode 0x1f277a9a6dc403fa>)(#0 #<buffer  *temp*>)
  apply(#f(compiled-function (object stream) #<bytecode 0x1f277a9a6dc403fa>) #0 #<buffer  *temp*>)
  cl-print-object(#0 #<buffer  *temp*>)
  cl-prin1(#0 #<buffer  *temp*>)
  backtrace--print(#0 #<buffer  *temp*>)
  cl-print-to-string-with-limit(backtrace--print #0 5000)
  backtrace--print-to-string(#0 nil)
  backtrace-print-to-string(#0)
  debugger--insert-header((error #0 :backtrace-base eval-expression--debug))
  #f(compiled-function () #<bytecode 0x299e53d67d1ac62>)()
  backtrace-print()
  debugger-setup-buffer((error #0 :backtrace-base eval-expression--debug))
  debug(error #0 :backtrace-base eval-expression--debug)
  eval-expression--debug(#0)
  (if (< n 1) a (cons a (fib1 (1- n) b (+ a b))))
  fib1(0 259396630450514843945535792456880074043523940756078363514486570322782139633750401577338505233670220572153381665 419712564636128966418863068957011388899128076671547993021605479585858227224221424221791102364954108601240491394)
  (cons a (fib1 (1- n) b (+ a b)))
  (if (< n 1) a (cons a (fib1 (1- n) b (+ a b))))
  fib1(1 160315934185614122473327276500131314855604135915469629507118909263076087590471022644452597131283888029087109729 259396630450514843945535792456880074043523940756078363514486570322782139633750401577338505233670220572153381665)
  (cons a (fib1 (1- n) b (+ a b)))
  (if (< n 1) a (cons a (fib1 (1- n) b (+ a b))))
  fib1(2 99080696264900721472208515956748759187919804840608734007367661059706052043279378932885908102386332543066271936 160315934185614122473327276500131314855604135915469629507118909263076087590471022644452597131283888029087109729)
....

Code Snippet 4: beginning of excessive-lisp-nesting error for our fib1 function

Because we’ve built up a long, deeply-embedded list of conses and Emacs has a limit of how deep it’s willing/able to go (you can see above that we’ve almost made it to the end, just needing to calculate fib1(0), when Emacs decides it’s had enough).

In Scheme and elsewhere, self-recursive calls at the ends (“tails”) of functions can be optimised to avoid these sorts of stack overflows (excessive-lisp-nesting).⁵ Tail-call optimisation lets procedure calls in tail positions be treated a specialised GOTO statements, which can be efficiently processed:

…only in cases where structures are explicitly declared to be dynamically referenced should the compiler be forced to leave them on the stack in an otherwise tail-recursive situation. In general, procedure calls may be usefully thought of as GOTO statements which also pass parameters, and can be uniformly encoded as JUMP instructions. This is a simple, universal technique, to be contrasted with […] more powerful recursion-removal techniques…

[Steele 1977:155⁶]

But our fib1 function doesn’t do this, and we end up flooded the stack with too many conses.

Back to loops

Recursive functions are perhaps most idiomatic in Scheme (among lisps, I mean). Some implementations of Common Lisp can do tail-call optimisation, but loops are perhaps more common, and certainly in Emacs Lisp (for reasons you can see above), loops are usually what are used. And so we can write a new Fibonacci function with a loop. It won’t be nearly as pretty, but it’ll work better.

Here’s one possible implementation:

;; -*- lexical-binding: t; -*-
(defun fib2 (n a b)
  "Calculate the first `n' Fibonacci numbers,
in a loop."
  (let ((result nil))
    (dotimes (c n)
      (setq result (cons a result))
      (setq tempa b)
      (setq b (+ a b))
      (setq a tempa))
    (nreverse result)))

(fib2 10 0 1) ; (0 1 1 2 3 5 8 13 21 34)

(setq bigfib-50000 (fib2 50000 0 1)) ; this will work - we can get 50,000 numbers

Code Snippet 5: a second go at a fibonacci function, with looping

The result is prettier at least: a proper rather than an improper list (because we started by cons'ing onto an empty list). Our fib2 function itself isn’t as mathematically pleasing as our fib1 function, we end up with a lot of setq's (the nreverse at the end reverses our list, because the way we build up our list is by cons'ing the first results first, so they end up at the end until we flip them with nreverse). But it works well. If you try to (fib2 100000 0 1), it’ll fail, but not because of stack overflow, just because we end up with numbers that are too big for Emacs. But you can certain get the over 50,000 members of the Fibonacci sequence, which is much better than fib1's limit of 529.

And dotimes is just one loop procedure available. (See cl-loop for a more powerful one.)

Optimal Tail with Emacs

Ok, so, practically, we should probably generally prefer loops over tail-recursive functions in Emacs. But, what if we just like the latter more?⁷ Are there any other possibilities?

Wilfred Hughes has an emacs package tco.el which implements a special macro for writing tail-recursive functions.⁸ It works by replacing each self-call with a thunk, and wrapping the function body in a loop that repeatedly evaluates the thunk. Thus a function foo defined with the defun-tco macro:

(defun-tco foo (...)
  (...)
  (foo (...)))

would be re-written as:

(defun foo (...)
   (flet (foo-thunk (...)
               (...)
               (lambda () (foo-thunk (...))))
     (let ((result (apply foo-thunk (...))))
       (while (functionp result)
         (setq result (funcall result)))
       result)))

And this delays evaluation in such a way as to avoid stack overflows. Unfortunately, at least currently for me (Emacs 30.0.93 again), tco.el seems to have some issues.

In Emacs 28.1, cl-labels (one of the ways of sort of doing let's for functions) gained some limited tail-call optimisation (as did named-let, which uses cl-labels).

;; -*- lexical-binding: t; -*-
(defun fib3 (n)
  "Calculate the first `n' Fibonacci numbers,
recursively, with limited tail-call optimisation
through `cl-labels'?!"
  (cl-labels ((fib* (n a b)
                (if (< n 1)
                    a
                  (cons a
                        (fib* (1- n)
                              b
                              (+ a b))))))
    (fib* n 0 1)))

(setq bigfib3 (fib3 397)) ; 396 highest that works

Code Snippet 6: a third go at a fibonacci function, with cl-labels

At least the way I’ve written it, it seems to suffer an overflow even sooner (at 397 rather than 529 as for our fib1), because it has to come back and do the cons after the tail-call — so fib* isn’t actually in the tail. (We can fix this in at least one way, which we’ll do in fib5.)

We could try to write an accumulator as a hack, where we try to do ours conses one at a time and pass along the results, but this fares no better than our fib1:

;; -*- lexical-binding: t; -*-
(defun fib4 (n a b accum)
  "Calculate the first `n' Fibonacci numbers, recursively,
but collect conses as we go and keep track of the length of
the `accum' cp. against `n'."
  (let* ((accum (cons a accum))
         (accum-lng (length accum)))
    (if (< n accum-lng)
        (nreverse accum)
      (fib4 n b (+ b a) accum))))

(setq bigfib4-529 (fib4 529 0 1 nil)) ; last good
(setq bigfib4-530 (fib4 530 0 1 nil)) ; overflows

Code Snippet 7: a fourth go at a fibonacci function, with an accumulator

If we combine cl-labels and the accumulator trick, however, we do seem to be able to escape stack overflows, because now we’ve got fib* properly in the tail:

;; -*- lexical-binding: t; -*-
(defun fib5 (n)
  "Calculate the first `n' Fibonacci numbers, recursively,
using both cl-labels and the accumulator trick."
  (cl-labels ((fib* (a b accum)
                   (let* ((accum (cons a accum))
                         (accum-lng (length accum)))
                     (if (< n accum-lng)
                         (nreverse accum)
                         (fib* b (+ b a) accum)))))
            (fib* 0 1 nil)))

(setq bigfib5-10000 (fib5 10000)) ; ok
(setq bigfib5-50000 (fib5 50000)) ; very slow, but ok

Code Snippet 8: a fifth go at a fibonacci function, with cl-labels and an accumulator

Now we’re back in the realms of what our fib2 non-recursive loop-style function could do. Although (setq bigfib5-50000 (fib5 50000)) calculates very slowly (worse than our looping fib2), so that’s not ideal.

Stream of Conses-ness

But here’s another possibility: Nicholas Pettton‘s stream package for emacs, where “streams” are delayed evaluations of cons cells.

;; -*- lexical-binding: t; -*-
(defun fib6 (n)
  "Return a list of the first `n' Fibonacci numbers,
implemented as stream of (delayed evaluation) conses."
  (cl-labels ((fibonacci-populate (a b)
                (stream-cons a (fibonacci-populate b (+ a b)))))
    (let ((fibonacci-stream
           (fibonacci-populate 0 1))
          (fibs nil))
      (dotimes (c n)
        (setq fibs (cons (stream-pop fibonacci-stream) fibs)))
      (nreverse fibs))))

(setq fib6-10k (fib6 10000)) ; ok
(setq fib6-50k (fib6 50000)) ; little slow, but works
(setq fib6-100k (fib6 100000)) ; little slow & overflow error

Code Snippet 9: a sixth go at a fibonacci function, with delayed evaluation conses

This works well. (fib6 50000) still turns out to run a bit slower than our (fib2 50000), so loops are still probably more efficient, but streams are pretty interesting. They can can used to represent infinite sequences. So here, above, fibonacci-stream (set by (fibonacci-populate 0 1)) is actually an infinite stream of Fibonaccis numbers, but lazily evaluated, so we just get the next one each time we call stream-pop on our fibonacci-stream local variable. (What happens is that stream-pop takes the car of fibonacci-stream, evaluates and returns it, and then sets fibonacci-stream to be its cdr (i.e., popping off and “discarding” the first element; which which captured in our fibs collector.))

Cascades of Fibonacci numbers

Oh, incidentally and irrelevantly, if you inspect the contents of your fib6-50k, it’s very aesthetically pleasing, a cascade of numbers:

Figure 2: fibonacci numbers burst forth from their seeds and spill out into the buffer

`excessive-lisp-nesting`: (overflow-error)

I had hoped to get to the Y Combinator today (and think I might have suggested a promise of that), for that’s where things really get interesting. And we need to get back to lambda calculus, of course. But we may be near the limits of excessive lisp nesting ourselves here.

However, the recursion discussion here has set the stage for the Y Combinator, which we’ve already talked a couple of times, especially in connection to John McCarthy’s claims about “not really understanding” lambda calculus and the fact that these really centre on his not seeing how one could get recursion without direct Self-reference (and thus the need for LABEL) because of not knowing about the Y Combinator.

Figure 3: a close-up of a section of Michał “phoe” Herda’s hand-illuminated Y Combinator Codex showing part of a Fibonacci defun

And, the Y Combinator ties in with all sorts of other curious things. Paradoxes, types, calligraphy.

[Thus, I ended up with an excursus on this excursus as the next post. I’ll put a link here to the proper third part when it’s up.]

McCarthy, John. 1978a. History of Lisp. In History of programming languages, ed. Richard L. Wexelblat, 173–185. New York: Association for Computing Machinery. https://dl.acm.org/doi/10.1145/800025.1198360 ↩︎
“t if and only if p and q or if r then s and not not not not not not t” ↩︎
A number of Indian philosophers, at least as far back as Virahāṅka (ca. AD 600–800), gave formulations for what is usually called the Fibonacci sequence. See Singh, P. (1985). The so-called fibonacci numbers in ancient and medieval India. Historia Mathematica, 12(3), 229–244. https://doi.org/10.1016/0315-0860(85)90021-7 [pdf] ↩︎
You might actually want to do something like (setq my-fib1-100 (fib1 100 0 1)) to put the result into a variable, because the echo area at the bottom of Emacs isn’t big enough for all of the numbers. (Oh, to eval things in an Emacs buffer, put your cursor/point at the end of the expression and press C-x C-e. But don’t do that for this one. If you want Emacs to just stick the results in directly into the buffer rather than echoing them, press C-u C-x C-e. But don’t do that here either, because Emacs will still end up printing an ellipsis because it thinks it’s too long.) And then press C-h v and type my-fib1-100 and enter to see the result. ↩︎
See further here, for instance, for more about tail calls and tail call optimisation. ↩︎
Steele, Guy L., Jr. 1977. Debunking the “expensive procedure call” myth or, procedure call implementations considered harmful or, LAMBDA: The Ultimate GOTO. ACM ‘77: Proceedings of the 1977 annual conference, 153–162. [https://dl.acm.org/doi/10.1145/800179.810196] ↩︎
If you’ve read any of Paul Graham’s Common Lisp books (e.g., On Lisp) or any of the Little Schemer books (the latter with Duane Bibby‘s lovely artwork in them), you may be disposed towards using recursion rather than loops.

Figure 4: close-up of Duane Bibby’s cover illustration for “The Little Schemer”

↩︎
One of my emacs packages, Equake, actually used to use tco.el (until commit #0ab0801 [2020-08-24 Mon]). ↩︎

Lambda Calculus and Lisp, part 1

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Tue, 18 Feb 2025 03:05:00 -0600

The first of a series of envisioned blog posts on lambda calculus, and Lisp. It’s unclear exactly where to start: there is a whole heap of interesting issues, both theoretical and in terms of concrete implementations, which tangle and interconnect.

A particular application of lambda calculus is a very salient part of my “day job” as a formal semanticist of natural language. And my interests in Emacs and lisp(s) feel like they tie in here as well—though that’s a question in itself which is probably as good of a starting point into this (planned) series of posts as any.

There is much to explore: origins of John McCarthy‘s Lisp and Alonzo Church‘s lambda calculus; encodings of the simple made complex by restriction to a limited set of tools; recursion, fixed points, and paradoxes; infinities, philosophy, and engineering. But much of this requires stage setting.

And finding an exact entry point is yet tricky. But perhaps we start with λ: the divining rod, the wizard’s crooked staff, as it is the key component of much magic of a sort.

Lisp: LAMBDA the Ultimate?

We’ll start on the programming side, before turning to more philosophical or mathematical abstractions, with lambda.

It is now not only Lisps which contain lambda as a keyword, many/most programming languages have lambda as a keyword, usually for the introduction of an anonymous function. That is, an unnamed function, sometimes for one-off use.

But in McCarthy’s original formulation of LISP in 1958, LAMBDA was used as the basis for the implementation of functions generally:

Let f be an expression that stands for a function of two integer variables. It should make sense to write f (3, 4) and the value of this expression should be determined. The expression y^2 + x does not meet this requirement; y^2 + x(3, 4) is not a conventional notation, and if we attempted to define it we would be uncertain whether its value would turn out to be 13 or 19. Church calls an expression like y2 + x, a form. A form can be converted into a function if we can determine the correspondence between the variables occurring in the form and the ordered list of arguments of the desired function. This is accomplished by Church’s λ-notation. [p.6]

….

{λ[[x_1;…; x_n]; 𝓔]}∗ is (LAMBDA, (x∗_1,…, x∗_n), 𝓔∗). [p.16]

[McCarthy 1960:6, 16¹]

As in lambda calculus, LAMBDA binds variables, and replaces any occurrences of them in the scope of the operator with whatever it receives as arguments.

So the expression:

(LAMBDA (x y z) (+ (* y x) (* z x)))

if given the arguments 5, 2, 3, would replace x's with 5; y's with 2, and z's with 3:

(+ (* 2 5) (* 3 5))  ;; = (+ 10 15) = 25

The illusion of a blue-suffused platonic universe of `car`'s

The origin of lambda keywords in LISP, and the origin of LISP’s LAMBDA in lambda calculus has suggested the idea that LISP was something like an implementation of lambda calculus as a programming language, and the certain mysticism² attaching to both suggests perhaps a tighter surface association than there is direct evidence for.

Figure 1: xkcd 224 [see https://www.explainxkcd.com/wiki/index.php/224:_Lisp]

This is the topic of a 2019 blog post based on his talk for the Heart of Clojure conference in Daniel Szmulewicz expands on the theme “Lisp is not a realization of the Lambda Calculus”.³ One of the points Szmulewicz draws attention to is McCarthy’s own words:

…one of the myths concerning LISP that people think up or invent for themselves becomes apparent, and that is that LISP is somehow a realization of the lambda calculus, or that was the intention. The truth is that I didn’t understand the lambda calculus, really.

[McCarthy 1978b:190⁴]

In the paper version of the talk, McCarthy makes a similar point, but it’s worth looking at it in the larger context:

…how do you talk about the sum of the derivatives, and in programming it, there were clearly two kinds of programs that could be written.

One is where you have a sum of a fixed number of terms, like just two, where you regard a sum as a binary operation. And then you could write down the formula easy enough. But the other was where you have a sum of an indefinite number of terms, and you’d really like to make that work too. To make that work, what you want to be able to talk about is doing the same operation on all the elements of a lit. You want to be able to get a new list whose elements are obtained from the old list by just taking each element and doing a certain operation to it.

In order to describe that, one has to have a notation for functions. So one could write this function called mapcar. This says, “Apply the function f to all the elements of the list.” If the list is null then you’re going to get a NIL here. Otherwise you are going to apply the function to the first element of the list and put that onto a front of a list which is obtained by doing the same operation again to the rest of the list. So that’s mapcar. It wasn’t called mapcar then. It was called maplist, but maplist is something different, which I will describe in just a moment.

That was fine for that recursive definition of applying a function to everything on the list. No new ideas were required. But then, how do you write these functions?

And so, the way in which to do that was to borrow from Church’s Lambda Calculus, to borrow the lambda definition. Now, having borrowed this notation, one the myths concerning LISP that people think up or invent for themselves becomes apparent, and that is that LISP is somehow a realization of the lambda calculus, or that was the intention. The truth is that I didn’t understand the lambda calculus, really. In particular, I didn’t understand that you really could do conditional expressions in recursion in some sense in the pure lambda calculus. So, it wasn’t an attempt to make the lambda calculus practical, although if someone had started out with that intention, he might have ended up with something like LISP.

[McCarthy 1978a:189–190⁵]

The Discovery of LISP

Two bits from the end of this I want to highlight. The first, well, it’ll come up in future posts, and probably later in this post itself, and it has to do with recursion:

“I didn’t understand that you really could do conditional expressions in recursion in some sense in the pure lambda calculus”

And the second is that McCarthy hedges his “LISP as a realisation of the lambda calculus is myth” stance slightly:

“So, it wasn’t an attempt to make the lambda calculus practical, although if someone had started out with that intention, he might have ended up with something like LISP."

This does fit rather well with (and perhaps suggested) the framing that Paul Graham does of McCarthy as the “discoverer” of Lisp — like Euclid of geometry — rather than its inventor in his “The Roots of Lisp” paper:

In 1960, John McCarthy published a remarkable paper in which he did for programming something like what Euclid did for geometry. He showed how, given a handful of simple operators and a notation for functions, you can build a whole programming language. He called this language Lisp, for “List Processing,” because one of his key ideas was to use a simple data structure called a list for both code and data.

It’s worth understanding what McCarthy discovered, not just as a landmark in the history of computers, but as a model for what programming is tending to become in our own time. …. In this article I’m going to try to explain in the simplest possible terms what McCarthy discovered. The point is not just to learn about an interesting theoretical result someone figured out forty years ago, but to show where languages are heading. The unusual thing about Lisp—in fact, the defining quality of Lisp—is that it can be written in itself.

[Graham 2002:1⁶ (emphasis mine)]

Graham’s paper itself, as well as some of Graham’s other publications/postings (e.g., talking about Lisp as a sort of “secret weapon” in comparison to “blub languages”)⁷ is perhaps another contributor to the mystique of Lisp. (With the flip side of “Lisp as the Cleveriest Hacker’s Secret Weapon” enchanted coin being “the Curse of Lisp”⁸.)

There are other components of the history of LISP, which are suggestive of discovery rather than invention. McCarthy’s initial aim for LISP was more akin that of the Turing Machine: as a formal abstraction describing a mathematical model whose components were simple and few but yet was capable of performing any (and all) arbitrary computational operation:

One mathematical consideration that influenced LISP was to express programs as applicative expressions built up from variables and constants using functions. I considered it important to make these expressions obey the usual mathematical laws allowing replacement of expressions by expressions giving the same value. The motive was to allow proofs of properties of programs using ordinary mathematical methods. This is only possible to the extent that side effects can be avoided. Unfortunately, side effects are often a great convenience when computational efficiency is important, and “functions” with side effects are present in LISP. However, the so-called pure LISP is free of side effects, and Cartwright (1976) and Cartwright and McCarthy (1978) show how to represent pure LISP programs by sentences and schemata in first-order logic and prove their properties. This is an additional vindication of the striving for mathematical neatness, because it is now easier to prove that pure LISP programs meet their specifications than it is for any other programming language in extensive use. (Fans of other programming languages are challenged to write a program to concatenate lists and prove that the operation is associative.)

Another way to show that LISP was neater than Turing machines was to write a universal LISP function and show that is briefer and more comprehensible than the description of a universal Turing machine. This was the LISP function eval[e,a], which computers the value of a LISP expression e, the second argument a being a list of assignments of values to variables. (a is needed to make the recursion work.) Writing eval required inventing a notation representing LISP functions as LISP data, and such a notation was devised for the purposes of the paper with no thought that it would be used to express LISP programs in practice. Logical completeness required that the notation used to express functions used as functional arguments be extended to provide for recursive functions, and the LABEL notation was invented by Nathaniel Rochester for that purpose. D.M.R. Park pointed out that LABEL was logically unnecessary since the result could be achieved using only LAMBDA — by a construction analogous to Church’s Y-operator, albeit in a more complicated way.

S.R. Russell noticed that eval could serve as an interpreter for LISP, promptly hand coded it, and we now had a programming language with an interpreter.

[McCarthy 1978:179⁵]

(There’s a lot going on in this passage, and its context.

Notions of reasons to avoid side-effects (an ideal of “functional programming”, emphasised in Haskell and Clojure (another lisp)): thus the “functional” mode of lambda calculus rather than the “everything is state” mode of Turing machines and their infinitely long memory tapes.

Recursion again (which we’ll get to, repeatedly).

And Alonzo Church, the originator (discoverer?)^{see “X” below} of lambda calculus, who we’ll talk more about soon, and who was also Alan Turing’s PhD supervisor at Princeton.

But, first: let’s turn back to the topic of the concrete instantiation of Lisp on physical hardware by Stephen Russell.)

Figure 2: cover of Byte Magazine August 1979 [full mag here]

Elsewhere, McCarthy makes clear that he hadn’t thought at that point of LISP being instantiatable but as a theoretical exploration of computing at an abstract level. But, instead, the theoretical description translated fairly easily and directly to a runnable program, a LISP interpreter:

This EVAL was written and published in the paper and Steve Russell said, “look, why don’t I program this EVAL” … and I said to him, “ho, ho, you’re confusing theory with practice, this EVAL is intended for reading, not for computing”. But he went ahead and did it. That is, he compiled the EVAL in my paper into IBM 704 machine code, fixing bugs, and then advertised this as a Lisp interpreter, which it certainly was. So at that point Lisp had essentially the form that it has today…

[McCarthy 1974a:307⁹]

“No compute. Only read.” “Hell, EVAL that.”

This is the EVAL code “in the paper” referred to above:

eval[e; a] = [
   atom [e] → assoc [e; a];

   atom [car [e]] → [
        eq [car [e]; QUOTE] → cadr [e];
        eq [car [e]; ATOM] → atom [eval [cadr [e]; a]];
        eq [car [e]; EQ] → [eval [cadr [e]; a] = eval [caddr [e]; a]];
        eq [car [e]; COND] → evcon [cdr [e]; a];
        eq [car [e]; CAR] → car [eval [cadr [e]; a]];
        eq [car [e]; CDR] → cdr [eval [cadr [e]; a]];
        eq [car [e]; CONS] → cons [eval [cadr [e]; a]; eval [caddr [e]; a]];
        T → eval [cons [assoc [car [e]; a]; evlis [cdr [e]; a]]; a]];

   eq [caar [e]; LABEL] → eval [cons [caddar [e]; cdr [e]];
                                cons [list [cadar [e]; car [e]; a]]];

   eq [caar [e]; LAMBDA] → eval [caddar [e];
                                 append [pair [cadar [e];
                                               evlis [cdr [e]; a]; a]]]

evcon[c; a] = [eval[caar[c]; a] → eval[cadar[c]; a]; T → evcon[cdr[c]; a]]

evlis[m; a] = [null[m] → NIL; T → cons[eval[car[m]; a]; evlis[cdr[m]; a]]]

Code Snippet 1: McCarthy 1960, p.17 (see fn. [1]) [Nb.: not actually `prolog' but highlights better as]

Translated into slightly more familiar Lisp style (with added named-function-making label's), it is:¹⁰

(label eval
       (lambda (e a)
         (cond
           ((atom e) (assoc e a))
           ((atom (car e))
            (cond
              ((eq (car e) 'quote) (cadr e))
              ((eq (car e) 'atom)  (atom  (eval (cadr e) a)))
              ((eq (car e) 'eq)    (eq    (eval (cadr e) a)
                                          (eval (caddr e) a)))
              ((eq (car e) 'car)   (car   (eval (cadr e) a)))
              ((eq (car e) 'cons)  (cons  (eval (cadr e) a)
                                          (eval (caddr e) a)))
              ((eq (car e) 'cond)  (evcon (cdr e) a))
              ('t                  (eval (cons (assoc (car e) a)
                                               (cdr e))
                                         a))))
           ((eq (caar e) 'label)   (eval (cons (caddar e) (cdr e))
                                         (cons (list (cadar e) (car e)) a)))
           ((eq (caar e) 'lambda)  (eval (caddar e)
                                         (append (pair (cadar e)
                                                       (evlis (cdr e) a))
                                                 a))))))

(label evcon
       (lambda (c a)
         (cond ((eval (caar c) a)
                (eval (cadar c) a))
               ('t
                (evcon (cdr c) a)))))

(label evlis
       (lambda (m a)
         (cond ((null m) '())
               ('t (cons (eval  (car m) a)
                         (evlis (cdr m) a))))))

Code Snippet 2: EVAL in more recognisable lisp form

The above, given the few additional definitions for convenience immediately following, is a full LISP interpreter.¹¹

(label null
       (lambda (x)
         (eq x '())))

(label and
       (lambda (x y)
         (cond (x (cond (y 't) ('t '())))
               ('t '()))))

(label not
       (lambda (x)
         (cond (x '())
               ('t 't))))

(label append
       (lambda (x y)
         (cond ((null x) y)
               ('t (cons (car x)
                         (append (cdr x) y))))))

(label pair
       (lambda (x y)
         (cond ((and (null x) (null y)) '())
               ((and (not (atom x)) (not (atom y)))
                (cons (list (car x) (car y))
                      (pair (cdr x) (cdr y)))))))

(label assoc
       (lambda (x y)
         (cond ((eq (caar y) x) (cadar y))
               ('t (assoc x (cdr y))))))

Code Snippet 3: additional convenience functions for EVAL

The brevity of the code combined with the details of the story of the implementation: seemingly the theoretical code works without translation, suggesting a sort of natural discovery. But, the nature of even the theoretical, pre-implementation code did not exist in some Platonic heaven of mathematics, as can be seen by the nature of some of the operations, particularly car and cdr (often in modern Lisps, especially Scheme and Scheme-influenced Lisps, rendered instead more transparently as first and rest.)

The Non-Platonic mechanics of 1950s IBM mainframes

LISP was designed with the IBM 704-style architecture in mind, and car and cdr make some reference to particular details of the hardware, where the IBM 704 had “address” and “decrement” fields in memory index registers (≈locations in physical RAM), and car referenced the “address” field of the register (so CAR = “C-ontents of the A-ddress part of the R-egister”) and cdr the “decrement” field of the register (CDR = “C-ontents of the D-ecrement part of the R-egister”).¹²

Figure 3: my other car is a cdr

Lists in Lisp are singly-linked lists, with each node in the list having a “value” field and a “next” field, which points to the next node; these “value” and “next” fields correspond to the car and cdr operations. So, if we have a list like (a b c), the first node in the list has a “value” field of a and a “next” field pointing at another node, with this second node actually itself being — not b — but rather the list (b c). A proper list in Lisp is nil-terminated: that is, the last item in the list is actually nil (which is the empty list '()).

The operation cons above (for CONstructor) is a pair-forming operation (where the pairs correspond to the “value” and “next” fields), returning what are variously called (in different lisps) “conses” or “pairs”. Not all conses/pairs are lists (at least in most Lisps), since a proper list in Lisp is nil-terminated.

The operation (cons 'a 'b) will return a cons'ed object with essentially a single node, where the “value” field will be a and the “next” field will be b. Lisps will usually print such non-list conses (sometimes called “improper lists”) as “dotted lists”, i.e., (cons 'a 'b) will print out as (a . b).

The list (a b c) is actually the result of doing (cons a (cons b (cons c nil))) and would be printed in dotted-pair notation as (a . (b . (c . nil))) [which is equivalent to (a . (b . (c . ()))), since nil is the empty list].

Unsurprisingly, a lot of early/traditional Lisp programming involves manipulations of lists. But all modern Lisps implement other types of data structures as well, including vectors/arrays, hash tables, objects, and so on.

But, on the main topic of (eq 'lisp 'lambdacalculus), others have also pointed out the concrete hardware-connections of LISP from early days, telling against the “Lisp-as-pure-formal-~~invention~~-discovery” or “Lisp as (semi-)direct implementation of lambda calculus” notions:

Lisp was intended to be implemented on a computer from day 0. For their IBM 704. Actually Lisp is based on earlier programming experience. From 56 onwards John McCarthy implemented Lisp ideas in Fortran & FLPL. Then 58 the implementation of Lisp was started. 59 a first runnable version was there. 1960 there was the Lisp 1 implementation. The widely known paper on the Lisp implementation and recursive function theory was published in 1960. But his original prime motivation was not to have a notation for recursive function theory, it was to have a list processing programming language for their IBM 704 for AI research.

Lisp as designed by McCarthy was very different from lambda calculus.

[going on to point to the Stoyan (1984) Early LISP history (1956 - 1959) paper.]

— lispm‘s comment on r/lisp thread on this topic

There’s obviously much more to explore for the early history of Lisp and the nature of its connections to lambda calculus, but this much at least should give a general sense of the distinctions/divergences between Lisp and lambda calculus, while not ignoring important interconnections between them.

Reaching the `'()` of the line

And so while it’s tempting to delve off into other interesting features (homoiconicity!) of LISP/Lisps and their history and dialects (the Common Lisp of Endor; Schemes, Rackets, Chickens and Guile (oh my!), Clojure, Fennel and others), I’d wanted to get to lambda calculus proper much earlier in this piece already, and so we’ll set our (lispy) parens down for a moment, and trade in our LAMBDA for a λ.

Cattle-prodding functions: the lambda calculus

The aforementioned Alonzo Church¹³, a Princeton mathematician who supervised 31 doctoral students during his career, influencing others important researchers (including Haskell Curry [for whom the programming language Haskell is named; as well as the operating of currying]), developed (the) lambda calculus as part of his research into the foundations of mathematics.

Lambda calculus is Turing complete, thus equivalent in computation power to a Turing machine and a universal model of computation.

There are very few bits of machinery in basic untyped lambda calculus. (A reason for which it seemed to be attractive to McCarthy as a touchstone.)

Lambda calculus has variables and lambdas; function application; a reduction operation (which may follow function application); and a convenience variable-renaming operation.

More specifically, we have:

variables, like x, which are characters or strings representing “a parameter”.
lambda abstraction: essentially just the definition of a function, specifying its input (by a bound variable, say, λx) and returning an output (say, M). E.g., the expression λx.M will take an input, and replace any and all instances of x in the body M¹⁴ with whatever the input was. (The . separates the lambda and specification of bound variable (here x) (“input taker”) from the body (the “output”).)
function application: a representation like (M N), the function M applies to N, where both M and N are some sort of lambda terms.
β-reduction (beta reduction): bound variables inside body of the expression are replaced by inputs “taken” by the lambda expressions. The basic form: ((λx.M) N) → (M[x := N]). (That is, an expression (λx.M) combining with an expression N returns (M) where where all instances of x inside of M are replaced with N.)

(Setting aside rule 4 for the moment.)

For example, we might have an expression:

λf.λx.(f x)

This would be an expression which combines, one at a time, with two inputs, the lambdas operating from left to right, and then applying the f input to the x input.

To see this in full working order, we need to specific one of the two remaining operations (both reduction operations):

So, turning to rule 4 for β-reduction, taking our expression from above and providing it with inputs, and walking through the (two) application+β-reduction steps one at a time:

((λf.λx.(f x)) b a) =

((λx.(b x)) a) =

(b a)

That is, first, the leftmost lambda, λf, “takes” the leftmost argument b (in “taking” an argument, it “discharges” and disappears) and the (single) bound instance of the variable f in the body is replaced by b. Then, the same thing happens with the remaining lambda, λx, and the remaining argument, a. The result is a function where b applies to a. (Though since in this case there are no more lambdas, nothing more happens.)

Since (b a) looks somewhat unexciting/opaque, we can imagine a sort of hybrid proper untyped lambda calculus/Lisp hybrid language — let’s call this toy language of ours ΛΙΣΠ — and illustrate what things might look like there (assuming here that numbers are numbers and #'* is a lispy prefix multiplication function):

((λf.λx.λy.(f x y)) #'* 6 7) =

((λx.λy.(* x y)) 6 7) =

((λy.(* 6 y)) 7) =

(* 6 7) =

42

This isn’t how mathematics works in classical untyped lambda calculus — because we only have the 4 rules/entities enumerated above [plus a variable-clash reduction operation called α-reduction]¹⁵ and nothing else¹⁶: no integers, no stipulated mathematical operations, no car or cdr or cons or eq or anything — we can do all of these things in lambda calculus with the tools we have, and we’ll explore that in another post, but for now I just wanted to show you the toy ΛΙΣΠ language snippet as I find something that feels a bit more familiar and concrete can be helpful for understanding the notional unpinnings of what’s going on in lambda calculus.

Ok, so there’s no integers or cars, but what’s all this about cattle prods?

Well, why is λ / LAMBDA the “function”-making operator? Maybe just eeny-meeny-miney-moe amongst Greek letters, but at least at one point Church explained that [A. Church, 7 July 1964. Unpublished letter to Harald Dickson, §2] that it came from the notation “x̂” used for class-abstraction by Whitehead and Russell in their Principia Mathematica (which we’re refer to later on), by first modifying “x̂” to “^⁣x” to (and then for better visibility to “∧x”) to distinguish function-abstraction from class-abstraction, and then changing “∧” (which is similar to uppercase Greek “Λ”) to (lowercase Greek) “λ” for ease of printing (and presumably to avoid confusion with other mathematical uses of “∧”, e.g., logical AND).¹⁷ [So maybe “x̂” ⇒ “^⁣x” → “∧x” → “λx”.]

The Greek lambda (uppercase Λ, lowercase λ) as an ortheme itself derives ultimately from a Semitic abjad, specifically from the Phoenician lāmd 𐤋, which (like most letters began as a pictogram of sorts) is considered to originate from something like an ox-goad, i.e., a cattle prod, or else a shepherd’s crook, i.e., a pastoral staff. (The reconstructed Proto-Semitic word *lamed- means a “goad”.)¹⁸

Are We in Platonic Heaven Yet?

Lambda calculus, not being tied to any particular hardware and being a true formula abstraction, feels like something that might have a better claim to being something like a property of the universe that one might discover (I admit I find it hard not to feel something of the sort — but then I’ve used lambda calculus for work on natural language within a framework that wants to assign some sort of reality to our formalisations, so it’s hard not to be pulled in this direction), however, Church says (of his own formalism):

We do not attach any character of uniqueness or absolute truth to any particular system of logic. The entities of formal logic are abstractions, invented because of their use in describing and systematizing facts of experience or observation, and their properties, determined in rough outline by this intended use, depend for their exact character on the arbitrary choice of the inventor.

[Alonzo Church 1932:348¹⁹]

(This seems part of Church’s constructivist philosophy, in common with Frege.)²⁰

What’s `next`? : `λy.(equal? (cdr L) y)`

We’ve pulled at a lot of disparate threads here, trying to explore the nature of the connections between Lisp and (the) lambda calculus. Neither the idea that Lisp is a direct instantiation of lambda calculus nor the idea that McCarthy was largely ignorant of properties of lambda calculus are quite right. But that there are an interesting interplay of connections.

But. This is really to set the stage for me to talk about things I’m interested in which draw on different aspects of Lisp(s) and lambda calculus and formal or applied applications to do with one or the other.

I was going to talk about Montague Grammar, because it’s fascinating (and it’s my day job), for which lambda calculus is a crucial component.²¹ But we’re at length now, so it should be another day.

What I do want to look at next is a combination of Lisp and lambda calculus, in various ways, starting with attempts to implement aspects of lambda calculus in Emacs Lisp, and the challenges therein.

[fingers crossed that (next 'blog) does not eval to undefined.]

(Update: (eval (next 'blog)): Part 2: Recursion Excursion.)

McCarthy, John. 1960. Recursive functions of symbolic expressions and their computation by machine. Communications of the Association for Computing Machinery 3(4):184-195. [available at the Wayback Archive; page nos. refer to those of the PDF at the link, a reformatted version in LaTeX, not those of the original publication.] ↩︎
On Lisp’s mystical aura, see How Lisp Became God’s Own Programming Language (and the associated Hacker News discussion). ↩︎
“Lisp ≠ Lambda Calculus” | Daniel Szmulewicz: Perfumed nightmare (A blog for the somnambulisp) ↩︎
McCarthy, John. 1978b. Transcript of presentation. History of Lisp. In History of programming languages, ed. Richard L. Wexelblat, 185–191. New York: Association for Computing Machinery. https://dl.acm.org/doi/10.1145/800025.1198361 ↩︎
McCarthy, John. 1978a. History of Lisp. In History of programming languages, ed. Richard L. Wexelblat, 173–185. New York: Association for Computing Machinery. https://dl.acm.org/doi/10.1145/800025.1198360 ↩︎
Graham, Paul. 2002. The Roots of Lisp. Ms., https://paulgraham.com/rootsoflisp.html. [PDF version] ↩︎
See, e.g., Graham’s 2001 “Beating the Averages”. ↩︎
On the “Curse of Lisp”, see for instance Rudolf Winestock’s “The Lisp Curse”; but also this 2021 discussion on r/clojure, where it’s mentioned that Winestock said in 2017 in a comment on (one of the innumerable) Hacker News threads on the topic “I wrote that essay five years ago. Nowadays, just use Clojure or Racket and ignore what I’ve written.” ↩︎
J. McCarthy: LISP History. Talk at MIT, Spring or Summer 1974 (Written from tape 7/10/75, unpublished: cited in Stoyan, Herbert. 1984. Early LISP history (1956 - 1959). LFP ‘84: Proceedings of the 1984 ACM Symposium on LISP and functional programming, eds. Robert S. Boyer, Edward S. Schneider, Guy L. Steele, 299–310. New York: Association for Computing Machinery. [https://doi.org/10.1145/800055.802047]) (Quote from p.307.) ↩︎
Descriptively:
1. (quote x) returns (the symbol) x (rather than the value of x).
2. (atom x) returns t if the value of x is an atom or else '().
3. (eq x y) returns t if the values of x and y are the same atom, or both the empty list; otherwise returns ().
4. (car x) expects the value of x to be cons expression of some sort, like a list, and returns its first element.
5. (cdr x) expects the value of x to be cons expression of some sort, like a list, and returns everything after the first element.
6. (cons x y) returns an object composed of a link between x and y; this is how lists are formed, if the second argument of the innermost cons is ().
7. (cond (p₁ e₁)…(pₙ eₙ)) evaluates the p expressions in order until one returns t, at which point it returns the co-indexed e.
8. ((lambda (p₁…pₙ) e) a₁…aₙ) evaluates each aᵢ expression and replaces the occurrence of all occurrences of each pᵢ in e with the value of the corresponding aᵢ expression, and then evaluates e.
9. (label f (lambda (p₁…pₙ) e)) makes all occurrences of the symbol f behave like the following (lambda (p₁…pₙ) e) expression, including inside of e.
↩︎

With just tad bits more sugar to reduce car / cdr recursion spam:

(label caar
       (lambda (x)
         (car (car x))))

(label cadr
       (lambda (x)
         (car (cdr x))))

(label caddr
       (lambda (x)
         (car (cdr (cdr x)))))

(label cadar
       (lambda (x)
         (car (cdr (car x)))))

(label caddar
       (lambda (x)
         (car (cdr (cdr (car x))))))

↩︎

See McCarthy, John, Paul W. Abrahams, Daniel J. Edwards, Timothy
1. Hart, and Michael I. Levin. 1962. /LISP 1.5 Programmer’s
Manual/. Cambridge, Mass.: M.I.T. Press [pdf], at p.36f and also McCarthy (1960:26–7), as well as the Wikipedia page on CAR and CDR. ↩︎
Church is also well-known for his proof that the Entscheidungsproblem is undecidable (Church’s theorem), the Church-Turing thesis, the Frege-Church ontology, amongst much else. I don’t know any particularly interesting details of his personal life (he was no Richard Montague, about whom more later); he was apparently a lifelong Presbyterian. ↩︎
The M here can be standing in for something more complex, e.g., the whole expression might really be λx.(a (b d)), where M here is a place-holder for (a (b d)). ↩︎
The α-conversion (fifth) rule is: bound variables can be substituted with different bound variables. So, λx.M[x] can be α-converted to λy.M[y]. Bound variable names have no significance except as place-holders, so these two expressions are equivalent. The reason for doing this is to avoid name-clashes (see Two Hard Things?) when combining expressions.

E.g., if we wanted to combine λxλy.(x y) with itself, e.g. ((λxλy.(x y)) (λxλy.(x y))) (this is of the function application form (M N)), we could do α-conversion so we don’t end up with accidental variable capture.

If we didn’t this would happen:

((λxλy.(x y)) (λxλy.(x y))) =

((λy.((λxλy.(x y)) y)) =

(λy.(λy.(y y))

Since variables are arbitrary names, whose point is distinctionness, we need instead to α-convert on one of the expressions:

λxλy.(x y) = (by α-conversion)

λaλb.(a b)

And then we can combine as we tried to before, and correctly arrive at:

((λxλy.(x y)) (λaλb.(a b))) =

(λy.((λaλb.(a b)) y)) =

λy.(λb.(y b))

(We’ll talk more about α-conversion and its implement in a later post.) ↩︎
Well, there’s also (optional/debated) sixth rule: η-conversion, which touches on some more philosophical concerns (though ones which will eventually concern us too), to do with extensionality (vs. intensionality) [essentionally Frege’s Sinn und Bedeutung].

η-conversion says that in a case where (f x) = (g x) for all possible values of x, then f = g. It’s not always something which is implemented, and it raises questions around extensionality vs intensionality vs hyperintensionality we’ll probably touch on a later point.

But, for a taste, by η-conversion (because it’s about extensionality, roughly what’s true in the world rather than conceptually), the following two expressions are identical (G has an extra component involving an identity function):

F = λx.x

G = λx.(λy.y)x

Do they conceptuality (intensionally) count as the same function, F and G? No, perhaps, but their extensions are the same (they’re always produce the same results), and so F counts as the α-reduction of G. ↩︎
On the Church λ story, see Cardone, Felice & Hindley,
1. Roger, 2006. History of Lambda-calculus and Combinatory Logic. In
Gabbay and Woods (eds.), Handbook of the History of Logic, vol. 5. Elsevier. [pdf] On uses of ∧ as notation in formal theories, see here. ↩︎
See Wikipedia on Lamedh. ↩︎
Church, Alonzo. 1932. A set of postulates for the foundation of logic. Annals of mathematics 33(2):346-366. [pdf] ↩︎
Thanks to Dr E.J. Slade for discussion on this point (Church’s constructivism, and the connection with Frege’s positions). ↩︎
And one I’ve worked on implementing in a Lisp at various points, e.g., Frege, the beginnings of a Racket-based DSL for natural language semantics. ↩︎

Group-agnostic previous-focussed-window memory in StumpWM

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Sat, 25 Apr 2020 14:25:00 -0500

I’ve started using StumpWM’s groups (like “workspaces” in other window managers) more extensively, but this broke a behaviour I like: the ability to easily switch back to the last focussed window, because StumpWM’s “last focussed” is group-specific. So I wasn’t easily about to switch quickly back and forth between two windows that were inb different groups, which turns out to be something I frequently want to do (e.g. switch back and forth between an emacsclient frame in my “emacs” group and a Firefox instance in my “web” group).

Here’s my fix [updated with some fixes on [2025-01-27 Mon]]:

;; Global 'last focussed window'
(setf *global-ante-earlier-focussed-window* 'nil)
(setf *global-earlier-focussed-window* 'nil)
(setf *global-prev-focussed-window* 'nil)
(setf *global-cur-focussed-window* 'nil)

(defun panrecord-of-last-focussed-window (currwin lastwin)
  "Record last visited windows and their group."
  (unless (or (search "*EQUAKE*[" (window-name currwin)) ;; don't record Equake
              (equal (cons (current-window) (current-group)) *global-cur-focussed-window*))
    (when (find-window-globally
           (car *global-earlier-focussed-window*) (screen-groups (current-screen)))
      (setf *global-ante-earlier-focussed-window* *global-earlier-focussed-window*))
    (when (find-window-globally
           (car *global-prev-focussed-window*) (screen-groups (current-screen)))
      (setf *global-earlier-focussed-window* *global-prev-focussed-window*))
    (when (find-window-globally
           (car *global-cur-focussed-window*) (screen-groups (current-screen)))
      (setf *global-prev-focussed-window* *global-cur-focussed-window*))
    (setf *global-cur-focussed-window* (cons currwin (current-group)))))

(defun find-window-globally (window group-list)
  "Check for presence of window in all groups."
  (if (equal (car group-list) 'nil)
      'nil
      (if (member window (group-windows (car group-list)))
          window
          (find-window-globally window (cdr group-list)))))

(defcommand switch-to-last-focussed-window () ()
  "Switch to last focussed window, irrespective of which group it is in and what group we're currently in."
  (progn
    (if
     (and
      (not (equal *global-cur-focussed-window* *global-prev-focussed-window*))
      (or
       ;; we're in the same group [same logic below]
       (equal (car (screen-groups (current-screen)))
              (cdr *global-prev-focussed-window*))
       ;; or we can switch to the previous group
       *global-prev-focussed-window*))
     (progn
       (switch-to-group (cdr *global-prev-focussed-window*))
       (focus-window (car *global-prev-focussed-window*) t))
     (if
      (and
       (not (equal *global-cur-focussed-window* *global-earlier-focussed-window*))
       (or
        (equal (car (screen-groups (current-screen)))
               (cdr *global-earlier-focussed-window*))
        *global-earlier-focussed-window*))
      (progn
        (switch-to-group (cdr *global-earlier-focussed-window*))
        (focus-window (car *global-earlier-focussed-window*) t))
      (if
       (and
        (not (equal *global-cur-focussed-window* *global-ante-earlier-focussed-window*))
        (or
         (equal (car (screen-groups (current-screen)))
                (cdr *global-ante-earlier-focussed-window*))
         *global-ante-earlier-focussed-window*))
       (progn
         (switch-to-group (cdr *global-ante-earlier-focussed-window*))
         (focus-window (car *global-ante-earlier-focussed-window*) t))
       (message "No window to switch to."))))))

(add-hook *focus-window-hook* 'panrecord-of-last-focussed-window)

;; my binding; set as you will
(define-key *root-map* (kbd "s-f") "switch-to-last-focussed-window")

The unless statement in panrecord-of-last-focussed-window prevents my drop-down terminal Equake “window” from “counting” for history tracking purposes.

The switch-to-last-focussed-window function essentially just switches to the last focussed window, after making sure it still exists. (If not, switch to the window which was focussed before that one, or the one before that one, or else don’t switch and display message indicating this to the user.)

The last line, (define-key *root-map* (kbd "s-f") "switch-to-last-focussed-window"), means that I can double tap s-f (since my StumpWM prefix key is set to s-f with (set-prefix-key (kbd "s-f"))) to switch to the last focussed window, no matter which group it belongs to.

I continue to really enjoy the power that StumpWM’s Common Lisp underpinnings provides the user!

Running pdfpc in StumpWM

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Mon, 29 Jul 2019 15:27:00 -0600

pdfpc is a fantastic application for presenting PDF slides, including perhaps especially those produced using LaTeX Beamer. It creates two (full-screen) windows, one a presenter viewer which shows the time elapsed and a preview of the next slide, and one the presentation view which is what is shown to the audience. It also has a bunch of other cool features like being able to draw on slides; highlight areas of slides, &c.

Here is an example, with the presenter’s view shown on the left; the audience’s view on the right:

In many environments pdfpc is pretty smart about getting the presentation view to the external display, but in StumpWM both end up getting created in the same frame in the same display.

(General tip: in StumpWM, after connecting/activating an external display, you may need to run refresh-heads to get StumpWM to display things properly.)

For whatever reason, I can’t get any of the Screen-related things in StumpWM to behave like I have more than one screen. E.g. *screen-list* shows a singleton list even when an external display is connected and activated:

STUMPWM> *screen-list*
(#S<screen #<XLIB:SCREEN :0.0 1366x768x24 TRUE-COLOR>>)

A passable solution is to use define-frame-preference, adding the following to your init.lisp StumpWM configuration:

;; pdfpc rule - for 'Default' group
;;  move to frame '1';
;;  non-focussing ('nil) the presentation view;
;;  not only matching (t) the target-group ('Default');
;;  moving the 'pdfpc' 'presentation' window
(define-frame-preference "Default"
  (1 nil t :instance "pdfpc" :role "presentation"))

This should properly move the presentation window to the external display when pdfpc is run.

Guix, Nix: You are in a maze of twisty little $PATHs, some undefined

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Wed, 24 Jul 2019 00:48:00 -0600

Some notes on interactive fiction/text adventure games and PATHs in Guix, and StumpWM.

Maze no. 1

There may (likely is) some way of programmatically setting the X Windows PATH variable in Guix System (née GuixSD) via the base configuration (e.g. config.scm), but I haven’t been able to uncover anything that works. This is relevant for being able to use locally installed static binaries or local shell scripts via the window manager.

As a window-manager-specific workaround, in StumpWM, one can programmatically set PATH variables via (setf (getenv "VARIABLE_NAME") "variable-value"). Thus, if you store local static binaries and shell scripts in ~/bin, the following (which you could include in StumpWM’s init.lisp) will add that to your PATH variable:

(setf (getenv "PATH") (concat "/home/YOURUSERNAME/bin:" (getenv "PATH")))

I use this with a static Haskell binary greenclip, which adds clipboard functionality to rofi, and with shell scripts that give “pretty names” to Flatpak run commands.

For example, the literate/natural-language-based programming interactive fiction design language Inform7 (which is due to be open-sourced sometime this year) is now conveniently available as a Flatpak. But the run command after installing is flatpak run com.inform7.IDE, which is non-ideal. So I made a simple shell script named inform7 placed in ~/bin:

#!/bin/sh
flatpak run com.inform7.IDE

Maze no. 2

Nix can be installed as a standalone package manage on top of other distros, including Guix System, which is useful for be able to obtain software currently lacking in Guix System (including, ironically, Hugo, used by this blog, which is present in Nix). Packages available in Nix but not in Guix include Gargoyle, a very nice interactive fiction front-end client that supports a number of different backends, including Frotz and Glulxe. One of the benefits of Gargoyle is that it “cares about typography”. However, Nix applications by default seem to have trouble finding/seeing fonts, including system fonts, local fonts, and even fonts installed via Nix.

This can be fixed by (1) setting the FONTCONFIG_PATH and FONTCONFIG_FILE, e.g. in StumpWM this can be done with:

(setf (getenv "FONTCONFIG_PATH") "/home/YOURUSERNAME/.config/fontconfig/")
(setf (getenv "FONTCONFIG_FILE") "fonts.conf")

And (2) forcing Nix to look in the right places by manual specification in ~/.config/fontconfig/fonts.conf, adding right before the final </fontconfig> (as appropriate):

<cachedir prefix="xdg">fontconfig</cachedir>
<dir>/home/YOURUSERNAME/.local/share/fonts/</dir>
<dir>/home/YOURUSERNAME/.nix-profile/share/fonts/</dir>
<dir>/home/YOURUSERNAME/.guix-profile/share/fonts/</dir>
<dir>/usr/share/fonts</dir>

And regenerating the font cache (via fc-cache -fv) [possibly you may need to install Nix’s fontconfig package].

Equake(!) Quake-style overlay console in StumpWM

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Wed, 22 May 2019 22:26:00 -0500

I’ve been alternatively using both KDE Plasma 5 and StumpWM on various machines and have got a working model for using the Equake drop-down in StumpWM.

The StumpWM #'invoke-equake command hides (using StumpWM native hide-window, rather than Emacs’s make-frame-invisible as the latter creates various issues in finding and fetching the Equake window) the Equake frame if it’s the currently active window; it searches through all windows in all groups on the current screen/monitor, and calls emacsclient -n -e '(equake-invoke)' to create an Equake frame if no extant Equake window is found; and if an Equake window does already exist for the current screen, it is yanked into the current group, pulled into the current frame, and unhidden (if necessary).

This seems to work pretty well, though I haven’t figured out how to have StumpWM ‘skip’ the Equake frame when cycling through Emacs windows. (And, of course, having a proper partial height floating Equake window would be ideal.)

EDIT: Here’s a better, more Quake-like set-up, which actually makes use of recently added partial-height floating windows in non-floating groups:

(defun calc-equake-width ()
  (let ((screen-width (caddr (with-input-from-stringp (s (run-shell-command "emacsclient -n -e '(equake-find-workarea-of-current-screen (equake-calculate-mouse-location (display-monitor-attributes-list)) (display-monitor-attributes-list))'" t)) (read s))))
        (desired-width-perc (read-from-string (run-shell-command "emacsclient -n -e 'equake-size-width'" t))))
    (truncate (* screen-width desired-width-perc))))

(defun calc-equake-height ()
  (let ((screen-height (cadddr (with-input-from-string (s (run-shell-command "emacsclient -n -e '(equake-find-workarea-of-current-screen (equake-calculate-mouse-location (display-monitor-attributes-list)) (display-monitor-attributes-list))'" t)) (read s))))
        (desired-height-perc (read-from-string (run-shell-command "emacsclient -n -e 'equake-size-height'" t))))
    (truncate (* screen-height desired-height-perc))))

(setq *equake-width* 1368) ; TODO: programmatically get screen dimensions before Emacs starts
(setq *equake-height* 768)

(defcommand invoke-equake () ()
  (let* ((on-top-windows (group-on-top-windows (current-group)))
         (equake-on-top (find-equake-in-group on-top-windows)))
    (if equake-on-top
        (progn (setf (group-on-top-windows (current-group)) (remove equake-on-top on-top-windows))
               (unfloat-window equake-on-top (current-group))
               (hide-window equake-on-top)) ;; then hide Equake window via native Stumpwm method.)
      (let ((found-equake (find-equake-globally (screen-groups (current-screen))))) ; Otherwise, search all groups of current screen for Equake window:
        (if (not found-equake)          ; If Equake cannot be found,
            (progn
              (run-shell-command "emacsclient -n -e '(equake-invoke)'") ; then invoke Equake via emacs function.
              (setq *equake-height* (calc-equake-height)) ; delay calculation of height & width setting until 1st time equake invoked
              (setq *equake-width* (calc-equake-width))) ; (otherwise Emacs may not be fully loaded)
          (progn (raise-window found-equake)
                   (move-window-to-group found-equake (current-group)) ; But if Equake window is found, move it to the current group,
                   (unhide-window found-equake) ; unhide window, in case hidden
                   (float-window found-equake (current-group)) ; float window
                   (float-window-move-resize (find-equake-globally (screen-groups (current-screen))) :width *equake-width* :height *equake-height*) ; set size
                   (focus-window found-equake)
                   (toggle-always-on-top))))))) ; make on top

(defun find-equake-in-group (windows-list)
  "Search through WINDOWS-LIST, i.e. all windows of a group, for an Equake window. Sub-component of '#find-equake-globally."
  (let ((current-searched-window (car windows-list)))
    (if (equal current-searched-window 'nil)
        'nil
        (if (search "*EQUAKE*[" (window-name current-searched-window))
            current-searched-window
            (find-equake-in-group (cdr windows-list))))))

(defun find-equake-globally (group-list)
  "Recursively search through GROUP-LIST, a list of all groups on current screen, for an Equake window."
  (if (equal (car group-list) 'nil)
      'nil
      (let ((equake-window (find-equake-in-group (list-windows (car group-list)))))
        (if equake-window
            equake-window               ; stop if found and return window
            (find-equake-globally (cdr group-list))))))

;; Set the mouse focus policy;  - :click is best for proper Equake functioning
(setf *mouse-focus-policy* :click) ;; also StumpWM default (I think)

;; Keybindings
;; bind to an appropriate key
(define-key *top-map* (kbd "F12") "invoke-equake")

This is what it looks like:

video link here

Browsing the Web with Common Lisp

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Sat, 20 Oct 2018 13:22:00 -0600

I was a long-time user of Conkeror, a highly-extensible browser with an Emacs ethos. It still exists, but since the changes in the Firefox back-end away from XULRunner, which Conkeror uses, running Conkeror became increasingly difficult to use, so I’ve largely switched to just using plain Firefox.

However, John Mercouris has been developing Next Browser (originally styled nEXT Browser), a browser with a Common Lisp front-end, allowing for customisability and extensibility along Conkeror/Emacs lines:

The back-ends are – if I understand correctly – planned to be Blink for the QT port and WebkitGTK+ for the GTK port, with the Mac port of Webkit for the Mac version. But the front-end, the user-facing side, is Common Lisp.

John is currently running an Indiegogo campaign to properly port it to Linux and other non-Mac Unix variants (it apparently runs well already on the Mac, John’s main platform it seems [there’s no accounting for taste ;) ]). The raised money would be used in part to pay a professional C/C++ developer for their time.

Ambrevar is currently working on packaging Next Browser for Guix, which is exciting and promises to add to the amount of Lisp front-end software we’ll be able to use. Currently I’m running Emacs (elisp) for the majority of my non-browser productivity (writing papers & creating class slides using AUCTeX; reading composing email with mu4e; note-taking and scheduling with Org mode; &c. &c.) and, at least on one machine, StumpWM (Common Lisp window manager) for my ‘desktop environment’; and GNU GuixSD with a Guile-based package manager, Guile-based cron (mcron), and Guile-based init/daemon-manager (Shepherd). A functional, configurable, Lisp-based browser would be a most welcome addition. As excellent as Firefox is, especially its backend, I do really miss the halcyon days of Conkeror, and Next Browser could represent a return to those heady days of configurable browsing Emacs-style.

So, if this sort of thing appeals to you (i.e. if you like Lisp, Emacs, and/or highly-extendable browsers), you might want to support the Linux/Unix-port of Next Browser: https://www.indiegogo.com/projects/next-browser-nix-support

There’s only about a week left in the campaign.

Confusion: PDP-10 Zork

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Sun, 23 Sep 2018 23:52:00 -0500

I grew up playing Infocom, Magnetic Scrolls, and Level 9 text adventures, with the Zork trilogy, the Enchanter trilogy, Planetfall, Wishbringer, The Guild of Thieves, The Pawn, Knight Orc, and Silicon Dreams being particularly prominent in my memory (somewhat re-activated through recent listening to the Eaten by a Grue podcast). I would have played all of these on an Atari 8bit or ST computer, and didn’t have any access to anything like a mainframe, and so never actually played the original Zork, which was written in the Lisp-derived MDL language (which formed the basis for the MDL-subset Infocom-specific ZIL language used for their subsequent offerings) for the DEC PDP-10.

Fortunately, some years ago, Matthew Russotto created Confusion, “a MDL interpreter which works just well enough to play the original Zork all the way through” and runs on Linux/Unix and presumably macOS. I’d tried to install this from Arch’s AUR a couple of years ago, without any luck. Inspecting the package build, the AUR Confusion tries to get around errors by using an ancient version of gcc. I found that a few fairly trivial fixes were enough to get it to compile with a recent version of gcc, and so was finally able to try the original Zork:

The most fun I always had in text adventures was exploring, and I’ve always found Infocom’s various “time-limit” elements (batteries running out, health deteriorating, &c.) somewhat irritating. I remembered that there’s a torch in Zork I, and that is also true in the original Zork, and I managed to obtain it early on so as to save my (trusty brass) lamp’s batteries. I then cheated somewhat wildly (though, to be fair, a lot of the puzzles I had figured out once upon a time in Zork I or Zork II; and the bank puzzle and some of the others I wouldn’t have patience for anymore).

The original code has some bugs in it which made things more interesting.

Narrow Ledge
You are on a narrow ledge overlooking the inside of an old dormant
volcano.  This ledge appears to be about in the middle between the
floor below and the rim above. There is an exit here to the south.
There is a very large and extremely heavy wicker basket with a cloth
bag here. Inside the basket is a metal receptacle of some kind.
Attached to the basket on the outside is a piece of wire.
The basket contains:
 A cloth bag
 A braided wire
 A receptacle
 The receptacle contains:
  A newspaper
 A blue label
There is a small hook attached to the rock here.

> untie braided wire from hook

 *ERROR*
 "FIRST-ARG-WRONG-TYPE"
 "First arg to NTH must be structured"
LISTENING-AT-LEVEL 2 PROCESS 1
Atom REP has neither LVAL nor GVAL

I had a similar issue involving the result of a disagreement with a suspicious-looking individual holding a bag. It turns out (thanks to Matthew Russotto for the following information) that this has to do with issues in saving and restoring files, and (failure of) either properly recording or decoding certain values. The balloon issue has to do with a record about the object burning in the receptacle. It is saved as BINF!-FLAG, which should be a boolean-type flag, but at some point it became an object (recording what is burning) and apparently isn’t decoded properly on a restore. Saving-and-restoring during a battle with the suspicious-looking individual produces a similar error to the balloon-burnable error, due to the THIEF-ENGROSSED!-FLAG (which apparently really is a flag) not being saved properly. The upshot (for a player) is that you shouldn’t save during either of these bits of the game.

With these additional lurking grues out of the way, I was able to make it to the end of the game:

Speaking of saving, MDL Zork only has a single restore file. It is stored in the MTRZORK directory and is named ZORK.SAVE. Of course you can create additional saves by copying out this save file to different names.

If you too want to play the original Zork using the original PDP-10 MDL source code, you should be able to build the MDL interpreter necessary from the release at Matthew Russotto’s site (where my minor patches should be incorporated): http://www.russotto.net/git/mrussotto/confusion/releases or from the git repo with my patches at https://github.com/emacsomancer/confusion-mdl.

Or, the easiest of all, install it via Guix (which can installed on top of your current distro as a standalone package manager), as I have created a Guix package which was accepted upstream (the package name in Guix is confusion-mdl). [note: this Guix package stopped building some time ago and I need to figure out what needs fixing. Right now, have a look at the mdlzork project at https://github.com/jordanhubbard/mdlzork , which provides a way to easily play early versions of Zork online at https://jordanhubbard.github.io/mdlzork/.]

Since I played through this right before designing the propositional logic homework for my Semantics class, my students are currently “enjoying” doing Zork-flavoured propositional logic translations, e.g.:

The thief only steals your things when you’re in the cellar.
Your sword glows blue whenever the thief or the troll is near and never any other time.
Whether or not you are lost in a maze of twisty little passages (all alike), the grues are lurking in the darkness.
The grues will eat you unless you bring a light source.

Quake-style drop-down terminal in StumpWM

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Tue, 18 Sep 2018 20:45:00 -0600

One thing I’ve missed in StumpWM is a Quake-style drop-down terminal, like what Guake provides (and I have a Lua-one in my AwesomeWM config). It may be that I’m still haven’t fully absorbed the StumpWM-mindset and that I should be doing this a different way. But up until now when I’m using StumpWM I’ve tended to end up with a heap of terminal windows that are a pain to navigate through (I have a run-or-raise command associated with xterm, but it starts from the first xterm window and usually I want the last – something else to figure out how to do). It’s nice to have a ‘working terminal’ that one can quickly summon and dismiss.

There’s a number of issues for creating a Quake-type drop-down in StumpWM (at least based on my current knowledge). The first is fairly easily dealt with - I don’t want more than one Quake terminal around.

run-or-raise will prevent multiple instances from being created:

(defcommand urxvt () ()
  "Start an urxvt instance or switch to it, if it is already running."
  (run-or-raise "urxvt"'(:title "urxvt")))

(I’m cheating here a bit because I’m not quite sure how to handle title-management. I know windows can be retitled, and I initially tried setting the window title in the above function to “quake”, but somehow I ended up also sporadically re-titling other windows as “quake”, so I’ve side-stepped the whole issue by just using urxvt for my drop-down terminal, with my regular terminal being xterm.)

What we want then is a function that toggles our Quake terminal on-and-off: essentially if the current window is not urxvt, then call the urxvt function defined above, and if the current window is urxvt then switch back to the last used window before urxvt was called. The basic thing we need for this is (if (equal (window-title (current-window)) "urxvt").

The trouble is that StumpWM has a nice feature which is similar to doing C-x b RET in Emacs: it essentially switches to the last window used, so it can be a nice feature for switching back and forth between two different windows you’re working in. The function is pull-hidden-other and it is invoked with “prefix prefix”, which for me is s-f s-f (Super+F, twice), but by default is C-t C- (Ctrl+t twice). Why this creates a problem is because our Quake terminal will muck that up because it will often end up being the last used window, and that’s not what I want. I want pull-hidden-other to switch to the last used window that’s not the Quake drop-down.

I tried a number of different things, and this may still not be the best solution, but it works:

(defcommand rxvt-quake () ()
  "Toggle rxvt-quake window."
  (if (equal (window-title (current-window)) "urxvt")   ; if the current window is quake
      (progn
	(other-window)    ; switch back to window quake was called from
	(select-window-by-number *real-other-window*)  ; switch to the 'real' "other-window"
	(other-window))  ; switch back to the original window - this way after quake finishes, the original configuration is restored
      (progn          ; otherwise, if the current window is NOT quake
      (other-window)   ; first switch the current "other-window"
	(if (not (equal (window-title (current-window)) "urxvt")) ; if the current
								  ; "other-window" is
								  ; quake itself, do
								  ; nothing
	    (setf *real-other-window* (window-number (current-window)))) ; otherwise store the window-number of the current other-window
	(other-window) ; switch back to the window originally called from
	(urxvt)))) ; run-or-raise urxvt

So this function indeed checks to see whether the current window is our Quake drop-down terminal. If it’s not, it first switches to the currently last used window with (other-window) and stores the number of this window in *real-other-window* (but only if this last-used-window isn’t itself urxvt - otherwise we’ll sometime end up ‘over-writing’ the actual last-used-window we want to keep). It then switches back to window that was active when rxvt-quake was invoked, and then it calls the urxvt function which either launches a new urxvt (if none currently exists), or raises the currently running urxvt window. Storing the value in *real-other-window* gives us a way to remember what the ‘real’ last-used window should be.

If rxvt-quake is invoked while the Quake urxvt drop-down is the currently active window, then first (other-window) is invoked, switching us back to the window that was active before we called the drop-down terminal, then we switch to our stored window (which was the ‘real’ last-used window before Quake was invoked), and then we switch back with (other-window) to the window that was active when Quake was first invoked.

This way the window configuration that existed before we summoned our Quake drop-down terminal is restored, and pull-hidden-other effectively ignores the Quake drop-drop.

Bind this command to the traditional F12 with:

(define-key *top-map* (kbd "F12") "rxvt-quake")

And you’re got a ‘traditional’ Quake drop-down terminal in StumpWM.

It is a ‘full-length’ drop-down terminal, however. It might be nice to have it be a fraction of the screen like a more traditional Quake drop-down.

Dockerised Firefox on GuixSD

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Sat, 15 Sep 2018 20:18:00 -0600

So GuixSD doesn’t currently package Firefox (though hopefully that is changing), but only IceCat (which is now EOL). On freenode#guix, pkill9 suggested that Firefox (and Chromium etc.) could be installed on Guix via the Nix installer (install as per instructions on their site and then nix-env -i firefox) with the following trick, create a file ~/.local/bin/firefox with the following content:

# Wrapper to run the Firefox built and packaged by Nix
MESA_LIB=$(dirname $(realpath /run/current-system/profile/lib/libGL.so)) #To get webgl working
export LD_LIBRARY_PATH="$MESA_LIB${LD_LIBRARY_PATH:+:}$LD_LIBRARY_PATH"
#export FONTCONFIG_PATH="$(guix build fontconfig)/etc/fonts${FONTCONFIG_PATH:+:}$FONTCONFIG_PATH"
export FONTCONFIG_PATH="$(guix build fontconfig)/etc/fonts"
exec -a "$0" "/nix/var/nix/profiles/per-user/$USER/profile/bin/firefox" "$@"

And then add ~/.local/bin to your $PATH in ~/.profile.

Unfortunately I haven’t been able to get that to work, though I could be doing something daft.

I figured out the following (elaborate) alternative workaround for Firefox (no luck for Chromium):

1. Install Docker via Nix. Launch with sudo dockerd.
1. Create a Dockerfile for Firefox

# firefox in a docker container
# the following line will start firefox in the container, thus
# video and sound will be played on the host machine
FROM alpine:edge
MAINTAINER slade@jnanam.net

# add testing repo + install packages + add user and group
RUN echo "http://dl-cdn.alpinelinux.org/alpine/edge/testing/" >> /etc/apk/repositories \
    && apk add --no-cache \
       xz \
       dbus-x11 \
       ttf-dejavu \          # a bunch of fonts; you may not want all of these
       ttf-freefont \
       ttf-ubuntu-font-family \
       font-noto \
       font-noto-extra \
       font-noto-emoji \
       font-noto-oriya \
       font-noto-tamil \
       font-noto-avestan \
       font-noto-gothic \
       font-noto-myanmar \
       font-noto-telugu \
       font-noto-deseret \
       font-noto-bengali \
       font-noto-kannada \
       font-noto-ethiopic \
       font-noto-armenian \
       font-noto-tibetan \
       font-noto-sinhala \
       font-noto-gurmukhi \
       font-noto-malayalam \
       font-noto-gujarati \
       font-noto-devanagari \
       font-noto-thai \
       font-noto-adlam \
       font-noto-nko \
       font-noto-lisu \
       font-noto-carian \
       font-noto-buhid \
       font-noto-osage \
       font-noto-hebrew \
       font-noto-arabic \
       font-noto-chakma \
       font-noto-gothic \
       font-noto-khmer \
       font-noto-cypriot \
       font-noto-kayahli \
       font-noto-mandaic \
       font-noto-olchiki \
       font-noto-thaana \
       font-noto-georgian \
       font-noto-shavian \
       font-noto-cherokee \
       font-noto-oldturkic \
       font-noto-osmanya \
       font-noto-glagolitic \
       font-noto-tifinagh \
       font-noto-adlamunjoined \
       font-noto-nko \
       font-noto-lao \
       arc-theme \
       hunspell \
       hunspell-en \
       firefox \
       libcanberra-gtk2 \
       pulseaudio \
    && rm -fr /var/cache/apk/* \
    && adduser -D -u 1000 -g 1000 user

# add user's work
WORKDIR /home/user

# switch to user
USER user

ENTRYPOINT ["/usr/bin/firefox", "--no-remote"]

Save the above out to a file Dockerfile in some directory. cd to that directory. Then create a Docker container with docker build -t someprefix/firefox . (with someprefix being whatever you like).

1. Create in ~/.local/bin/firefox:

#!/bin/sh
# Wrapper to run the Chromium built and packaged by Nix
xhost +local:docker@; \
docker run --rm -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix \
-v /dev/snd:/dev/snd \
-v /run/user/$USER_UID/pulse:/run/pulse:ro \
-v /home/$USER/.mozilla:/home/user/.mozilla \
-v /home/$USER/.cache/mozilla:/home/user/.cache/mozilla \
-v /tmp/Downloads:/tmp/Downloads \
-v /home/$USER/.gtkrc-2.0:/home/user/.gtkrc-2.0 \
-v /home/$USER/.config/gtk-3.0/:/home/user/.config/gtk-3.0 \
-v /home/$USER/.nix-profile/share/fonts/truetype:/usr/share/fonts/truetype-nix \ # for nix fonts, if you have them here
-v /home/$USER/.nix-profile/share/fonts/ubuntu:/usr/share/fonts/ubuntu-nix \     # ditto
-v /home/$USER/.nix-profile/share/fonts/noto:/usr/share/fonts/noto-nix \         # ditto
-v /home/$USER/.guix-profile/share/fonts/truetype:/user/share/fonts/truetype-guix \ # for guix fonts, if you have them here
-v /home/$USER/.guix-profile/share/fonts/opentype:/usr/share/fonts/opentype-guix \ # ditto
 --shm-size 2g  --privileged someprefix/firefox  # substitute your prefix here for "someprefix"

Where again someprefix is whatever you chose before. chmod +x ~/.local/bin/firefox and then you can run Firefox with firefox. (You’ll have to make sure dockerd is running.)

Perhaps this could be useful elsewhere. It has the advantage of being a nice Alpine Linux musl-libc/libressl build of Firefox running inside your GuixSD (or whatever you’re using).

What doesn’t work:

Spellchecking - I don’t quite know why, hunspell should be available. Just disable spellchecking for now (otherwise everything will be underlined in red).
Wire‘s web portal.
Saving or uploading from anywhere but /tmp/Downloads (but this is a feature really).
If you want CJK fonts, you’ll have to install them in Nix or Guix (Alpine doesn’t seem to have any); this is indicated in the Firefox launch script above.

Managing emacsclient windows in StumpWM

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Sat, 18 Aug 2018 21:36:00 -0600

I’m still working on getting my GuixSD machine configured, including working on getting familiar with StumpWM – a windows manager written in Common Lisp – which is the desktop paradigm I’ve decided upon for this Lisp-centric machine.

I’m somewhat habituated to (my) AwesomeWM keybindings, which involve the Super key in combination with various other keys, including say s-1 for tag/workspace 1, s-3 for tag/workspace 3, &c., and s-E (i.e. hold Super and Shift and press e) to launch an emacsclient (see below on the Emacs client/daemon configuration). StumpWM could be configured in a somewhat similar fashion (though it doesn’t seem to quite use tag/workspaces in the same fashion), but the ‘tradition’ seems to be to use a prefix, which is by default C-t (that is, hold Control and press t), which is then released and followed with another key or key combination. I don’t really like using Control for windows management since it tends to conflict with bindings in Emacs and elsewhere, so I’m testing out s-F (hold Super, press f) as a prefix (though whether I’ll stick with prefixed bindings or go back to single action bindings, I’m not yet certain).

From browsing other stumpwm configs, I came across a useful bit of configuration:

(defun run-or-raise-prefer-group (cmd win-cls)
  "If there are windows in the same class, cycle in those. Otherwise call
run-or-raise with group search t."
  (let ((windows (group-windows (current-group))))
    (if (member win-cls (mapcar #'window-class windows) :test #'string-equal)
	(run-or-raise cmd `(:class ,win-cls) nil T)
	(run-or-raise cmd `(:class ,win-cls) T T))))

This function can then be used with specific applications, e.g.:

(defcommand run-or-raise-icecat () ()
	    (run-or-raise-prefer-group "icecat" "Icecat"))

The above function leverages run-or-raise-prefer-group to either launch Icecat, if it is not already running, or else focus the Icecat window, and successive calls will cycle through multiple Icecat windows/windows if more than one Icecat window exists. This is extremely useful as it’s much less cognitively-tasking than figuring out which window number Icecat currently is associated with.

This can then be assigned a binding(s) like:

(define-key *root-map* (kbd "W") "run-or-raise-icecat")
(define-key *root-map* (kbd "C-W") "run-or-raise-icecat")
(define-key *root-map* (kbd "s-W") "run-or-raise-icecat")

This means that one first presses the prefix (s-f for me) followed by either W or C-W or s-W (that is, Shift+w, Control-Shift+w or Super-Shift+w) to either launch Icecat or focus/cycle through existing Icecat windows/windows.

Now, the way I typically use Emacs is to invoke it as a daemon (i.e. emacs --daemon) and then connect Emacsclients to this daemon (i.e. emacsclient -c for the ‘windowed’ gtk-application, emacsclient -t in the terminal). However, if we define a parallel function for Emacs, a particular edge-case arises:

(defcommand run-or-raise-emacsclient () ()
		(run-or-raise-prefer-group "emacs" "Emacs"))

This works rather like the Icecat one as long as at least one Emacsclient window exists (so if there are multiple window, successive calls of run-or-raise-emacsclient will cycle through them). However, if no emacsclient is currently open, it will launch an undaemon’ed Emacs (requiring loading the entire init.el), even if/though an Emacs daemon is currently running and thus could be attached to.

At least one solution to this is the following function (with relevant keybindings):

(defcommand decide-on-emacsclient () ()
 (if (equal (run-shell-command "pgrep \"emacsclient\"" t) "")
     (run-shell-command "emacsclient -c")
     (run-or-raise-emacsclient)))

(define-key *root-map* (kbd "s-e") "decide-on-emacsclient")
(define-key *root-map* (kbd "C-e") "decide-on-emacsclient")
(define-key *root-map* (kbd "e") "decide-on-emacsclient")

The above function executes the shell-command pgrep "emacsclient" and evaluates whether the output of that shell-command is equal to the empty string (which will be the case only when no emacsclient is running). Where at least one emacsclient is running, it executes the run-or-raise-emacsclient function defined earlier, focussing/cycling through running emacsclients. Where no emacsclient is currently running, it executes instead emacsclient -c, opening a windowed emacsclient. And the bindings let me press the prefix and then either Super+e, Control+e or simply e to execute this new function.

I also have the following definitions:

(defcommand emacsclient-launch () ()
  (run-shell-command "emacsclient -c"))

(define-key *root-map* (kbd "s-E") "emacsclient-launch")
(define-key *root-map* (kbd "C-E") "emacsclient-launch")
(define-key *root-map* (kbd "E") "emacsclient-launch")

So that if I want to launch a new emacsclient window instead of switching to an existing one, I can do so using one of series of keybindings parallel to the previous set, but with a capital E rather than a lowercase one.

This is working well for me, and is a nice example of the power of using Lisp-based ‘desktop environment’.

Guix: You are in a maze of lispy little passages, (map equal? ′(′all ′alike) ′(′all ′alike))

Benjamin.Slade@fakeEmailToMakeValidatorHappy.com (Benjamin Slade) — Sat, 04 Aug 2018 21:47:00 -0500

So I finally made a serious go of running GuixSD, a GNU Linux distro which is largely built on GNU Guile Scheme (a dialect of Lisp) on one of my machines (one I had actually put together with GuixSD in mind: an X200 Thinkpad, which I Libreboot‘ed and put a Atheros Wi-Fi card in), and, to increase both the quantity and variety of Lisps involved, am trying to use with StumpWM (which is written in Common Lisp).

It’s a fascinating distro, modelled on Nix, but implemented in Guile. It’s not been exactly easy to get running (one of the videos on GuixSD from Fosdem 2017 included the line “[GuixSD] is like Gentoo for grown ups”), in part because its architecture is rather different from what I’ve experienced with other Linux distros, which use different package managers perhaps and sometimes even different libc’s, but generally follow a similar design philosophy. Rather than spreading out configuration across lots of different pieces, GuixSD seems to largely try to concentrate it in specific configuration files which are transactional in that they can be rolled back (and thus the system can be rolled back to known working states).

It is a GNU-blessed distro, and does take the FSF’s hard line (and to my eyes sometimes weird line) approach to software. So no proprietary software is included in the Guix repos, including firmware (and it runs on the linux-libre kernel). That by itself is fine, but it means the state of affairs for Guix-packaged browsers is pretty poor. No Chromium, no Firefox. IceCat 52 is essentially what’s currently available (if IceCat were up to the latest Firefox ESR 60, it might be easier) in terms of browsers which might be considered secure.

This led me to try to use the Nix installer by itself^* to try to install Firefox and Chromium. Sadly, I can’t get Nix’s Chromium to work at all on GuixSD, and while Firefox works fine, I can’t get it to see my locally installed fonts (or other fonts I’ve installed via Nix).

Hopefully at some point Next Browser will be packaged for Guix, to bring in another major component written in (Common) Lisp. And when (if?) IceCat 60 comes out, that will alleviate the pain somewhat. (I was a long-time Conkeror user, and I briefly tried it again in GuixSD, but I’m not certain of its security and uBlock Origin no long works with it, which I believe is why I stopped using it in the first place).

Other interesting Lispy pieces include mcron, a cron which accepts (as well as Vixie cron style, I think) Guile config files. The examples in Guix manual I couldn’t really get to work. But via the help-guix listserv I found that one can put simple guile scripts in ~/.config/cron/job.guile. Working out how to do a ‘run this every N minutes’ was not immediately obvious, but I figured out how to do it, e.g.:

; execute run_me every 5 minutes
(job '(next-minute (range 0 60 5)) "run_me")
; run execute_me every 2 hours
(job '(next-hour (range 0 24 2)) "execute_me")

One of the other great things about GuixSD is that its init manager, GNU Shepherd, is also written in Guile Scheme. I’ve only had a chance to play with it a little bit, but it seems very nice and it’s good to find other innovative init managers (I would mention here also runit and s6) which take very different approaches to systemd (another innovative init, or perhaps init+, but one that creates more problems than it solves in the end, in my experience).

On the Guix package manager itself: I learned the hard way that searching for packages in Guix is really only comfortable within Emacs: so do guix package -i guix-emacs and then do everything else from guix-emacs within Emacs ( M-x guix-search-by-name to search package names by regex; and M-x guix-search-by-regex to search names+descriptions by regex). The results returned by guix package -s .... in a terminal are not very browseable (though I tried valiantly for some time). But if you’re interested in Guix, you’ll likely interested in Emacs anyway.

What I’m trying to build on this machine is something with lots of Lisp. Of course the kernel is still a kernel written in C, as are lots of the other pieces like the terminal &c., but much of the user-facing things: the package manager, the windows manager, the init, the job scheduler (=cron), and (most importantly perhaps) the ‘text editor’ (read: document composer, email interface, irc interface, twitter interface, blog post interface, code editor …) are all largely written in and interacted with using some form of Lisp (Guile Scheme, Common Lisp, Emacs Lisp).

Guix is a bit like Emacs, I think. It’s an incredibly powerful tool with lots of interesting possibilities, but when you start using it you’re presented with an empty screen with little indication of what you can do. I’ll be sticking with it, I think. Now I’ve got to get to grips with StumpWM and figure out how to configure polybar…

[And if you’re curious about why Guix is pronounced like “geeks”, have a look at this post over on my linguistics blog.]

* As well as being the package managers of both of their respective distros, both the Nix and Guix package managers can be used on top of other distros. Nix doesn’t have quite the same hard line approach to software licences as Guix.