Using Guile for Emacs

Benefits for LWN subscribers

The primary benefit from subscribing to LWN is helping to keep us publishing, but, beyond that, subscribers get immediate access to all site content and access to a number of extra site features. Please sign up today!

Emacs is, famously, an editor—perhaps far more—that is extensible using its own variant of the Lisp programming language, Emacs Lisp (or Elisp). This year's edition of EmacsConf, which is an annual "gathering" that has been held online for the past five years, had two separate talks on using a different variant of Lisp, Guile, for Emacs. Both projects would preserve Elisp compatibility, which is a must, but they would use Guile differently. The first talk we will cover was given by Robin Templeton, who described the relaunch of the Guile-Emacs project, which would replace the Elisp in Emacs with a compiler using Guile. A subsequent article will look at the other talk, which is about an Emacs clone written using Guile.

LWN looked at Guile-Emacs way back in 2014, when Templeton had completed the last of several Google Summer of Code (GSoC) internships working on it. Around that time, Templeton had a fully functional prototype, but they moved on to other things until recently reviving the project.

Guile-Emacs

Guile is an implementation of Scheme, which is a language in the Lisp family, as is Elisp, they began; Guile is also the official extension language for the GNU project. The goal of the Guile-Emacs project is to use Guile to implement Elisp in Emacs. There are two main components to that: an Elisp compiler built on Guile and an Emacs that has its built-in Lisp implementation completely replaced with Guile.

There are several benefits that they see for Emacs from the project. They believe it will improve performance while also increasing expressiveness for Elisp. The latter will make it easier to extend Elisp and to experiment with new features in the language. It will also reduce the amount of C code in Emacs because it will no longer need a Lisp interpreter since Guile will be providing that. In answer to a question, Templeton said that roughly half of Guile is C code, but it is largely only used for the lower layers, while the C code in Emacs is more widespread in its use. Also, with Guile, much more of Emacs can be written in Lisp than can be done now, which also reduces the amount of C code needed.

Guile is a good choice for a few reasons, Templeton said. While it is primarily a Scheme implementation, it also has support for multiple languages using its compiler tower. In order to support a new language in Guile, it is only necessary to write a compiler for the source language to Tree-IL, "which is essentially a low-level, minimal representation of Scheme". All of Guile's compiler optimizations are done on Tree-IL or at lower levels, so a new language will benefit from those.

Guile also has some features that are uncommon in other Scheme implementations, including a nil value that is both false and the empty list, as it is in Elisp. Guile also has GOOPS, which is a version of the Common Lisp Object System (CLOS) and its metaobject protocol.

The idea of using Guile for Emacs has a long history, going back at least three decades, they said. There have been half-a-dozen previous implementation attempts along the way. The current project got its start from a series of GSoC projects, beginning with Daniel Kraft's 2009 project, followed by Templeton's five GSoC internships, 2010-2014 (which can be found in the GSoC archive under the GNU Project for the year).

Over that time, they modified the Emacs garbage collector and data structures for Lisp objects to use the libguile equivalents. They also replaced the Lisp evaluator in Emacs with the Elisp compiler from Guile. A year after their last GSoC project, they had an Guile-Emacs prototype that "was completely compatible with Emacs functionality and with external extensions". The performance was poor, because they were focused on correctness and ease of integration with the C code in Emacs, but it was "a major milestone for the project".

They gave a brief demo of Guile-Emacs by typing some Elisp into an Emacs scratch buffer and evaluating it. Scheme/Guile can be easily accessed from within Elisp, which was shown by calling the Guile version function and by producing a Scheme rational number from integer division. Perhaps the most interesting piece was the demonstration that the classic recursive factorial function did not actually call itself, because Scheme requires tail-call optimization, which turns those calls into jumps.

The demo (and the rest of the talk) can be seen from the EmacsConf page for the talk, where there are videos of the talk and Q&A session along with a transcript of the talk and, at the time of this writing, an unedited transcript of the Q&A. There is some additional information on that page for those wishing to dig in more.

Resurrection

In 2015, Templeton left their university to go to work on web technologies, so Guile-Emacs went dormant. That has been changing recently, because they have been working with Larry Valkama over the last few months to rebase Guile-Emacs onto a development branch of the Emacs upstream code. There were actually a series of rebases onto various versions of Emacs, which worked increasingly poorly since Emacs internals had changed over time. Currently, they have "a version of Emacs 30 which boots correctly and can be used for interactive debugging, as well as the ability to bisect the revisions of Emacs and find out where regressions were introduced".

The immediate goal is to finish the rebase, but performance is next up. They want to get the Guile Elisp performance to be competitive with the existing Emacs Elisp; it is roughly half as fast right now. "Guile Scheme is quite often an order of magnitude faster than ordinary Elisp" based on microbenchmarks like the "Gabriel benchmarks" (from the book Performance and Evaluation of Lisp Systems by Richard P. Gabriel), so there is still a lot of room to improve Guile-Emacs. The hope is to have a usable version of Guile-Emacs based on Emacs 30 sometime in the next four or five months (by northern-hemisphere spring, Templeton said).

There is also an effort to get some of this work upstream. On the Guile side, that includes optimizing the dynamic-binding facilities; dynamic binding is rarely used in Scheme, but is used frequently in other Lisp dialects, including Elisp. For Emacs, they want to work on "abstracting away the details of the Lisp implementation where they're not relevant", which will make it easier to integrate Emacs and Guile Elisp. It also cleans up some code that will make things easier for anyone working on Elisp.

They also have plans to add features to Elisp, including the Scheme numeric tower, tail-call optimization, and more Common Lisp compatibility. Access to the Fibers Guile Scheme library is planned. Fibers is based on ideas from Concurrent ML and "provides much more powerful facilities for concurrent and parallel programming than what Emacs currently offers".

The idea is that this work furthers the goals of Guile-Emacs and it can perhaps be integrated into the upstream projects relatively soon. Templeton said that it is worth considering "what effect Guile-Emacs might have on Emacs if it becomes simply Emacs". The amount of C code in Emacs has increased by 50% in the last decade and now is around 1/4 of the codebase. But that C code can be a barrier to extending and customizing Emacs; writing more of Emacs in Lisp will make it more customizable. C functions that are called both from Lisp and C in Emacs (around 500 of them) cannot practically be redefined from Elisp.

Common Lisp

One way to speed up the process of writing more of Emacs in Lisp would be to use a Guile implementation of Common Lisp, they said; the essential ingredients for doing that are already present in the Scheme and Elisp implementations in Guile. Implementation code could be shared with other open-source projects, such as Steel Bank Common Lisp (SBCL) and SICL, too. Even though Common Lisp has a reputation as a large language, they think getting it running on Guile would be a matter of "months rather than years".

Common Lisp would bring other advantages, including the ability for Elisp to adopt some of its features. From the perspective of reducing the amount of C in Emacs, though, Common Lisp "would also provide us with instant access to a huge number of high-quality libraries" for handling things that Guile is lacking, such as "access to low-level Windows APIs" and "interfaces to GUI toolkits for a variety of operating systems".

Templeton did not mention it, but LWN readers may remember that Richard Stallman is not a fan of Common Lisp; rewriting Emacs using it is not likely to go far.

Meanwhile, if most of Emacs is written in Lisp, it may be possible to use Guile Hoot to compile it to WebAssembly and run it in browsers and other environments. Writing more of Emacs in Lisp would also be "a great victory for practical software freedom" because it would help make the freedom to study and modify programs easier.

When Emacs is implemented primarily in Lisp, the entirety of the system will be transparent to examination and open to modification. Every part of Emacs will be instantaneously inspectable, redefinable, and debuggable.

It would also allow Emacs extensions to do more. An experiment that Templeton would like to try would be to use the Common Lisp Interface Manager (CLIM) as the basis of the Emacs user interface. CLIM is "a super-powered version of Emacs's concept of interactive functions", they said, but trying that in today's Emacs would be difficult; if the lowest layers were customizable via Lisp, though, it would be almost trivial to do. They noted that there was another EmacsConf 2024 talk on using the McCLIM CLIM implementation with Elisp.

They closed the talk with suggestions on how to get involved with Guile-Emacs. Trying it out and providing feedback are one obvious way; there is a Codeberg repository for the project that will contain both a tarball and Guix package, they said. Bug reports and feature requests are welcome as well, as are contributors and collaborators. The project is being developed by a "small worker cooperative", so donations are a direct means of supporting it.

Q&A

The talk was followed by a Q&A session. Templeton reviewed the IRC log and Etherpad, noting that they had expected the Common Lisp piece to be the most controversial (it "would piss people off") because it is not part of either the Emacs or Guile communities. In addition, one of the motivations for transitioning from C to Lisp was left out of the talk. As more Lisp is added atop a high-performance Lisp implementation, the less sense it makes to call out to C to speed up operations, in part because of the cost of the foreign-function interface (FFI). C limits the use of some "advanced control structures" like continuations, as well, so there is more to be gained by providing ways to make the Lisp code faster, Templeton said.

A few of the questions related to Common Lisp, including one on whether there is active work on it for Guile. Templeton said that they have been working on it in their spare time over the last few years and have implemented a few chapters of the Common Lisp HyperSpec (CLHS). Lately, their focus has been on researching ways to ergonomically support a polyglot Lisp environment, where Common Lisp, Scheme, and Elisp can all work together easily. One of the problem areas is the differences between a Lisp 1, which only has a single namespace, as with Scheme, and a Lisp 2, like Common Lisp and Elisp, where a name can have different definitions as a function or variable. They have been looking into some ideas that Kent Pitman has on combining the two.

The "elephant in the room" question was asked as well: does Templeton know if the Emacs maintainers are interested in using Guile? They said that they are unsure how the current maintainers feel about it, though the reception overall has been "generally cautiously optimistic". There are political aspects to a change of that sort, but from a technical perspective, some previous Emacs maintainers "didn't think that it was a bad idea". Templeton knows that current Emacs maintainer Eli Zaretskii is concerned about cross-platform compatibility, so Guile-Emacs needs to be "rock solid" in that department before any kind of upstreaming can be contemplated.

The talk was interesting, as is the project, though there is something of a quixotic feel to it. An upheaval of that sort in a codebase that is as old as Emacs seems a little hard to imagine, and the political barriers may well be insurmountable even if the technical case is compelling. Based on this talk (and others at the conference), though, there is some pent-up interest in finding ways for Emacs to take advantage of advances in other Lisp dialects.