An agent in 100 lines of Lisp

I haven’t seen an LLM generate syntactically invalid code in any language in … a year? So I don’t know if “more likely to be syntactically valid” is a good enough selling point.

But maybe I’m interpreting you too narrowly?

> A Lisp program that writes a Lisp program really just needs to produce a list of (nested lists) of tokens. A JavaScript program that writes a javascript program needs to generate a string that is syntactically valid JavaScript code. That is a much bigger task than just constructing a (nested) list.

> Because Lisp syntax is so much simpler than that of JavaScript etc. it is much easier to avoid errors when generating code. In JavaScript you can use JSON to generate data, but JSON can not carry functions around.

First of all, the LLM does not produce structured tokens, it produces (tokens of) text. It does not have a concept of nested or structured tokens. Which means that producing a Lisp program and a JavaScript program is basically the same difficulty, i.e. LLMs producing function foo () {} is about the same task as producing (defun foo () ()).

In fact, because most Lisps uses the same token ( and ) for almost all delimiters, the LLM in fact gets confused a lot more than Algol-family languages that uses various different tokens for different purposes. It generates thinking traces that are a few screens long while trying to count the closing parenthesis and the depth, something that I have not found to be the case for other languages, even with languages much more obscure than Lisp. (And no, it is not a training data issue, because the Lisp family as a whole is pretty well represented in the data set, due to Emacs Lisp.)

> I think this idea makes a lot sense. Instead of making the LLM generate JSON or XML, why not make it generate Lisp, which can carry both programs and data?

You do realize that all programming languages contains both programs and data, right? i.e. JSON is literally a subset of the JavaScript language, all JSON documents are valid JavaScript code, can be embedded in JavaScript programs, and so on. This isn't even a JavaScript-specific thing, almost all recent programming languages have data structure literals.

The thing that makes Lisp unique is that it can modify programs as data in the language easily, and why would that be a unique capability that would be beneficial for LLMs, when it can just sed/awk or tree-sitter-parse programs with more conventional languages and modify it as easily?

> Elixir is basically Lisp, but with better syntax

Which so helpfully removes many of the benefits of lisps :P I don't understand this argument at all, if it's not s-expressions nor quacks like a lisp, in what way is it lisp at all? Making it algol/C-like already makes it like the rest of the 99% of the languages, without any of the easy benefits of the neat and simple syntax of lisps.

> I think a more accurate description is that lisp code is just cons cells and cons cells is both how we write the code and how the runtime itself implements lists. So there’s basically no distinction between a lisp list and the text representation of the source.

Heresy I suppose, but doesn't feel (to me) like it "specifically" has to be cons cells, as long as it's a "list" of some sort, regardless of the specific type of list, while I'd say "cons cells" is a specific implementation of a list, famously implemented by Common Lisp and similar lisps.

Besides that I agree with you, the code you write and the code the compiler uses is the same, that's why it gets easy to edit with code itself. Rust and similar don't have this feature at all, it's very different experience writing macros with lots of special syntax, compared to just writing "normal" code doing the same sort of operations. "Doing a loop of X" isn't the same inside of a normal runtime function body, as when you want to do a loop of arguments for the function signature in a macro, as just one simple example.

Not to mention in Rust you have two different types of macros, declarative vs procedural macros, already speaking to that the entire concept of homoiconicity isn't there. You want it to be easy to implement a read, evaluate, print and loop back workflow by passing native data through the entire chain, except parsing the initial user-input string into your AST (whatever shape that ends up being), which pretty-printed (basically) looks the same as the user-input.

Lately I've been adding a repl service to my docker compose configs so the agent can easily send fragments of code for execution in the project context without incurring the clojure startup costs each time.

So cool to watch the AI get into a tight learning loop when it has access to all the internal data structures.

Clearly, you’re not going to do it for just Fibonacci. And yes, you can always rewrite the agent code, quit, and restart to modify the agent. This is just like using an LLM to modify Pi and then restarting, which is done today. But being able to rewrite code in a live, running system is useful for all the reasons that Lispers have done it that way for 50+ years. In some cases, you can’t just serialize your state and reload it easily. Sometimes, for instance, you might not even have access to the whole state because some of it resides on a remote server.

That’s a distinction in search of a reason though.

Being able to run code in the same unix process or a new one doesn’t really matter all that much in the context of self modifying code. But even if we cared about that, this isn’t a LISP specific feature. All dynamic languages support eval.

And having the agent cache the tool for reuse is a really trivial problem to solve. Though I do agree that LISP makes this much easier than in many other languages.

This is certainly a cool tech demo. But the claims of its novelty are overstated

That's technically true but practically an agent can just save the script file/rerun it/write a tool that lets it call a reply with memory etc.This aspect is a bit more elegant when it's in the execution context, but the core of "you don't need to predefine tools, the agent can dynamically write its own code" is not exactly new - that's pretty much the basis of why Claude code and codex and all the other agent harnesses are so much more effective than old ways if working with llms - they can just write giant incomprehensible bash scripts on the fly to accomplish random things without having pre-defined tools, write state to files, etc

Right but given that the agent only outputs text, and agents are perfectly happy writing python, the supposed benefit of Lisp is completely irrelevant here.

you can use LFE (Lisp flavored erlang) = lisp on BEAM runtime.

you get a snippet from LLM, compile it to module, and hot-load it into the running node. the module lives in the node's code table, so it persists and every other agent can call it. not just the one that wrote it.

the agents themselves are seaprate supervised processes, so if one crashes - e.g. because the snippet was crap, it doesn't take down whole system.

of course you can do that in just elixir too, the lisp is just cosmetics really.

It doesn't matter if a human or a machine wrote it, it's just unpleasant to read. I don't know what the OP's motivation for their post was, but I think it's reasonable to point out that something is harder to read than it should be.

Why should I bother reading something that the “author” couldn’t even be bothered to write themselves?