> This is maybe the best introductions to Lisp i have seen, especially for a js dev like me it could hardly get more approachable and convincing.
The classical one is "The Nature of Lisp"[0], which introduces data-as-code through XML and Java build tools. Same idea, just with examples more relevant at the time of writing. Still worth a read for non-webdev programmers.
Having learned Lisp, it's half funny, half disheartening to watch the industry repeatedly tries to rediscover "code as data", as people's configuration or data files in markup language du jour grow in complexity and eventually start directly encoding executable code... and then stop shy of embracing the code/data duality.
I think that should be the marketing of Lisp: A necessary evil ;)
But seriously speaking, for me Lisp would be much more appealing if it had been introduced to me as a specialized niche language, though for an important niche nonetheless, instead of as the be-all-and-all language for your superpowered startup [1].
(For those that don't already have a goto list of counterarguments to "Lisp all the things": My main contra-point to Lisp is that with all the meta programming powers you get, you write yourself into your own little corner where no-one but you and your friends live. You want advanced syntax highlighting, linting, automatic refactoring for your special features? Write it yourself! You want outsiders to participate (think: new employees)? Write all documentation yourself, too!)
Having worked in a Common Lisp startup, I'll only agree with half of your counterarguments :).
> You want advanced syntax highlighting, linting, automatic refactoring for your special features? Write it yourself!
That's true to an extent. Simple things are simpler in Lisps, because the syntax is trivial. So highlighting and structured editing are easy. The rest, is near impossible, at least for a full-featured Lisp like Common Lisp. That's a consequence of being an extremely dynamic language, that ships a compiler and intertwines parsing, compiling and execution. The flip side of being able to run arbitrary computation at compile time is that, in general, you can't know what the program will do until you run it. The dominant free CL IDE, SLIME, does just that: it queries your running Lisp image for its current state, to provide you with formatting and autocomplete and other hints.
That said, "near impossible" is a function of community size. Had CL anywhere near as much popularity as Java or Python does, I'm sure folks at IDEA would make automatic refactoring work for CL as well :).
> You want outsiders to participate (think: new employees)? Write all documentation yourself, too!
That I strongly object to. Code generation and compile-time execution aren't magic, or even particularly hard concept. They're just another flavor of code. You manage it the same way as regular code - you package it into modules with well-defined interfaces, and document them. I've worked with CL in a team settings, and I've worked on legacy CL code that's as old as I am; it's not harder than "regular" legacy code. And you always have to write your documentation yourself, there's no escape from that, no matter what language you use.
>in general, you can't know what the program will do until you run it
It needs to be added that in Common Lisp, once the program is running you can do everything with it while it's running: Inspect the stack frames, change variable values, rewrite/recomiple/update functions, update class definitions, update objects to said new class definitions, save the current program state to disk,etc.
So it's programming that is geared to RUNNING the program and modifying it while its running.
Seems you're arguing for using all the powers a programming language gives you, nothing in particular about Lisp there. As with many things, it depends mostly on the people writing the code and their process, rather than what programming language they are using.
I never inherited a Clojure-codebase that has been written during long duration, but I have digged around in a fair amount of Clojure-codebases that are open source, both user focused and libraries. Same with JavaScript. And I can say, as someone with more JS experience than Clojure, that the Clojure projects tend to be a lot easier to understand than the JS ones, probably not because of the language, but because of the habits that the language "forces" you into.
I'm mostly arguing against extensive use of meta programming, which is promoted as one of the main reasons to view data as code, as meta programming is inherently disadvantaged for automated tooling. Formulated as a trade-off: When I have to choose, I prefer richness of IDE (and other tooling) automation to program creation automation that I write myself (i.e. meta programming).
This trade-off is not restricted to Lisp, but also applies to e.g. C++ template meta programming. Compilers have gotten betters with templates, but still debugging the more advanced usages of templates can become hellish. Codifying the features the meta programming supplies in the language itself or in well supported libraries means that error messages get better and many usage scenarios are documented on Stackoverflow.
I don't doubt your experience regarding Clojure vs JavaScript code bases, but this probably has to do with other reasons than the meta programming the original blog post is about?
> I'm mostly arguing against extensive use of meta programming, which is promoted as one of the main reasons to view data as code, as meta programming is inherently disadvantaged for automated tooling.
Which is weird, because in most Lisps meta programming happens at compile time. It should be possible for tooling to just show you the expansion of any given macro invocation. In fact, when I used Clojure a decade ago, there was an Emacs command that would expand the macro invocation under your cursor, so you could see what code was actually being generated.
So I don't really think that's a fundamental limitation. I suspect it's more related to the fact that Lisps aren't especially popular, and don't get the attention from tooling that other languages do.
> Codifying the features the meta programming supplies in the language itself or in well supported libraries
But without the meta programming, those libraries might not actually be possible to write. You will either end up with a more dynamic interface (doing meta-stuff at runtime), or a clunkier and more verbose interface. I think being able to expand a macro invocation to see what it turns into is enough for all but the hairiest of macros.
> It should be possible for tooling to just show you the expansion of any given macro invocation. In fact, when I used Clojure a decade ago, there was an Emacs command that would expand the macro invocation under your cursor, so you could see what code was actually being generated.
Exactly. On common lisp, for example, it's just a keypress, and it has a "macro stepper" so it shows the first expansion possible, then the second expansion, and so on and so on... until you end up with compiler primitives!
> You want outsiders to participate (think: new employees)? Write all documentation yourself, too!)
Shouldn't you be doing this anyway? Not trying to be snarky: it's that every job I've worked at has a classic underdocumentation problem. Tribal knowledge dominates, and the practicalities of shipping product overrun the need for teaching employees, new and old, about the idiosyncrasies of the system. [edit] This seems hardly an issue with LISPs or DSLs in particular.
> My main contra-point to Lisp is that with all the meta programming powers you get, you write yourself into your own little corner where no-one but you and your friends live. You want advanced syntax highlighting, linting, automatic refactoring for your special features? Write it yourself!
This is contrary to my experience (though I've mostly used Racket rather than Common Lisp). Tracking what's a function vs macro, where an identifier is introduced/used, etc., is all baked into the underlying language and not something I have to write myself for every language extension I make.
> You want outsiders to participate (think: new employees)? Write all documentation yourself, too!
Were you planning to not document your internal-use utility code?
"Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp."
Also:
> Hacker Robert Morris later declared a corollary, which clarifies the set of "sufficiently complicated" programs to which the rule applies "...including Common Lisp."
>Hacker Robert Morris later declared a corollary, which clarifies the set of "sufficiently complicated" programs to which the rule applies "...including Common Lisp."
This is referring to CL implementations that depend heavily on C.
Many CL implementations now are almost 100% Lisp code.
JavaScript isn't based on Scheme, it's what would have been Scheme if deadlines and marketing needs didn't make Netscape use a hacky toy language instead.
The place where JavaScript contains "an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp" is Babel.
If you take anything away from that: brackets are the most compact way to serialize a tree, for some reason they are called s-expressions. The parse tree for lisp is the syntax tree, you just rename each bracket node to whatever the first atom in the bracket happens to be.
One thing that XML could have done that s-expressions can't is a way to serialize a dag if proper nesting of tags wasn't required. I'm playing around with a language based on that, the mental load of having no clear way to delineate blocks visually makes it hard to reason about, unfortunately.
The classical one is "The Nature of Lisp"[0], which introduces data-as-code through XML and Java build tools. Same idea, just with examples more relevant at the time of writing. Still worth a read for non-webdev programmers.
Having learned Lisp, it's half funny, half disheartening to watch the industry repeatedly tries to rediscover "code as data", as people's configuration or data files in markup language du jour grow in complexity and eventually start directly encoding executable code... and then stop shy of embracing the code/data duality.
--
[0] - https://www.defmacro.org/ramblings/lisp.html