132 KiB

Raw Blame History

Why Ruby is an acceptable LISP | Random Hacks

Random code snippets, projects and musings about software from Eric Kidd, a developer and entrepreneur. You're welcome to contact me!

Why Ruby is an acceptable LISP

Dec 03, 2005 • by Eric Kidd

Years ago, I looked at Ruby and decided to ignore it. Ruby wasn't as popular as Python, and it wasn't as powerful as LISP. So why should I bother?

Of course, we could turn those criteria around. What if Ruby were more popular than LISP, and more powerful than Python? Would that be enough to make Ruby interesting?

Before answering this question, we should decide what makes LISP so powerful. Paul Graham has written eloquently about LISP's virtues. But, for the sake of argument, I'd like to boil them down to two things:

LISP is a dense functional language.
LISP has programmatic macros.

As it turns out, Ruby compares well as a functional language, and it fakes macros better than I'd thought.

Ruby is a denser functional language than LISP

A dense language lets you say things concisely, without obfuscation. You can see more of your program in one glance, and there aren't as many places for bugs to hide. Beyond a certain point, the only way to make programs denser is to use more powerful abstractions.

One particularly powerful abstraction is lambda. Using lambda, you can create a new function on the fly, pass it to other functions, and even store it for later use. For example, if you wanted to double each number in a list, you might write:

(mapcar (lambda (n) (* n 2)) mylist)

mapcar creates a new list by transforming each element of mylist. The transformation, in this case, could be read as "for each value n, multiply n by two." In JavaScript, you'd write lambda as function, which is perhaps a bit clearer:

map(function (n) { return n*2 }, mylist)

Of course, this is only a hint of what you can do with lambda. Languages which favor this style of programming are called functional languages, because they work with functions. A dense functional language can be very concise indeed, and quite clear once you learn to read it.

How does Ruby stack up against LISP for functional programming? Let's consider Paul Graham's canonical example, a function which creates an accumulator:

(defun foo (n) (lambda (i) (incf n i)))

This code is marginally shorter in Ruby, and the notation will be more familiar to C hackers:

def foo(n) lambda {|i| n+=i} end

acc = foo 3
acc.call(1)   # --> 4
acc.call(10)  # --> 14
acc.call(0)   # --> 14

But there's an interesting special case in Ruby which saves us even more typing. Consider a (very silly) function which takes a lambda as an argument:

;; Call 'fn' once for each natural number.
(defun each-natural-number (fn)
  (loop for n from 1 do (funcall fn n)))

;; Print 1, 2, 3...
(each-natural-number
 (lambda (n) (format t "~D~%" n)))

Now, we could write the same function in Ruby:

def each_natural_number(fn)
  n = 0
  loop { fn.call(n += 1) }
end

each_natural_number(lambda {|n| puts n })

But we can do better. Let's get rid of lambda and fn using yield:

def each_natural_number
  n = 0
  loop { yield n += 1 }
end

each_natural_number {|n| puts n }

Yes, yield is a special-purpose hack, and yes, it only works for functions which take a single lambda. But in heavily functional code, yield buys us a lot. Compare:

[1,2,3].map {|n| n*n }.reject {|n| n%3==1 }



(remove-if (lambda (n) (= (mod n 3) 1))
           (mapcar (lambda (n) (* n n))
                   '(1 2 3)))

In a large program, the difference adds up. (In LISP's defense, it's possible to write a reader macro which makes lambda more concise. But this is rarely done.)

Ruby gives you about 80% of what you want from macros

At this point, the LISP hackers are saying, "A good syntax for lambda is nice, but what about macros?" And this is a good question. LISP macros are functions that:

Run in the compiler, and
Transform custom syntax into raw LISP.

The most common use of LISP macros is to avoid typing lambda quite so much:

(defmacro with-each-natural-number (n expr)
  `(each-natural-number (lambda (,n) ,expr)))

(with-each-natural-number n
  (format t "~D~%" n))

defmacro defines a function that takes a list as an argument, and returns another list. In this example, our macro is called every time the compiler sees with-each-natural-number. It uses LISP's "backquote" syntax to quickly construct a list from a template, filling in n and expr. The list is then passed back to the compiler.

Of course, this macro would be useless in Ruby, because it's working around a problem we don't have.

The second most common use of LISP macros is to create mini-languages for defining stuff:

;; Generate some bindings to our database
;; using a hypothetical "LISP on Rails."
(defmodel <order> ()
  (belongs-to <customer>)
  (has-many <item> :dependent? t))

Using Ruby on Rails, we could write:

class Order < ActiveRecord::Base
  belongs_to :customer
  has_many :items, :dependent => true
end

Here, belongs_to is a class function. When called, it adds a bunch of member functions to Order. The implementation is pretty ugly, but the interface is excellent.

The real test of any macro-like functionality is how often it gets used to build mini-languages. And Ruby scores well here: In addition to Rails, there's Rake (for writing Makefiles), Needle (for connecting components), OptionParser (for parsing command-line options), DL (for talking to C APIs), and countless others. Ruby programmers write everything in Ruby.

Of course, there's lots of advanced LISP macros which can't be easily ported to Ruby. In particular, macros which actually compile mini-languages haven't appeared yet, although they might be possible with enough work. (Ryan Davis has done some promising work in this direction with ParseTree and RubyInline, and I'll be writing about related techniques as I discover them.)

Ruby's libraries, community, and momentum are good

So if LISP is still more powerful than Ruby, why not use LISP? The typical objections to programming in LISP are:

There aren't enough libraries.
We can't hire LISP programmers.
LISP has gone nowhere in the past 20 years.

These aren't overwhelming objections, but they're certainly worth considering.

Once upon a time, Common Lisp's standard library was considered huge. But today, it seems painfully tiny. Java's manuals fill a wall, and Perl's CPAN archive has a module for anything you can imagine. Common Lisp, in comparison, doesn't even have standard way to talk to the network.

Similarly, LISP programmers are scarce. If you're around Boston, there's a small pool of grizzled hackers who can very nearly work magic. Elsewhere, there's a thin scattering of curious young hackers. But LISP has always been a minority language.

Ruby, on the other hand, is growing rapidly in popularity. The big driver seems to be Rails, and the ramp-up started in late 2004. If you're trying to launch a company, it's more-or-less a cliché that every potential employee is a Rails nut. Rails will soon trickle back into ordinary web consulting, and from there--eventually--into big business.

Ruby has also been around long enough to develop a good standard library, and a large archive of add-on libraries. If you need to download a web page, parse RSS, generate graphs, or call a SOAP API, you're all set.

Now, given a choice between a powerful language, and popular language, it may make excellent sense to pick the powerful one. But if the difference in power is minor, being popular has all sorts of nice advantages. In 2005, I'd think long and hard before choosing LISP over Ruby. I'd probably only do it if I needed optimized code, or macros which acted as full-fledged compilers.

(Thank you to Michael Fromberger for reviewing an early draft of this essay.)

Want to contact me about this article? Or if you're looking for something else to read, here's a list of popular posts.

Excellent Article wrote on Dec 03, 2005:

Eric, thank you for this excellent article. I have toyed with Lisp for several years now, and while I agree it is more powerful, I have made the decision to build my sites in Ruby for many of the reasons you have cited.

I am constantly re-evaluating my choice of language tools, and have wondered if I made a mistake not building my site in Lisp. (It’s still in development now, but keep an eye our for Zifus around the first part of 2006).

I recently read an article by the developers of reddit that suggested they are considering re-writing their site at some point in the future. He cited as a specific difficulty with Lisp that many of the libraries that quickly become available in other languages are slower to appear in Lisp because of the smaller community.

Who knows if he was dissembling to throw potential competitors (like me) off track, but in the end it doesn’t matter, the point is valid. Whereas I have been incredibly happy with the progress of Rails as a framework, and I have been utterly delighted with my personal productivity in this wonderful framework.

Excellent article, too bad I can post it on Zifus yet, but soon, very soon.

zlxcgrogdss wrote on Dec 03, 2005:

I’m looking forward to your next editorial, perhaps it could be titled something like “Why Java is an acceptable Forth”. Maybe, if I’m lucky, “Why assembly is an acceptable binary”.

What I’m, err, hinting at, is that Ruby is not Lisp. A Lisp is a Lisp because it’s code is represented as its basic data structure. Ruby’s isn’t (unless you argue that Ruby’s code consists of strings, and string is a baic data structure; though I’m sure you’ll agree that this would be pure sophistry). Lisp without this property is like a wheel that isn’t round.

The above distinction, along with all of its implications, is the reason Ruby can not be a substitute for Lisp, which, I am assuming, was what you meant to say when you erroneously called Ruby “a Lisp”. Ruby is neat and all, but let’s not get carried away.

bill a wrote on Dec 03, 2005:

I took basically the opposite path – I abanboned Ruby in favor of Lisp.

You have some good points, and I agree that Ruby is the most Lispy language that isn’t actually a Lisp (CL, Scheme, Dylan). But although blocks can get you surprisingly close to macros, they can’t take you all the way there. As you pointed out, the implementation of macro-like code in Ruby tends to be ugly behind the scenes. Ruby’s syntax isn’t regular, and so can’t be operated on with macros. This means you’ve got to shoehorn various abstractions into doing what you want. Instead of processing the code passed to a block as a list and simply transforming it into something else, you have to define the elements of your mini-language as methods in some class. The pervasiveness of Ruby’s OO becomes a hassle here. (Sure, lots of defmacro’s end up being ugly too, but at least they’re conceptually cleaner). And all user-defined languages and structures immediately stand out because they have to be set off with the “do” keyword.

There are too many things about Lisp that no other language will ever get unless it is also a Lisp. The ability to move through sexp’s with Emacs keychords is priceless. The simple structure of Lisp and its lack of arbitrary syntax makes it easy to provide good editing support. The whole idea of interactive development rocks. The ability to, in most implementations, compile to native code rocks as well. Lisp is pretty mature, and while there are a few quirks to it, 50 years of existence will go a long way toward making a programming language great.

I guess what bothers me about Ruby is its quirks. That might sound crazy, since it’s well-known that Common Lisp has more than its share of historical cruft. But I really cannot get my mind back into the framework of arbitrary syntax. It is ultimately useless. It prevents you from writing macros, it prevents editors from having flawless support for your language, and it prevents you from expressing yourself as concisely as you need to.

Your lambda sample is indeed shorter in Ruby than in Lisp. But Ruby’s OO is not quite so clean in this situation:

(defmethod sum-children ((c item)) (loop for child in (children-of item) sum (height-of child))) (mapcar #’sum-children list-o-children)

I don’t think that can be done quite so easily in Ruby. The best I can come up with is:

class Item def sum_children children.inject( 0 ) { |c, s| c.height + s } end end list_o_children.map { |c| c.sum_children }

It isn’t possible to (easily?) treat the concept of summing the children’s height as a function to pass to others.

Also, blocks are little more than callable code. You can try to impose some structure on them by defining appropriate methods and evaluating the block in a special context, but this is not the same as the code transformation macros provide. You cannot, for instance, selectively evaluate different expressions in different contexts. Consider:

(let ((x 45)) (defclass foo () ((name :initform x))))

That will work because defclass will expand the initform (and only the initform!) into a lambda-expression and thus close over the x variable; nothing else will be evaluated in that context. It’s also important to notice that defclass does not look any different than let, even though defclass is simply a macro, and let is a built-in special form.

I will clearly go on for days unless I stop now. I was about to write about how let’s rock because you can limit a variable’s scope with absolute precision (not possible in Ruby) and then about how Marco Baringer implemented continuations in straight Lisp. To sum up: yes, Ruby is not that bad (maybe even “acceptable”) but Lisp owns.

Eric wrote on Dec 03, 2005:

Ruby’s most frustrating quirks, from a LISP perspective, tend to involve its scoping rules. Methods, variables and constants each have subtly different behavior. And as for block locals, even Matz admits that they’re pretty broken.

Let’s consider a Ruby version of LET:

def let(*args) yield *args end
let(1,2,3) {|x,y,z| z }

This does exactly what you want if the variables are unbound: It introduces a new scope with three local variables. On the other hand, if any of the variables were previously bound, Ruby clobbers the existing bindings. Nasty.

The biggest limitation of trading LISP’s compile-time macro expansion for Ruby’s metaclass hackery is launch time. Rails works around this using FCGI and on-demand class loading.

These headaches aside, I’ve had very good luck porting LISP macros to Ruby. At this point, I could probably reimplement about half of the macros in On LISP without much trouble. Another quarter could be brute-forced using ParseTree, but that’s (arguably) cheating. I’ll talk more about these techniques in the coming months.

Anonymous wrote on Dec 03, 2005:

Hi. I’ll just assume here you always meant Common Lisp when you wrote LISP (it isn’t capitalised there, just like PERL and JAVA aren’t right).

Before answering this question, we should decide what makes LISP so powerful. Paul Graham has written eloquently about LISP's virtues. But, for the sake of argument, I'd like to boil them down to two things:
LISP is a dense functional language.
LISP has programmatic macros.

Sorry to say, but that doesn’t reflect reality. If those were the two main reasons to use Common Lisp, most people probably wouldn’t.
Incidentally, Paul Graham’s Common Lisp style is definitely atypical (google for “Graham Crackers”). His Scheme background/affection certainly shows, and he usually doesn’t mention all the other things that make Common Lisp so useful (unlike in Scheme, avoiding side-effects and using recursion instead of iteration in Common Lisp is rather rare).
Despite the roots, Common Lisp isn’t much of a “functional” language. It has all the necessary features, but imperative constructs and OOP play a major role in the language. It’s not Common Lisp if it doesn’t have the Common Lisp Object System; it also wouldn’t be anywhere as powerful without the condition system and all the other things (full numeric tower; the type system; etc.).

I have no doubt that Ruby might match CL in the functional department, but that won’t get you very far. Common Lisp has come a long way, and even though many people seem to reduce it to macros and higher-order functions, that view does certainly not reflect the way people use it in reality — it’s not very attractive or particularly useful (in comparison) when reduced to that set of features.

2005, I'd think long and hard before choosing LISP over Ruby. I'd probably only do it if I needed optimized code, or macros which acted as full-fledged compilers.

I’m sure Ruby is a good choice, but if you’re using Lisp just for the macros, you’ve probably missed all the interesting features…
(In case you’re curious, Practical Common Lisp is a good, freely available book that covers a lot more ground than just macros and higher-order functions)

null wrote on Dec 03, 2005:

You guys don’t understand…your precious little languages is a moving target. You’ll never write code that’ll survive 20 years with them.
If all you’re doing is web pages, I suppose that’s all right. But only to a point where data integration and AI comes to the web.
Than, Ruby, Perl and Python are gone.

What Lisp needs is more people. If the lisp community had /half/ of the people who enthusiastically throw themselves full hearted in every new hype, we’d have those libraries.

But don’t kid yourself, we already have our mini CPAN, some stuff for web, and free stuff is progressing all the time.

Erik Enge wrote on Dec 03, 2005:

Nice article, though I think zlxcgrogdss (second comment) is spot on with why Ruby is not a Lisp. Anyways, “there aren’t enough libraries” is being worked on and you can see current progress at http://common-lisp.net/projects.shtml and http://cliki.net/Library for those who might be curious.

Hope that’s an ok comment to leave, if not , feel free to pull it.

Dookus Binglebib wrote on Dec 03, 2005:

But don't kid yourself, we already have our mini CPAN

Wow, already?

eh wrote on Dec 03, 2005:

try embedding prolog in ruby.

Eric wrote on Dec 03, 2005:

Anonymous, I agree that CLOS is a Good Thing.™ In particular, generic functions lead to much cleaner designs than traditional message-passing. This is a subject near and dear to my heart. You’re also right to call out Common Lisp’s condition system. It’s an elegant approach to exception-handling, and one that future language designers should study carefully.

That said, if your Common Lisp style doesn’t rely heavily on macros, you’re missing out on half the fun, not to mention your number one sales pitch. :-)

Eric wrote on Dec 03, 2005:

Null, Dookus, could you please be polite? Heated arguments are perfectly OK, but please address each other with respect.

EH, I’ve been wrestling with the Prolog-in-Ruby problem for a few weeks now. The backtracking is easy, and I’ve gotten a decent pattern matcher running. The right-hand side of the rules, though, is proving a little tricky.

Garrett Snider wrote on Dec 04, 2005:

Thanks Eric for the article and everyone else for the comments. I’ve been coming at Ruby as a long-time Java guy with a now and again interest in LISP and wandering what the LISP community thinks of Ruby. I love it as a step up from Java and I’ve been thinking that it is reasonably close to LISP, at least in power (not in language design). However, I wanted to get a deeper undestanding of the (practical) differences.

Also, I’ve taken the pill that allows me to love the “arbitrary syntax” mentioned by ‘bill a’ above. I find it hard to convince others who have the PERL vs. Python dualing philosophies in mind and stand on the Python (/Java) side, but as a way to implement Domain Specific Languages, I think Ruby’s open syntax shines. Where I’ve had a question is on the following issue:

If the additional syntax options are layered on top of a solid consistant core (like with LISP), it seems to be solid advantage, perhaps even outshining the venerable LISP in this respect. (Specifically, I’m thinking of Executable DSLs.) So commentary here regarding the fundamental soundness and consistancy of the Ruby language vs LISP is especially interesting and appreciated.

bill a wrote on Dec 04, 2005:

OK, guys….you can say whatever you’d like about Lisp, but please, please, please stop capitalizing it. It’s Lisp, not LISP.

jimmy wrote on Dec 04, 2005:

Thanks, Bill. That was bugging me too.

Smalltalking with a Lisp wrote on Dec 04, 2005:

Ruby was inspired mainly by Smalltalk and Lisp. If you really want your hair blown back, check them out. Ruby and Python are gateway languages to the above.

bill a wrote on Dec 04, 2005:

I am curious how much of Ruby comes from Lisp. I would tend to think that people who are familiar with Lisp do not go around writing languages. :)

Unordinary Programmer wrote on Dec 04, 2005:

“Some may say Ruby is a bad rip-off of Lisp or Smalltalk, and I admit that. But it is nicer to ordinary people.”

- Matz, LL2

Anonymous Coward wrote on Dec 04, 2005:

Hmm. Lisp macros are far more powerful than the trivial use cases you’ve listed. I could give a lot of examples here, but just ask yourself: why is most of the programming community so fond with “Design Patterns”, while the Lisp community generally isn’t?

Well, that’s because patterns are nothing but high-level specifications for code being rewritten again and again and again. The Lisp approach is to create some macros and auxiliary functions that actually implement the pattern, thus extending the language capabilities and avoiding continuous reinvention of the wheel.

It has happened for object orientation (CLOS anyone?), aspect-oriented programming, OO<→SQL mapping… All without touching the basic language syntax and implementations.

Ruby features may compete in some specific cases, but I bet that the same consideration would apply for other languages too.

Finally, regarding the Lisp libraries: it’s true that the language specs are full of historical cruft (e.g. mandatory support for the now-dead versioning filesystems), and lack things like standard sockets. But it’s also true that there’s a good amount of free (as in freedom) libraries that fill the gap. Have a tour on http://cliki.net/ for an appetizer.

Beowulf wrote on Dec 04, 2005:

I’ve made a relatively lengthy response to the posts here on my blog. http://zifus.blogspot.com/2005/12/power-vs-popularity.html

You may not agree with my reasoning, but realize this is the reasoning process I went through when I decided to build my system in Rails instead of Lisp. Realize that I love Lisp and I would have used it if I could’ve focused on solving my actual problems rather than trying to integrate various frameworks together and struggling to get it all working as a cohesive whole. The biggest reason I abandoned Java and C# is that I wanted to focus on my problem, and not fight with getting the language and framework to do what I want. This is coming from a lover of Lisp, imagine what someone who either isn’t a fan of the language or doesn’t even know it exists is thinking.

Tayssir John Gabbour wrote on Dec 04, 2005:

Ruby’s creator is outspokenly influenced by Lisp and Smalltalk, yet Ruby’s success is supposed to be a terrible thing to Lisp users? Particularly since Smalltalk’s creator also speaks of his influence and admiration of Lisp? ;)

I (perhaps badly) recall that things in motion tend to have lower friction coefficients. If Ruby’s technical advantages get the mainstream moving, great.

I think this is a more reasoned article than I remember seeing from the Java and Python worlds.

However, I think it has two honest errors. For one thing, I wouldn’t call Common Lisp a functional language, if by that we mean it is biased towards recursion over other techniques. I use iteration heavily; Lisp’s LOOP macro is pretty impressive in this regard, and there are even more powerful 3rd party iteration constructs.

Perhaps Rubists focus on functions because they correctly find a lot of power in them, but then they’re viewing Lisp through Ruby-colored glasses.

Second, as for plain macros, these are just functions which (at some time like “compiletime”) take code and return code. They take a sexp and return a sexp.

What’s the purpose? Usually expressiveness. Readability.

Since they can use other aspects of the language like dynamic scope, they multiply the expressive power of the language. But they must be used tastefully, as power requires justification.

Since I don’t write lambdas much in the first place, my macro usage certainly ain’t to save effort typing lambda. ;) Completely orthogonal concepts.

Anonymous (again) wrote on Dec 04, 2005:

From Eric:
That said, if your Common Lisp style doesn't rely heavily on macros, you're missing out on half the fun, not to mention your number one sales pitch.

Ok, I agree with that; my initial response may have been a little rash :-)
My point remains, though, I’d say that more of half of the fun comes from all the other parts often not mentioned. Macros are an essential part, but just one of many.

To Beowulf: It sounds to me like you are playing the bitter ex-martyr yourself (no offense intended) there.
Multiple commercial Lisp vendors are doing just fine, even though their implementations are not cheap and even though there are various actively developed high-quality open-source implementations. That alone shows that the market exists. As David Thornley put it, “Lisp doesn’t look any deader than usual to me”, and all the people saying it the opposite won’t change that (funny, back when I used FreeBSD, “FreeBSD is dead” was the typical (periodically occuring) outcry of many not-users. I guess it’s not particular to Lisp after all).
I agree that it isn’t as widely used for open-source software as many contemporary languages are, but it is used surprisingly often once you look further; in particular, the harder the problems you’re solving, the more it seems to be used.
The nature of most free Lisp implementations tends to favour long-running programs (they don’t focus on small initial memory footprints or delivering small binaries), and the benefits of the language help you more the harder your problems get — abstractions scale (in dealing with the complexity of your program), syntactic shortcuts don’t. I suspect that this is also why most/many Lispers don’t care too much about getting a shorter syntax for (lambda (…) …): It simply doesn’t win you much (in terms of shorter code size) in non-trivial programs; other factors become much, much more important. (This is also why micro-benchmarks are usually not representative unless you write fibonacci or factorial programs all day)

You must occupy a niche if you will […]

No, (in my humble opinion) the whole power of Common Lisp comes exactly from the fact that it doesn’t focus on niches. It focuses on power. If you want to occupy a niche, you can easily extend Common Lisp to do that (I think it’s called “writing your own DSL” nowadays :-); Prolog-in-Lisp, Lisa, AllegroCache and AspectL are just a few examples of that (all implemented as normal libraries, IIRC). I’m certain that Common Lisp couldn’t have survived for this long weren’t it so extensible: Perl, Python, Java, they and most other new languages all reinvent themselves every other year, and I’m sure they would have been succeeded by “competing” languages by now if they didn’t — most niches are moving targets and the competition doesn’t sleep. The Common Lisp standard, in comparison, has not changed for a long time and probably won’t (need to) anytime soon.
Sure, that means that writing a simple webpage in PHP may be simpler; Perl is certainly more comfortable to use for a quick one-liner; Erlang should beat Common Lisp easily in the domain Erlang was created for, and so on. Where Common Lisp shines, however, is when your problem is not covered by a typical niche. It shines when your problem is hard and your requirements aren’t simple, when you actually can make use of the advanced features. If another language has a library/framework that does just what you want to do, and Common Lisp doesn’t, why use Common Lisp, anyway?
I think using one language for everything is usually a bad sign (lack of flexibility on part of the developer), not a good one (no language is best at everything, and I never understood people who think that. Also, life would be terribly boring if that were so).
It’s simply not true that people didn’t/don’t write frameworks in Common Lisp, but I suspect that what you consider a framework is usually just called a library in Common Lisp. If you think it is dead in the commercial world, franz.com (one commercial Lisp vendor) alone has more than enough success stories/impressive products to prove that wrong (I guess Lord of the Rings (the movie), Jak and Dexter and Orbitz are typical modern examples). cliki.net / lispwire.com have plenty of links to open-source libraries/software.

I’ll close with a quote from Kent Pitman:
“…Please don’t assume Lisp is only useful for Animation and Graphics, AI, Bioinformatics, B2B and E-Commerce, Data Mining, EDA/Semiconductor applications, Expert Systems, Finance, Intelligent Agents, Knowledge Management, Mechanical CAD, Modeling and Simulation, Natural Language, Optimization, Research, Risk Analysis, Scheduling, Telecom, and Web Authoring just because these are the only things they happened to list.”

Eric wrote on Dec 04, 2005:

Bill, the lowercase spelling “Lisp” is certainly correct for Common Lisp (and I’ve fixed it in the main article—thanks!). The uppercase spelling, however, has a long and respectable history. The new way certainly is easier on the eyes, but my LISP teachers were a bit old-school, and I haven’t overcome my nostalgia yet. :-)

As for your query about how often LISP hackers write new languages: Oddly enough, it’s a bit of an obsession. Yeah, I know, it makes no sense. But H. sapiens always did tend to mess with perfection.

Tayssir, I would be delighted if Ruby encouraged programmers to take a good look at LISP (and SmallTalk). And Ruby is certainly good news for metaprogramming in general. Maybe we’ll finally stop hearing about macros being “too powerful,” a personal peeve of mine.

Ben wrote on Dec 04, 2005:

Nice article. I’m not a Ruby user but I will say one language feature it has which Common Lisp doesn’t — first class continuations.

Also, given the success which Python has had in porting to the CLR, has anyone seriously thought about making a Lisp-like language for .Net? I mean, there are some things out there, but they seem either not well supported or rather ML-like (which is fine but not exactly like Lisp.)

Joku wrote on Dec 04, 2005:

The article didn’t really compare how things could be done in Python, even though it started from the premise that Ruby might be more powerful. Not really a surprise though, because it would’ve turned out that Python code can be nearly identical in complexity to Ruby code (sometimes slightly more complex, sometimes slightly less). And neither Ruby nor Python come close to providing the same power as macros in Lisps.

I don’t think you gave any reason to consider Ruby over Python.

bill a wrote on Dec 04, 2005:

Eric,

Sure, many years ago “LISP” was the correct rendition, but I don’t think that’s been the case for quite some time.

Incidentally, all of those examples you pointed out (Dylan, Scheme, Goo, Logo) are all dialects of Lisp. :-)

Anonymous Coward wrote on Dec 04, 2005:

About continuations in Lisp: here’s an example of a language extensions provided by using macros.

Maybe Common Lisp doesn’t have “native” continuations because they can be created in several ways. See “On Lisp” by Paul Graham for an example (the book is downloadable from his website, if you don’t want to spend money).

Maybe they are not syntactically as nice as the Scheme built-in continuations, but for practical pourposes they are just “good enough”. And more syntactic sugar is always available by using some macro magic.

First Rest wrote on Dec 04, 2005:

This article sounds like it was written for folks who really want to use Lisp, but have chosen Ruby because all the cool kids are using it and want to reasonably justify an emotional decision.

Lisp has been around a long time and has watched other, more popular languages attempt to catch up to it. Here’s a clue: they still haven’t. Figuring that out involves a difficult learning process that challenges most of the things you have learned and involves looking at things in an entirely new way. If you want to learn a language that can change with the times and incorporate whatever latest fad the programming cool kids have to offer, Lisp is the choice. Lisp is the red pill.

One question to ask one of the other commenters (who put his comments on a blog that doesn’t allow anonymous posting) that mentioned the small Lisp community: If everyone does like you did and decide to move on, how would the community ever grow? Low hanging fruit is so much easier to pick.

Lisp + Python = Lython wrote on Dec 04, 2005:

What about a sexpr-based language built on top of Python, that generates Python bytecode, can access Python libraries and even supports macros?

http://lemonodor.com/archives/000648.html

(Ok, it’s just a prototype, but here’s my 2c about the Ruby vs Python vs Lisp issue).

Eric wrote on Dec 04, 2005:

It’s true, Joku, that Python metaprogramming has become more common recently, using techniques very similar to those popular in the Ruby world. This is a good thing, and deserves broader notice.

Guido, however, has never been much a fan of functional programming, and has stated (PDF, page 4) that he doesn’t like lambda, map, filter, reduce, and similar constructs. Normally, this isn’t a problem, because Python has slick list comprehensions:

[ n*n for n in [1,2,3] if (n*n)%3!=1 ]

In a lot of common cases, list comprehensions will look better than either the Ruby or LISP examples above. But there’s an underlying problem: List comprehensions are a special-purpose feature, and I can’t use the same mechanisms for building my own domain languages. And since Python’s lambda is very limited, it generally won’t help much.

Effectively, Python allows me to create new kinds of definitions (by hacking metaclasses), but it doesn’t allow me to define new control constructs. Normally, this doesn’t matter much, but it’s a nuisance if you’re trying to build something like Rake.

bill a wrote on Dec 04, 2005:

One big difference between Lisp and Ruby is that Ruby gets its dynamicity and its metaprogramming capabilities by doing a lot of code evaluation at runtime. To people who aren’t familiar with Lisp, it might seem that this is the only way to do it.

So for instance in Lisp, I might write a macro to define part of a domain-specific language. That macro is then expanded into other code at compile-time, and that’s all there is to it. Ruby, on the other hand, takes a much more runtime-centric approach; instead of transforming code, I just have to make sure that a given block is evaluated in a context where the methods it calls can be accessed. So Ruby can’t make many assumptions about code it reads until that code is actually running.

The two approaches have advantages and disadvantages, but I (of course ;-) find Lisp’s approach cleaner. It means that the compiler can warn about calling undefined functions, and that development environments can easily do things like locate code definitions. With Ruby’s approach, in general the semantics of any given snippet of code can only be determined at runtime. To be sure, Lisp does have an eval function, but it doesn’t get used much at all outside of Lisp itself. This is a good thing. When I was using Ruby, I was sometimes frustrated about this situation – that all decisions about code are made at the last possible minute. Of course, this lets you do things like method_missing and so on, but I have yet to miss that (no pun intended) since switching to Lisp. I don’t mean to suggest that everything in Lisp must be statically prepared (as in C or Java), only that in general Lisp takes a different approach to dynamicity.

Just another point to keep in mind.

Eric wrote on Dec 04, 2005:

Sweet! Lython’s definitely a slick hack, and—judging from the sample code—the author noticed that Python’s bytecode is actually more LISP-like than its official syntax. Python’s if statement really is an expression; the parser just refuses to nest it. In general, the Python Language Services are a pretty promising framework for implementing a macro system; somebody just needs to roll up their sleeves and make it happen.

Bill, you’re absolutely right about LISP and compile-time versus runtime performance. Exhibit A: Ruby on Rails takes a couple of seconds to launch a new server process. With a good compiler, LISP could do the same thing as fast as the OS could read pages into memory.

The long-term trends probably favor the LISP (or even Scheme) approach, mostly for reasons of IDE support and better compile-time checking. The Ruby approach is actually dangerous if your team doesn’t maintain unit tests.

Beowulf wrote on Dec 04, 2005:

First Rest, I enabled anonymous comments, I didn’t realize you had to turn that on. :(

I agree with you that Low hanging fruit is easier to pick, which was exactly why I chose Rails for my system. That doesn’t mean I’ve moved on though, rather it means for this system at this time, I didn’t feel like Lisp was the right choice.

As someone else mentioned, increasing popularity of Ruby is a good thing for Lisp. Anything that gets people looking at languages more powerful than the Java/C/C++ paradigm is a good thing for Lisp.

Eric, my point wasn’t that Lisp should become a niche language. Although in most people’s mind Lisp already is a niche language, it’s the AI language to them. I was suggesting that Lisp needs to find a new place to occupy in people’s minds in order to increase in popularity. The best way to do that imho is to create a shiny new object (ala Rails) for people to play with and promote the heck out of it.

Lol, nice retort with the bitter ex-martyr. It made me laugh if nothing else. :-p.

I’ve followed and have been involved in numerous Lisp is better than language “x” discussions. Everyone that I have seen devolves into the all too familiar battle of syntax. I just think that people won’t switch languages because the solution to summing children is better in Lisp. I think that misses the point entirely. People are after real world, tangible benefits. Little 3 and 5 line snippets of code simply do not show the actual power and expressiveness of Lisp. Most people cannot extrapolate a 3 line snippet into a real system.

I think there is great news that Rails is what is driving adoption of Ruby. It serves to illustrate that people can and will pick up a new language, even an obscure one with relatively wonky syntax.

bill a wrote on Dec 04, 2005:

Beowulf, I don’t think I ever said the solution to summing children was a reason to switch to Lisp. I was countering Eric’s assertion that Ruby is more “functional” and “functionally dense” than Lisp. Of course it “misses the point entirely” if that’s how you’re going to interpret it.

Real-word, tangible benefits? OK, here you are:

- macros give you the power to increase the readablility, concision, and correctness of code. Consider the LOOP and ITERATE macros. Common patterns of looping are expressed with amazing simplicity simply because LOOP and ITERATE can perform code transformations. For instance, let’s say you want to loop over the elements in a sequence and return the one with the highest absolute value:

(iter (for x in seq) (finding x maximizing (abs x)))

I think that’s fancy. And when you consider that the ITER form is going to expand into wicked-fast code at compile-time (and then be compiled to machine code, in some LISP implementations), you realize that you’ve just gotten a powerful abstraction and increased readability for free. No runtime efficiency or spacecompletely is wasted with this construct, because ITERATE has efficient idioms to transform common looping patterns into damn fast code. Directly using the faster code would mean a huge loss in readability and maintainability – with Lisp macros, I don’t have to worry about it. For the curious, the macroexpansion for that ITER form is at the bottom of this common.

For more macros, take a look at Peter Seibel’s book Practical Common Lisp ( available free at http://www.gigamonkeys.com/book). In one of the final chapters he builds a binary file parsing system that’s plain amazing – and he does it with macros. Earlier in the book, he writes a test framework in something like 26 lines – using macros too. Here is a sample of the code used to parse ID3 tags in MP3’s:

(define-tagged-binary-class id3-tag () ((identifier (iso-8859-1-string :length 3)) (major-version u1) (revision u1) (flags u1) (size id3-tag-size)) (:dispatch (ecase major-version (2 ’id3v2.2-tag) (3 ’id3v2.3-tag))))

It specifices the types and order of the data to be read from the file, and then passes control to another binary-class based on the value of major-version – and even automatically throws an error if the major-version is invalid.

- interactive development. If you haven’t used this, you’re missing out. Most languages are about writing lines of code, feeding them to either an interpreter or a compiler, and then watching what happens. In Lisp, development is always done live and interactively. You write functions in files and then send them to a Lisp image. You develop each component of your system one by one and test it live. There is no compile-run-debug cycle – there isn’t even an interpret-debug cycle. It’s more like a write-test cycle.

Interactive development also means you have access to your running system. Not sure what parameters a fucntion takes? Start typing "(func " and the Lisp environment will remind you what arguments can legally follow that. Want information on an object you’re playing with? Type “(inspect x)” and an interactive, recursive inspector will let you walk through it, even making changes as you go.

Once you get your program working, you refactor a little, do some benchmarks, take advantage of Lisp’s completely optional type system to speed up bottlenecks, and sit back and admire your work.

- the condition/restart system. In most languages, there is not much that can be done after an exception is raised. Log it, send it to the user, end the program. In Lisp, an error can be restarted. For instance, in the UncommonWeb framework, when you browse to a URL and an error is signaled, you get the backtrace in your environment. You can now make changes to your code, recompile it, and then invoke the RESTART-REQUEST restart. UCW starts processing your request from the beginning, and the browser gets the result of running your latest, changed code.

- regular syntax. In Lisp, all forms can be broken down into (operator [args]). There is nothing else. This means that I can write macros, but it also means that editors can do really neat things with code, because Lisp is regular. I can move up, down, forward, backward, into, and out of lists with Emacs keychords. This might be difficult to appreciate until you’ve tried it, but I really recommend it. Refactoring code becomes trivial when you can treat control structures as a single entity and cut and paste them.

- almost 50 years of use and development. Lisp has survived longer than every language except FORTRAN. It survived C, C++, Ada, COBOL, and it will survive Java and Ruby. It is, to borrow Paul Graham’s term, “the hundred-year language.”

- more!

All of this said, I ought to admit that I do use Ruby and Rails in all of my professional projects. This is not because I don’t think Lisp is ready, but simply because I hadn’t learned Lisp when I started these projects. I now regret that, and my next web application will be in Lisp.

As promised, here is the crazy but efficient expansion of the ITERATE form above:

(LET* ((#:LIST6 NIL) (X NIL) (#:TEMP7 NIL) (#:RESULT5 NIL) (#:MAX8 NIL) (#:FIRST-TIME9 T)) (BLOCK NIL (TAGBODY (SETQ #:LIST6 SEQ) LOOP-TOP-NIL (IF (ATOM #:LIST6) (GO LOOP-END-NIL)) (SETQ X (CAR #:LIST6)) (SETQ #:LIST6 (CDR #:LIST6)) (SETQ #:TEMP7 (SQRT X)) (COND (#:FIRST-TIME9 (SETQ #:FIRST-TIME9 NIL) (SETQ #:MAX8 #:TEMP7) (SETQ #:RESULT5 X)) (T (COND ([snipped to prevent overflow -ed.]) (T #:RESULT5)))) (GO LOOP-TOP-NIL) LOOP-END-NIL) #:RESULT5))

- Bill (who is heartily enjoying this discussion)

Johnny wrote on Dec 04, 2005:

Ruby is more than just a dynamic or scripting language. But feel free to snob it. More “Rails-like” projects are on the way.

First Rest wrote on Dec 04, 2005:

Lisp lets YOU be Gosling or Matz or Guido. If you want to extend the language, it’s possible:

“DOLIST is similar to Perl’s foreach or Python’s for. Java added a similar kind of loop construct with the “enhanced” for loop in Java 1.5, as part of JSR-201. Notice what a difference macros make. A Lisp programmer who notices a common pattern in their code can write a macro to give themselves a source-level abstraction of that pattern. A Java programmer who notices the same pattern has to convince Sun that this particular abstraction is worth adding to the language. Then Sun has to publish a JSR and convene an industry-wide “expert group” to hash everything out. That process—according to Sun—takes an average of 18 months. After that, the compiler writers all have to go upgrade their compilers to support the new feature. And even once the Java programmer’s favorite compiler supports the new version of Java, they probably still can’t use the new feature until they’re allowed to break source compatibility with older versions of Java. So an annoyance that Common Lisp programmers can resolve for themselves within five minutes plagues Java programmers for years."

—Peter Siebel from Practial Common Lisp

Garrett wrote on Dec 04, 2005:

Thanks Bill. That’s very helpful stuff.

null wrote on Dec 04, 2005:

Some things not mentioned: meta-object protocol; sexpr (code is data, data is code); for commercial lisps, amazing IDEs.

Schemes also have advantages: smaller language – philosophy is move the rest of code to libraries; free near-C fast compilers (Bigloo) (free Schemes have better IDEs than Lisp); possibility prototype algorithms with call/cc and finally translating to efficient iterative algorithms in imperative languages (like C).

A final example: look at the 2 open source Computer Algebra Systems written in Common Lisp available as Open Source: 1) Axiom – originally from IBM Thomas Watson Research Center; 2) Maxima – originally Department of Energy (US).
This is software written in the 70s. You can’t write software that lasts so long with a language that is a moving target. This is the kind of survival and complex domain which shows the power of Lisp.

All these scripting languages are really about people experimenting with language syntax. This is nice, but look at it for what it is. Additionally, almost none of those languages have any theory attached to them. Lisp has (lambda calculus), Smalltalk has (Actors), ML has (type theory). This garantees good, lasting design. Take a look at Perl6 for an example of extreme design experiment. Where it’s going, nobody knows, but – hey! – they’re using Haskell in the implementation (I always thought Perl people were smart).

Lisp was very expensive and demanding of hardware in the 70s and 80s. This is not so anymore. There is no great shiny free Lisp IDE (yet), but at least one vendor sells a good one for a relatively good price. Surely, this can’t keep people from using lisp, since people have taken to relatively “rough” stuff like Perl. By the way, there’s a great Perl book called Higher Order Perl that is really about learning a lot of techniques from Lisp and applying to Perl. I bet a lot of them could be applied to Ruby.

All these scripting languages do is create a problem and sell you a solution. But to be fair, scripting languages empowered programmers like Lisp had not (“batteries included”, etc).

I think the problem with Common Lisp and Smalltalk is the size of the stuff you have to learn, syntax-wise. Some jobs that required scripting languages in the 90s required learning something fast (everything is more complex nowadays – but at least you’ve got frameworks). Also, Perl, Python, etc, were very portable. Syntax-wise, however, I think the approach taken by Scheme and Eiffel are more sensible (there are reasons for the size of Common Lisp as well as Smalltalk – but I believe it is an impediment).

Who knows, maybe people will try Lisp because of Ruby. I bet more people will get frustrated than contribute back. Also try Scheme. It’s a different philosophy and gets even less press than Common Lisp, but it’s worth it.

bill a wrote on Dec 04, 2005:

Boy I wish there was a comment editing facility here. More typos:

“the bottom of this comment” – not “the bottom of this common”
an extra “completely” in “no extra space or time is wasted”
“even allowing you to make changes’ instead of “making changes as you go”

Also, a clearer version of the iterate example would be:

(iter (for x in ’(-1 2 -10 4)) (finding x maximizing (abs x))) ;; => -10

Sorry, guys. I guess these don’t make too much difference, but hopefully this comment will clear up some confusion.

bill a wrote on Dec 04, 2005:

null, that’s good advice but those not familiar with Scheme should remember that Common Lisp and Scheme are very different languages. The superficial similarities in their syntax don’t make that obvious, and so some people perceive them as pretty much interchangeable. I personally do not care for Scheme, and yet Common Lisp is by far my favorite language. So if you’ve had exposure to Scheme and didn’t like it, don’t dismiss Common Lisp out of hand (and vice-versa, too).

I agree that it does take a lot of work and mind-warping to “get” Lisp and I also agree that that’s probably helped to hold it back. I think Peter Seibel’s book Practical Common Lisp (I cannot recommend this book strongly enough!) goes a long way to resolving that.

Johnny wrote on Dec 04, 2005:

You remember what the Japanese car companies did to GM? :-) Big, powerful, shiny, etc, aren’t enough to match simplicity and hardwork. Lisp people ignore 90% of the Ruby features.

bill a wrote on Dec 04, 2005:

Sigh. Johnny, have you even read the discussion? Do you know what’s being discussed, or are you just making emotionally-driven, knee-jerk reactions because the superiority of your favorite language is being called into quexstion? Lisp people are aware of all of Ruby’s features because many of them come from Lisp. No one here is attacking Ruby (at least I’m not). The point of the discussion is simply that Lisp is awesome. Ruby’s geography has nothing to do with its properties as a language, and many of the concepts in Lisp are actually, believe it or not, much simpler than Ruby’s. It’s really not intelligent to make claims about the Lisp community, especially when you quite plainly don’t know Lisp. Also, what exactly is wrong with being a “dynamic language”? You use the term as if it were an insult. What is Ruby if not a dynamic language?

Beowulf wrote on Dec 04, 2005:

Bill, that was a very well written comment. One definite pain point for me has been the lack of macros. There is enough similarity in my comment and scoring system in different areas that I really have wanted a better method for abstraction. Rails has the concept of Partials and Components, which can be used to abstract your design, however they also carry a pretty significant performance overhead. That overhead has forced me to abandon them on some pages because they simply were to slow with good abstractions in place. Several good macros on the other hand would have allowed me an elegant level of abstraction without that same type of performance hit.

Practical Common Lisp (PCL) is definitely an important book for Lisp. It took me a long time to find a good book to learn Lisp from (this was prior to the release of PCL), and I finally had to go back to Lisp by Winston and Horn. A book written in 80’s which is no longer in print. In PCL there is finally a well written introduction to Lisp that is relevant to solving the problems of the Internet age.

I honestly hope Lisp gets an upsurge in popularity. Things seem to be moving more and more in the direction of power and expressiveness. It just appears to be happening at a relatively glacial pace.
Lisp clearly has the power to stay the course though.

As Eric mentioned in the original article though, Ruby really is a good next choice. Paul Graham once said if you can’t use Lisp, use Python. I’ve used Python for a long time now, and I definitely prefer Ruby to Python at this point. As a language it is much closer to Lisp than Python, and it’s OO syntax is much cleaner. That being said, it’s not a bad thing to know all three (Lisp, Ruby, and Python).

Eric wrote on Dec 04, 2005:

Johnny, Bill, please keep it polite. We’re having a good discussion here, and I don’t want to take away anyone’s vowels. :-)

Bill’s point is excellent, and deserves elaboration. I can easily translate his code to Ruby:

class Array
def find_maximizing
result, f_result = nil, nil
each do |x|
f_x = yield x
if f_result.nil? || f_x > f_result
result, f_result = x, f_x
end
end
result
end
end

{|x| x.abs }

But my code is going to run much slower than Bill’s, because the iter macro lives in the compiler. The same goes for the IDv3 parser: It would look fine in Ruby, but get absolutely clobbered on performance.

A year from now, this could actually be a pretty good sales pitch for Common Lisp. “It’s like Ruby, but it runs at full hardware speed!”

Johnny wrote on Dec 04, 2005:

Bill, you are one of the few who claim to be proficient at both, Ruby and Lisp. The problem is that the features that you or other people are going to use in Lisp are just a subset of the features that Ruby is used for. For example, Ruby is very suitable for shell programming, instead of Bash, Sed, Awk, Perl, etc. I just recently started using Rake, and it’s a really awesome tool to use for shell programming. RubyC, mkmf.rb, setup.rb, etc, really make it easy to create extensions for Ruby that bind to external libraries. I don’t think people enjoy creating bindings for Lisp as much as people do in Ruby.

So, if all you want to do is to create WebApps in Lisp, fine. But that use is just a subset of what Ruby is normally used for. I know that Lisp is quite powerful as I said that myself. I know it’s used for tons of different things. But WebApps really are what most people seem to be doing these days. Though with Ruby you can use it for more than just Web site handling.

So, when I claim people don’t know most Ruby features, it’s just because they are spread among many use-cases, libraries, etc, just like in Lisp.

Code RESTful in Ruby!

Eric wrote on Dec 04, 2005:

Null wrote: I think the problem with Common Lisp and Smalltalk is the size of the stuff you have to learn, syntax-wise.

I’ve taught Scheme to a lot of non-programmers over the years. The syntax is problematic, but only until they find a good editor. (Emacs, sadly, doesn’t work for many non-programmers.) The other big impediment is usually LET, which introduces too much nesting for some folks.

The larger obstacle is learning to think in LISP (or SmallTalk, or Haskell, or any other powerful language). Bill can post a dozen excellent macros, and most people will say, “Huh?” Until you know how to use it, it’s just a weird feature.

It’s actually easier to sell a language by solving small problems (my web apps are ugly!) than by proposing profound new features. Ruby gets in the door because (1) Rails is slick, and (2) the syntax doesn’t look too odd. And once people have played with ActiveRecord for a while, they begin to understand why metaprogramming is cool.

Eric wrote on Dec 04, 2005:

Johnny, even though few people write Makefiles or shell scripts in Lisp, it’s certainly possible. Lisp is basically a toolkit for building programs like Rake.

So why don’t you see more great Lisp frameworks? Mostly because (until very recently) the Lisp community was small, fragmented, and didn’t have anything like CPAN or RubyGems. Most of the Good Tricks™ had been written down only once or twice, and the books had gone out of print.

Fortunately, a lot of the missing pieces have appeared in the last five years, and many of the rest will be along soon.

timsuth wrote on Dec 04, 2005:

Eric: ‘def find_maximizing’

Ruby 1.9 has ‘max_by’.

[1, 2, -3].max_by { |x| x.abs }

bill a wrote on Dec 04, 2005:

OK, here’s another example of ITERATE that I hope showcases a little more of macros’ sweetness:

(iter (for x in ’(1 2 3 -4 -10 -43 49 49 8934))

(until (= (sqrt 7) x))
(collect x into collection)
(finding x maximizing (abs x) into y)
(finally (return (list collection y))))
;; ((1 2 3 -4 -10 -43) -43)

This is a lot more complicated than the other example, and still not really useful, but I hope it illustrates how ITERATE provides a whole sublanguage for expressing any kind of iteration you can think of.

I’d also like to point out that ITERATE can be extended with user-defined macros. For instance, I’ve written a macro called AVERAGING that works with ITERATE:

(defmacro averaging (var) “Add a clause to ITERATE to average the value of VAR.” (with-unique-names (count total) `(progn (summing ,var :into ,total) (counting ,var :into ,count) (finally (return (/ ,total ,count)))))) (iter (for x in ’(1 2 3 4 5)) (averaging x)) ;; => 3

There’s more Lisping going on here, and some of this probably looks pretty esoteric (incidentally, this example also shows off Lisp’s docstrings – that string after the defmacro line will be associated with AVERAGING as its documentation, so I can run (describe ‘averaging) and get that string at runtime). But notice that I’m working with an extensible sublanguage for expressing the most common concepts in iteration. The examples Eric and timsuth gave could be defined in any language with first-class functions, but no language but Lisp could define a custom mini-construct like this. Note also that even though ITERATE provides new coding constructs, I still have full access to Lisp within:

(iter (for x initially 0 then (1+ x)) (if (evenp x) (print x)) (when (= x 10) (return x)))

In this (contrived) example, everything except the first line is straight Lisp.

This sort of thing isn’t possible in Ruby. Here’s another example:

(iter (for line in-file “foobaz” using #’read-line)

(collect line))

This will collect every line in file “foobaz” into a list.

(iter (for line in-file “foobaz” using #’read-line)

(collect line at beginning))

will collect the lines in reverse by adding each to the beginning of the result instead of the end.

(iter (for line in-file “foobaz” using #’read-line)

(collect line at beginning result-type ’vector))

will collect the lines into a vector instead of a list.

And so on and so on, ad infinitum (or nauseam, since my comments are starting to get repetitive). The point is that the authors of ITERATE have built a mini-language that is:

able to capture the most common iteration constructs, even relatively complex ones, like maximizing a function and collecting items based on some condition
user-extensible
written in Lisp and not part of the Common Lisp language itself – that is, you or I could sit down and write ITERATE and no one would know it wasn’t part of the language
fast, and yet still high-level

There is much more to macros than just ITERATE, but I think ITEREATE is such a good example of the kind of power that regular syntax and macros can give you. And keep in mind that all this ends up as fast, tight machine code. And if it still isn’t fast enough, throw in some type declarations and optimization settings:

(iter (for x in ’(1 2 3 4 5))

(declare (type fixnum x))
(declare (optimize (speed 3) (safety 0) (debug 0)
(compilation-speed 0)))
(averaging x))

I hope that these comments are useful to people who are curious about Lisp. I wish I’d known about some of these things many years ago.

All code untested.

- Bill

P.S. While I appreciate Eric’s request that we keep it civil here, Johnny, saying that Lisp only has a subset of Ruby’s features is outrageous. It is quite, quite the opposite.

bill a wrote on Dec 04, 2005:

OK, since I’ve spent the majority of my day commenting here, it is time to go do some other things, but I would like to also call attention to the fact that those optimizations in the last example are local only to the block that contains them – I think that’s pretty damn cool.

And just one more example of macros. AllegroCache is a commercial database for Lisp that lets you make queries in Prolog. DISCLAIMER: I do not know Prolog very well, and I remember the examples I’ve seen very vaguely, so this may not be exactly and completely accurate, but it gives you a rough idea:

(<— (parent ?x ?y) (db person ?x children ?c) (member ?y ?c)) (<— (grandparent ?x ?y) (parent ?x ?z) (parent ?z ?y))

Here, the <— macro indicates that the following forms will be in prolog. What we’ve done here is define parent and grandparent relations using Prolog. So we declare roughly how the two relate to each other, and then we can make queries on the database with these.

So say I have a variable x with myself in it. I can then run the Prolog query macro, ?-, like so:

(?- (parent me ?x))

and AllegroCache will return the name of anyone in the database who has me as a member of their children list. I could do the same with grandparents.

Folks, I think this is one of the coolest things I’ve ever seen. Again, these are macros, so all the prolog gets reduced to Lisp code at compile-time, but really have you ever seen anything this cool? Can you picture the hideousness of the SQL that would do this?

Lisp owns.

OK, for real, there are many more things in life than the RandomHacks weblog, and I must go and attend to them. Hope I’ve made a difference.

Johnny wrote on Dec 04, 2005:

Alright, Bill. Sorry about my “car” analogy. Sorry for pretending that Lisp users use a subset of what’s normally available for Ruby. It’s just that the world is too polite sometimes. If you use an “obscure” language and nobody talks about it, you are guilty of not spreading the world. If you use an “obscure” language and people talk about it, you are guilty of “hyping” it. It seems that we can only “hype” it if we present a revolutionary tool like Rails. Fine. Maybe in the future we will have some more of that. Till then, see you.

bill a wrote on Dec 04, 2005:

Nope, there will be one more comment.

First of all, the Prolog clauses do not actually expand into any serious code – they expand into calls to AllegroProlog functions and a little bit of bookkeeping. My bad. I think I gave the impression that they directly become code that is equivalent to the Prolog.

JAP wrote on Dec 04, 2005:

I could start this comment with LOL, because the proposed statement is showing so perfectly the real deep misunderstanding of many of us regarding Lisp:

So let me shortly explain my point of view:

The real reason that brought me to Lisp (and never will get me away from it) is simply this:

You can build it out of 7 (s-e-v-e-n) primitive operators!

And, as a consequence, no other language can be expressed in itself as short as Lisp.
(As you probably know, quite any language can be expressed in itself.)

So let me repeat: really no other language can be expressed in itself that short, and (as a natural consequence) can be built out of less primitive operators.

All the other features of Lisp are only natural consequences of the above.
Because McCarthy did not “create” Lisp (as any other language designer does), he did “discover” it!! (If you read this sentence for the first time, please read it another 2 times).

So the real point is: no other language can be more simple and more flexible than Lisp at the same time. BASTA!

JAP (just another Paul [not Graham])

P.S. For the philosophers of us: we are all together very difficult, and really have a hard time to become more simple for the rest of our lifes ;-)

Johnny wrote on Dec 04, 2005:

One last comment: maybe “Lisp is the most algorithmic language”.

Haskell Junkie wrote on Dec 04, 2005:

Well, macros are fine and dandy, but don’t let the lisp diehards lead you astray. Learning the Haskell language will show you how to do all the DSL and other tricks using just higher order functions. After putting about 3 months effort into learning Haskell, you’ll probably come to see most macro usages as unnecessary hacks. I’d even bet that someone’s already done monads in Ruby. Interesting examples to get you motivated include parser combinators and backtracking.

Eric wrote on Dec 04, 2005:

Bill, if you’re interested in combining Prolog with a Lisp dialect, I’ve heard that The Reasoned Schemer is an excellent introduction to logical programming in Scheme.

Haskell Junkie, thank you for the links to the parser and backtracking papers. Haskell is often unrepresented in these discussions, which is too bad, because it’s a powerful and unique language.

And yes, somebody is working on monads in Ruby.

Once again, a big thank you to everyone for having kept this discussion friendly. Good manners make it easier to have heated, passionate arguments about important stuff. :-)

bill a wrote on Dec 04, 2005:

Eric, I’ve got PAIP here when I finally get around to learning Prolog in Lisp. I was mostly just impressed that it could be used to make complex queries on a database.

Haskell Junkie, can you give some examples?

SB wrote on Dec 04, 2005:

“So let me repeat: really no other language can be expressed in itself that short, and (as a natural consequence) can be built out of less primitive operators.”

Except Forth. Or Factor. Or Joy. Let’s not be ignorant here :)

Ralph wrote on Dec 05, 2005:

As a humble current user of PHP and MySQL – but having been around since FORTRAN and Assembler and happy IBM 1620 Machine Language days – I want to say that I am enjoying the discussion here. In a way it sounds like a conversation among chess players, or artists.

The “coolness” of a language, its elegance, its expressive power, are all important to language designers. I have constructed a few little languages, one of which even went into production in a small way. Recursive descent compiler, byte code interpreter, multi-threading at the application level. I understand the basics.

But most of us, even those of us who do love elegance, nevertheless gravitate toward whatever language environment seems to get the job done with the least total effort. That means common operations must be expressed as easy-to-read idioms.

If you consider the spoken form of natural language for a moment, I think you will find that it is essentially all made out of idiom.

In writing, I can express something like, “The ship did not contain any containers containing containers, but it did contain several containers containing parts used to construct containers” — and here I use a bit of abstraction, made possible by a few minor regularities in the English language.

No one who has been reading this thread is likely to have had any trouble immediately understanding what I just wrote.

But in spoken language, it is quite difficult to express the above both clearly and concisely. Even people frequenting this forum might not be able to express or understand that thought quickly in spoken language unless something of the context had been pre-established. The reason for that, I think, is that there is probably no established idiom for even such a very slightly complex thought. If such thoughts had to be expressed with any frequency, an idiom of some kind would immediately and “spontaneously” arise. Then the above sentence would become trivial to express and to understand.

Syntactic sugar provides idioms for commonly performed actions. Obviously, then, it must also reduce generality. Idiom is supposed to reduce generality.

“Admitting,” then, that one uses Perl, or even (gasp!) PHP, is like admitting that one uses ordinary spoken English. But we all do that.

So, I’m eagerly awaiting Lisp on Rails, and all sorts of other great stuff, plus a hosting company that lets me write my web pages in that great stuff.

Please. Use some of that “hard problem solving” ability to get it all accomplished, and I will be thrilled to be your very first customer. My current provider gets about $20 from me each month, depending on disk space and network bandwidth used.

I am not trying to be facetious, much less rude. I am trying to raise a serious point, even though I did end on a rather familiar, even caustic note, for which I apologize if it jangled anyone’s nerves.

Jules wrote on Dec 05, 2005:

This is a really good discussion! I’m a Ruby programmer, but I want to take a look at lisp. It looks good and I like its simplicity (based on only a few principles). I want to learn lisp because I enjoy programming. Am I right that there are 2 main lisps: Common Lisp and Scheme? Which one should I learn, what are the main differences?

bill a wrote on Dec 05, 2005:

Jules, I would learn Common Lisp, but I’m biased :-). I linked to Practical Common Lisp earlier on. Take a look through the online chapters and see if you like it. That’s a pretty good way to get started.

There’s also Structure and Interpretation of Computer Programs, which is a free book from MIT and a classic in computer science. You can pick up Scheme from it, but you’ll also get a deeper understanding of how Lisp-like languages work.

There is a third Lisp in common use: Emacs Lisp. There are allegedly more lines of Emacs Lisp in use than Scheme and Common Lisp combined. Still, I don’t think I would recommend starting with Emacs Lisp, since it’s sort of quirky and sort of crippled.

Anyway, happy lisping!

Eric wrote on Dec 05, 2005:

Well, let me try to explain the differences between Common Lisp and Scheme, and other people can clarify—and fix my sweeping stereotypes. :-)

Scheme is a very clean dialect of Lisp, with an emphasis on functional programming (see part one, above). One of the most popular implementations is PLT Scheme, which is often used with How to Design Programs.

Common Lisp, on the other hand, tends to have better compilers and commercial support. Common Lisp’s culture embraces a diverse range of programming styles: functional, object-oriented, and imperative styles are all common, as are macros (see part two, above). Common Lisp also has more historical cruft than Scheme. Looking upthread, people seem to like Practical Common Lisp. You can download Common Lisp installers from the same site.

Cultural and implementation differences aside, Common Lisp and Scheme are pretty much interchangable. You can always find a macro package which adds Lisp features to Scheme, or vice versa. Either would be an excellent choice, and both are relevant to Ruby.

nemesis wrote on Dec 05, 2005:

Smalltalking with a Lisp:
“Ruby was inspired mainly by Smalltalk and Lisp”

Don’t forget Perl! Ruby is very perlish…

Tayssir John Gabbour wrote on Dec 05, 2005:

Ralph, well, the tech.coop offers Lisp web hosting, and one of the guys there is working on something he humorously calls “Lisp on Lines.” (LoL.) Though I haven’t heard that it’s ready for common use yet.

But keep in mind that I don’t “advocate” Lisp. At this moment, you will find barriers to entry. And while I think Common Lisp is a highly powerful tool, you need to take Lisp advocates’ words with a grain of salt. They’re good at explaining the advantages of Lisp, but many do not take care to discuss its current downsides. (Like the ones this blog entry discusses.) Therefore they may mislead.

Now, most advocates mislead, but I like to hold CL people to a higher standard. ;)

SB wrote on Dec 05, 2005:

nemesis:

Ruby used to be Perlish, but nobody uses the blatant Perlisms ($_ etc.) anymore…

Jamie wrote on Dec 05, 2005:

"But my code is going to run much slower than Bill's, because the iter macro lives in the compiler. The same goes for the IDv3 parser: It would look fine in Ruby, but get absolutely clobbered on performance.

A year from now, this could actually be a pretty good sales pitch for Common Lisp. "It's like Ruby, but it runs at full hardware speed!"

That is not really an argument for or against either language. It is just saying that Ruby currently lacks a decent compiler. That was true of Lisp, once, and it won’t be true of Ruby, one day. One day Ruby will have a better compiler than Lisp if it stays popular enough to motivate a good commercial compiler to be written.

Most of the arguments in favour of Lisp here seem to be around it’s macro facilities. Yet, any reasonably regular language can have similar facilities – especially if there is defined a normal form which all syntactic sugar reduces. And even better, if the sugar can be used in macro patterns as well as regular code.

The point about Lisp being built out of 7 primitives isn’t really an accurate comparison. Ruby is also only built from a few primitives, if you process the syntactic sugar – which is just the same as using a Lisp reader macro – and in both cases there are a plethora of extra features which you can treat as being built from macro extensions and additional I/O primitives.

I’m looking at all the Lisp examples in this discussion, and I’m seeing the Ruby examples also presented here, and (if you ignore execution time, as that is just compiler maturity), they look roughly equivalent in capabilities, just presented with different sugar – and more sugar with fewer parens in Ruby’s case.

— Jamie (who doesn’t use Lisp much and has never used Ruby, but does know what compilers can do)

Eric wrote on Dec 05, 2005:

Bill, I’m just Haskell newbie, but I’ll take a stab at your question. You can find a lot of Haskell examples in A Gentle Introduction to Haskell, and I’ve had good luck with the Glasgow Haskell Compiler.

From a Lisp perspective, you’ll notice two things: (1) Haskell variables, once set, can never be changed, and (2) because Haskell uses lazy evaluation, you can easily represent infinite data structures.

Lazy evaluation allows you to do all sorts of cool things. You can represent iteration and data traversal in stunningly gorgeous ways. (In effect, everthing’s a co-routine.) You can also, if you’re so inclined, define an infinite list of all primes.

Now, these features are very cool, but they come at a high price: Since you can’t assign to Haskell variables, you have to write everything in a purely functional style. Even I/O can be challenging, because it changes the outside world. So how do you write real programs in Haskell?

As it turns out, Haskell has a second powerful trick: monads. Monads are pretty esoteric, but in Lisp terms, you might think of them as a way to metaprogram control flow. The most common Haskell monad is IO, which makes Haskell statements execute sequentially, so you can talk to the outside world. Other monads implement assignment, list comprehensions, continuations, and domain-specific languages (like the parser combinators above).

Haskell can be a challenging language—some common programming idioms can only be expressed with monads, which are at least as hard as macros—but, like Prolog, Haskell makes certain programs look gorgeous. And of course, there’s no reason why you couldn’t apply many Haskell ideas in Lisp, if that’s what you want.

If you’ve mastered macros, and you’re looking for cool programming styles, Haskell’s not a bad place to start.

bill a wrote on Dec 05, 2005:

Jamie, these features of Lisp have nothing to do with the maturity of Lisp’s compiler. Instead, they have everything to do with Lisp’s regular syntax. It is indeed criminal that Ruby doesn’t even have a bytecode compiler, but this is not really the issue here.

The code in Ruby is much more than “syntactic sugar.” It is a complete formal language that has to be translated into an AST (abstract sy before being used. The following Ruby code:

if foo.bar? and bar.baz? puts foo.to_s end

does not simply expand into other code – it requires a full and complete Ruby parser to be transformed into anything useful. If you still don’t believe me, consider the fact that there can be aribitrary expressions in the if’s test, and that Ruby (like Lisp) treats if’s as expressions, and allows them to be used anywhere. I could write this:

if foo.bar? and bar.baz? and (89 + 44 + 54.0444).abs == 4 and (if x == 4; 56; nil) do_this end

There is no “syntactic sugar” that will simply and straightforwardly translate that into an AST.

Lisp’s syntactic sugar, which I guess you aren’t personally familiar with, is completely different. An example is the #’foo reader macro, which simply expands into (function ’foo). Or the ’foo macro, which expands into (quote foo). Lisp reader macros are merely shortcuts for other Lisp forms. They are syntactic sugar in the strictest sense, and do little more than very straightforward code transformations. As I hope my examples have made clear, this is pretty different from a language with syntax.

I think the phrase “built out of 7 primitives” is misleading (although I think a poster higher up the thread was the first to use it). A better term would be described in 7 primitives. There is code over at paulgraham.com that implements a full Lisp evaluator using only those seven operators. The whole thing is about a printed page long. So the only point that can be made is that Lisp’s semantics can be completely explained with seven primitives. Which is still a cool thing, but not the same as what was being claimed. To build a language you still need a function to take strings and turn them into lists, and other handy things like I/O support and so on. (incidentally, here is Paul Graham’s code: http://lib.store.yahoo.net/lib/paulgraham/jmc.lisp).

If you can point me to Ruby code equivalent to Paul Graham’s, I will concede the argument. But you won’t, and you can’t, because that isn’t how Ruby works. Ruby has a syntax, and Ruby needs a full parser to get from that syntax to an AST. Because Lisp code is naturally an AST, things are very different.

Also, it’s true that if Ruby could be transformed into a more regular language you could write macros with it. This is, in fact, probably possible with Ryan Davis’s ParseTree package. But now your macros have to output (and work from) an intermediate format, instead of straight Ruby code. That intermediate format will be some kind of AST and the mapping between real Ruby and that intermediate form will almost certainly be a hassle to manage, so why not ditch the syntax and use Lisp’s model, where all code is naturally and automatically an AST, and macros simply follow from that?

Haskell Junkie wrote on Dec 05, 2005:

bill,

Here’s an example of a simple arithmetic parser and evaluator using the Parsec library. And below is a little snippet of a non-deterministic program in action. Say you want find a pair of numbers which multiply out to 8633. Well, this isn’t the most efficient way to do it, but it looks nice and declarative…

-- try combinations of factors between 2 and 100 and cull the wrong ones
import Control.Monad
main = print ( do a <- [2..100]
                  b <- [2..100]
                  if a*b == 8633 then return (a,b) else mzero )

…nothing new there, something that could also be done in lisp without macros (especially if your lisp comes with an “amb” operator). But it is nice to note that the list monad which makes this work is implemented in 7 lines of code.

bdw wrote on Dec 05, 2005:

Well, I’ve considered the same thing, really. Ruby isn’t a ‘lisp’ because it is a denser functional language. Ruby isn’t really a functional language; LISP is.

Ruby would have been a ‘new LISP’ (in the same way that ‘yellow is the new black’) if it would have been just a little more consistent. Instead of
loop { obj.meth }
it should have done
loop.obj.meth
which would complete it being Consistently Object Oriented™.

You see, the great thing behind Lisp isn’t the macros, although they are nice.

Instead, there are two real things that Lisp provides, which aren’t found in other language. To sum:

1): In Lisp, everything is a function. ‘If’ is a function, ‘cond’ is a function , ‘defun’ is a function. ‘and’ and ‘or’ are not logical operators, they are functions, too. They return values, as well. With that in mind, it becomes possible to write very powerfull, dense code.

2): Lisp has only one syntax rule (with the exeption of read macros):
(function arguments). Any argument may be a function. This is simple and consistent, and one can rely on it being this way. That’s not to say it isn’t mind-bending and it definitely can drive one mad. But it is the only syntax. If ruby had been just a little bit more consistent, it would truly have been ‘the new LISP’; instead, it is the latest new OO language.

Mind you, Ruby is a very nice language and it has very nice (though imperfect) syntax, community and libraries, and for quick webapps, it definitely rocks.

But it isn’t a new LISP, not by a long shot.

Eric wrote on Dec 05, 2005:

BDW, the operators you list (if, defun, etc.) are technically “special forms,” not functions, because they don’t necessarily evaluate all their arguments. Other than that, your point is good.

Bill, here’s some Ruby code (based on ParseTree and an unpublished library of my own), which isn’t entirely unreasonable:

assert_body(call(lit(2), :+, lit(3))) do
2 + 3
end

The expression in parentheses builds a parse tree, which assert_body compares to the block’s AST. This is a small step towards recreating Paul’s example.

Jamie, speaking as a minor contributor to several Lisp compilers, I doubt that Ruby will ever be as fast as Common Lisp. There’s two big obstacles:

(1) Ruby metaprogramming happens at runtime, but Lisp macros run in the compiler. It’s hard to beat zero runtime cost. :-)

(2) Ruby supports a variety of dynamic features, such as duck typing and method_missing, which are much harder to optimize than the equivalent features in Lisp.

Of course, the best SmallTalk VMs are very good, and a hypothetical Ruby VM could certainly be adequate for almost all applications.

bill a wrote on Dec 05, 2005:

BDW, uh, a lot of the stuff in your comment is confusing and/or wrong. Look into that.

bill a wrote on Dec 05, 2005:

Eric, that’s cool. I figured ParseTree would eventually be able to do something like that. Still, I think that even writing macros in an intermediate format is a lot less convenient and maintainable than Lisp macros – what happens when the interface to ParseTree changes? What happens when a new version of Ruby comes out and breaks everything? I’m still sticking with Lisp. :)

PleaseDontKillMe wrote on Dec 05, 2005:

NOTE: I originally posted this on Zifus blog up there. Posting here for completion’s sake. You may accuse me of trolling, or you may read this document as something which has a degree of truth to it.

[Snipped, on grounds of excessive profanity. Please keep it polite. -Ed

You can find the original comment on the Zifus blog, starting with “I think most ‘dead’ or ‘dying’ languages…”]

bill a wrote on Dec 05, 2005:

OK then. Winter break starts in two weeks. I will write, document, and release Lisp on a Ladder.

I promise.

Will that satisfy the naysayers?

Eric wrote on Dec 05, 2005:

Bill, if the interface to ParseTree changes, I’ll just hack around inside my library until the test cases run again. The code above is actually written against my own high-level API, not against ParseTree itself.

The real danger is a VM change in Ruby 2.0, which would pretty much kill ParseTree. I suppose if somebody builds something stunningly cool right now, it might help keep ParseTree alive on a future VM. Not a good argument for a production system. :-/

Johnny wrote on Dec 05, 2005:

(I know, I know…)

I agree, my “PleaseDontKillMe” friend. Programming is a means, not an end in itself. Refurbishing your algorithms aren’t the purpose of programming. You need all the help that you can get, free or commercial, because programming is hard! Inviting people to the “hard side of the force” won’t work, ever.

Mike wrote on Dec 06, 2005:

You mention python in your article, but then you move to ruby. Ruby is a great language.

I started in LISP in the late 80’s then went to python in the early 90’s.

Python has always had native rich functional support.

- lambda
- map
- apply

Python 2.2 (release python now 2.4)
added
– iterators

Python 2.4 added decorators that simplify generic programming.

Here are some intersting python as LISP

Nice article on LISP and Python comparison

http://www.norvig.com/python-lisp.html

Functional Programming in Python
http://www-128.ibm.com/developerworks/linux/library/l-prog.html

http://www-128.ibm.com/developerworks/linux/library/l-prog2.html

MultiMethods:
http://www-128.ibm.com/developerworks/linux/library/l-pydisp.html

Generic Programming:
http://www-128.ibm.com/developerworks/library/l-cppeak2/

Eric wrote on Dec 06, 2005:

Thank you for the decorators link; that’s an especially nice approach. Ruby handles this a bit less gracefully.

For more on lambda and map in Python, and why they’re not quite as helpful as they might be, see the earlier discussion.

Brad wrote on Dec 06, 2005:

Having read about 1/2 through the book, what strikes me is that Ruby seems to be roughly the same language as smalltalk (caveat, i don’t know the meta-programming layer fo Ruby) with a lot of extra syntax.

As I read the Ruby code i see how with slight tweaks to turn it into smalltalk…which also looks a lot more elegant to me.

PleaseDontKillMe wrote on Dec 06, 2005:

Eric, I’m dissappointed. You could have just deleted the offensive words because the intent of the post was not to troll.

But this is your blog and that is fine.

Even though, there were some uncomfortable things that I said and I hope it wasn’t because of those uncomfortable truths (and I don’t meand the “F” words) that you decided to pull the comment whole hog.

this also illustrates one of the points I tried to make. An un-necessarily haughty attitude when it comes to ideas which challenge the status-quo in our mind.

My apologies for “excessive profanity”.

What I find even more “profanerer” is all the talk about the beauty of our computational navels, while supposedly “inferior” languages whiz by us in the race for survival (essentially mindshare)

You may be the sleekest species for a hundred million years, but if you can’t compete with the humble shrew, then YOU are the eventual loser. I don’t care how many shiny scales and sharp fangs you got!

Languages mold their speakers, and the speakers mold their languages. Latin and Sanskrit maybe the ‘best’ languages, but I wonder why the world don’t use them no-mo ??

My question to the Lispers/Forthers/Smalltakers was:

Do you want to be the Sanskrit of the world, spoken only by the high-priests and teetring always on the brink of extinction owing your existence to the continued existence of the high-priests?

or do you want to be the lingua franca, (lets say, English) of the world? Spoken by the masses to make their lives happen on a day to day basis?

The choice is yours, and if you “love” the language, then you will do things which are not short-sighted in nature (closed source/obfuscation/needless punditry) and which help the cause of your so called ‘love’ (open sourcing, helping, creation of killer apps for mass penetration)

By killer apps I don’t mean other dialects of the favorite language (or extremely expensive retail software)

In cany case, thanks for starting a thought provoking post.

::AAEBACMD::

Jim wrote on Dec 07, 2005:

bill a,

Please make Lisp on a Ladder.

Can you make it cross-platform? Will it run on more than one compiler? There’s a lot of fragmentation causing duplication of effort. What do we need to make a beautiful language like Lisp gain mindshare?

The Lisp community needs more development and collaboration. It needs some more exciting things going on!

bill a wrote on Dec 07, 2005:

Jim: Yes, it would have to be as cross-platform and cross-implementation as humanly possible. I will be sure to test it on CLISP, since anything that runs on CLISP will run on Linux, FreeBSD, OS X, Windows, etc.

I’m also planning to release tutorial videos as part of the project (as soon as xvidcap stops crashing when I hit record) on a whole variety of topics, from the basics of Lisp, to using Emacs + SLIME, to the specifics of the Lisp on a Ladder framework, to developing a toy application (let’s say a blog) in some ridiculous amount of time – let’s say four minutes.

While working on this, I’ve realized the answer to a challenge lots of people make: why is there no equivalent to Ruby on Rails in Lisp? I think the answer is that most frameworks exist to save their user the repetition and messiness of typing out similar constructs over and over. With macros, that need is less acute. With environments like SLIME giving you information about what parameters a macro or function takes, this is also less important.

In any case, here are some of the features I’m considering in LoaL:

- at the user’s option, LoaL will automatically manage your database schema (this can be turned on or off on either a database-wide or a table-wide basis). So if I make a change to my model code, LoaL would check to make sure the schema matches what’s actually in the table. If it doesn’t, and the user isn’t in production mode, it will offer to bring the schema into line with the model, being careful to warn about dropping columns with data in them and so on.

- compiled view templates, so minimal time is spent building a view to be sent to the user

- instead of shoehorning things into the object-oriented model, LoaL will be able to define language constructs for specific tasks. In RoR, for instance, a method of the controller is automatically available to the public unless I make it private; LoaL will use a (defaction …) form that will make this a non-issue – things are only actions if you make them actions, because actions and methods are different concepts.

- less stub generation than Rails. There will probably be some kind of stub generation available, but ideally it would only be to create the application directory and one or two files. With interactive development, figuring out what goes in a test file isn’t as difficult because the documentation for every macro and function is available at development-time; with macros, stubs aren’t needed as much because the user will be able to say things like (deftestsuite (deftestcase ……) ….) instead of relying on the framework to properly generate a test file.

- in development mode (at least), LoaL will ideally give you more certainty about your code – making sure you don’t render nonexistent views or redirect to nonexistent actions; making sure you don’t refer to nonexistent columns in the table. There will be zero cost to this, because it will all be done at compile-time/load-time. (I haven’t completely decided whether this is worthwhile or not).

- probably some more

LoaL is inspired by Rails but it is neither a clone nor totally an attempt to redesign it. It will be more for public-relations reasons than anything else, since there are certainly frameworks coming that will do things better than LoaL will.

Here is a darcs repository with an early, early, EARLY draft (DRAFT!) of some of the constructs LoaL MIGHT offer: loal repo . The file driver.lisp can be viewed in a browser and gives you an idea of what MIGHT (have I stressed the tentative nature of this enough yet? No? OK, well, driver.lisp is the result of only about an hour of thinking, so I’m sure you can see that it will almost certainly change in drastic ways). There are the beginnings of a portal at loal-web

Friendlier URL’s coming soon! Also, any tips on doing narrated video captures in X (on Linux) would be appreciated, since I can’t get xvidcap to stop hating me.

JAP wrote on Dec 07, 2005:

(for the sake of completeness):

SB: thank you for correcting my ignorance, but here I would like to bring a statement of Albert Einstein: "make everything as simple as possible, but not simpler!

That says it all: when you go too far, then you loose something (maybe important), and here it would be: flexibility.

So, you really can create some language that seems simpler than Lisp, but then at the same time you loose really too much!

(Isn’t that really interesting? You can create a language out of, say, 25 primitive operators, and have something very handy, and you can create a language out of, say, 3 primitive operators, and you have something as-concise-as-no-one-could-ever-beat, but hey, you have lost all of the flexibility, and never will be able to convince the really smart people of this planet!)

I would summarize it this way: the really “perfect formula” for making a programming language is to bring it to exactly that 7 primitive operators that are forming Lisp.
Then you have “touched heaven”! (Please don’t take this too literally, I know that in practice this has to be extended, but not in spirit. Anyway, that doesn’t really touch the theory).

Any comments, anybody?

-JAP

Jules wrote on Jan 23, 2006:

There is a language called IO:

http://www.guldheden.com/~sandin/amalthea.html

It is a very simple language, the only thing is continuations. You can build pairs with continuations, and thus lists, integers, strings, trees.

But it is really hard to understand. This:

write 5;
write 6;
terminate

Does a lot of things. It first calls “write” with TWO arguments: 5 and “write 6; terminate”. This prints it’s first argument, and then executes the second argument (the rest of the program). write 6 does the same: print 6 to the screen, and call “terminate”. Terminate doesn’t call the rest of the program, so the program will terminate.

Integers are continuations too:

1; → a;
write a;
terminate

1 is a “function” It calls the rest of the program with itself as an argument. The → thing assigns the argument to a, and calls the rest of the program.

So you can express everything as continuations. It’s just nice to have these built-in integers and strings to make your programs cleaner, but they are not needed.

So this language has only one concept: continuations. You can build for-loops with it, goto’s, etc. IO is VERY flexible.

But does this make the language “better”? I find it very hard to understand, but function calls and for loops are hard to understand too (if you are not a programmer). With some practice, first-class continuations are very powerful tools, it’s the only thing you need.

I think that every Lisp programmer should check this out. Hard to understand, but very powerfull, and you don’t need macros, because you can express everything concise with continuations.

Rhodry wrote on Feb 03, 2006:

Hey people,

Interesting discussion. As a Ruby / Rails afficionado its been quite interesting to watch. But as a newbie “Pragmatic Programmer” I am trying to learn a new language each year. After the discussions above, 2006 may just be the “Common Lisp” year :)

As a vaguely offtopic curiosity for the veteran Lisp hackers in the list …

Has anyone looked at “Intentional Programming”? It was originally (10 years ago) a Microsoft Research project for interactive graphical generation / transformation and reduction of Abstract Syntax Trees [AST] – recently it has been taken into other commercial hands who are trying (but are always delaying) to realize a commercial solution http://www.intentsoft.com/. The original hype sounded brilliant – a graphically rich domain specific language integrated development environment [IDE] – but nothing has yet come of it. (Unlike Rails )

Now, given the vaunted regularity of syntax and metaprogramming capabilities of Lisp (not to mention the existent IDE features of Emacs), surely such a wide-scale, platform-neutral, domain specific graphical visualiser and metaprogrammer environment would fit better within a Lisp variant than Microsoft C/C++ [the original technology bootstrap].

Anyone got any ideas as to why
a) No Lisp hacker group hasn’t cottoned on to the ideas in the published research and given the construction of an Intentional Programming environment a go?
b) Anyone would ever think that C/C++ would be an easier way to reason about the computation/compilation process than Lisp?

Wouldn’t it be good to be able to simply activate an edition enzyme in your IDE so that your new source package was instantly re-represented as Ruby or Python or even schematic diagrams? Whatever language is closer to your thoughts… and better fits the solution :) {Might even reduce some of the language holy wars :)}

Hoping to stimulate some thoughts…

Rhodry

Tayssir John Gabbour wrote on Feb 17, 2006:

Not only is the Lisp world aware of Intentional Programming, Interlisp was likely the forerunner — and a well-known Lisper 1 was the cofounder of Simonyi’s intentional programming company.
http://wiki.alu.org/AudioVideo

Kiczales, a coauthor of Lisp’s metaobject protocol. Last I heard, he departed Intentsoft.

Rhodry wrote on Mar 02, 2006:

I humbly withdraw my misguided implication that the Lisp programming community were not involved in intentional software development and apologise for any offence….

Rhodry

Eleanor McHugh wrote on Apr 13, 2006:

As an academically savvy Ruby programmer who works on real-world application development I very much enjoyed reading your article, but I find the religious divide that’s apparent in various of the reactions to it rather strange.

I came to Ruby via a circuitous route and my sole concern in using it is whether or not I can concentrate on the abstractions necessary to deliver a maintainable application efficiently and in a timely fashion: this is an area in which Ruby shines. It may have syntax, but it shares this feature with human language and most communications protocols – unsurprisingly really as at the end of the day both source code and real-world computational side-effects are intended for a human audience.

Lisp has in part failed to gain mindshare with the broader development community because ultimately it is a mathematical abstraction and (whilst rare amongst classic hackers) most people are perturbed by mathematics. It is an interesting notation if one wishes to develop software as a symbolic representation of a computational parse tree but that is only one theoretical model of the programming process.

Ruby on the other hand is an example of a class of languages that will become more prevalent in future years, where the abstractions that make Lisp so powerful are presented in a form that is palatable to those of us with an artistic/lingustic bias as opposed to a mathematical one.

This is not a new development. It seems to me that Ruby shares much in common with Icon and Pop-II (just as PERL does with SNOBOL), both languages that have found adherents outside the computer science fraternity despite their strong theoretical foundations. Where Ruby differs is that it has a killer application – the Internet. Anyone who has looked at the Rails web application framework can instantly see the potential compared to the slew of ad hoc scripting languages against which it is currently competing, and most of that power comes from the relative ease with which Ruby handles meta-programming.

Ruby makes object orientation, functional meta-programming painless for the non-technical user and introduces elegance and beauty into tasks which whilst achieveable in many apparently equivalent languages would require inelegant hacks. In my opinion this is a very good thing because elegant software is in general easier to maintain – something which I’m sure the hardcore of Lisp hackers would recognise.

For me it’s not really a case of Lisp or Ruby being the more powerful language in an absolute sense, but of which is the more powerful tool in the hands of a particular developer looking to use the highest levels of abstraction possible with the minimum level of theoretical knowledge. As an amathematical person I consider Ruby more powerful for me as an individual, even though that may not be the case for many previous commentators.

Given the ease with which a non-technical user can gain a good grasp of Ruby I suspect that it has the potential to be to network programming what BASIC was to the microcomputer revolution of the 1970’s and 1980’s. The speed with which Ruby on Rails is taking off in the web development arena is just the tip of this particular iceberg.

So to summarise: Ruby is a powerful language with a syntax that is easy even for non-programmers to get to grips with. It includes concepts to do with block closures, object-orientation and meta-programming which are usually absent from languages accessible to novice programmers and provides idioms for their usage which are easy to remember and use. There are without doubt architectural techniques which can be utilised more easily in Lisp, but as to whether or not those are techniques which are actively useful for general software development is something that I think is open to debate.

Eleanor

Daniel wrote on Apr 24, 2006:

Eleanor, excellent comment. In response, I offer a Dijkstra quote:

Don’t blame me for the fact that competent programming, as I view it as an intellectual possibility, will be too difficult for ‘the average programmer’, you must not fall into the trap of rejecting a surgical technique because it is beyond the capabilities of the barber in his shop around the corner.

cmy wrote on May 01, 2006:

Eleanor, excellent comment. In response, I offer a Dijkstra quote…

While I certainly hold Dijkstra in the highest respect, you have to rememebr that this is the same guy that counts GOTO statements and hand-writes his paper submissions.

While it’d be nice to adhere to ideals, some of us have to live in reality.

Alexander Medvedev wrote on May 17, 2006:

Yeah, Ruby example is more straightforward, compared to the Lisp’s one that you give:

(remove-if (lambda (n) (= (mod n 3) 1))

(mapcar (lambda (n) (* n n))

’(1 2 3)))

BUT. More can be said in Lisp’s defence than you say. Macros can be used not just to make lambdas more consize. If you consider the above expression hard to read, and prefer to read in the order of computation, as in your Ruby example, take a look at this:

(stream-2nd-arg ’(1 2 3)

(mapcar (lambda (n) (* n n)))

(remove-if (lambda (n) (= (mod n 3) 1))))

Note that mapcar and remove-if have no 2nd argument. Where do they get it from? From the previous computation. The first arg to ‘stream-2nd-arg’ macro is evaluated first. Other args are function calls evaluated after that in turn, and each gets their 2nd argument from the previous call.

Simple, concise and readable too.

The implementation of stream-2nd-arg macro is left as an excersize to the reader :)

John Collins wrote on May 17, 2006:

I am so glad I found a guru versed in all 3 of: Ruby, LISP, and functional programming.

In LISP and Python, I can use the apply function to pass a parameter list to a function declared to accept those parameters and bind them directly to each of the formal parameters in the declaration of the called function.

In Ruby, sure, I can pass a list of parameters as a single list to a *parameter. But that is not what I want to do.

Any idea of how to carry this off in Ruby, or is a close analog to the apply function just missing from Ruby?

Eric wrote on May 21, 2006:

Try this:

obj.meth arg1, arg2, *more_args

It’s roughly equivalent to Scheme’s “apply.”

Notav A. wrote on Jun 13, 2006:

(apply (lambda (a b) (+ a b)) ’(1 2))

can be done with:

lambda{|a,b| a+b}.call([*1,2])

someone wrote on Jun 24, 2006:

Do Lisp coders write programs or do they just sit around talking about their language?

It seems to be in use deep within the halls of CS departments and AI research, but in years and years of working with software, managing systems, writing for more OSes and problem domains than I can keep track of, I haven’t encountered a scrap of Lisp code or even a similar project to my own in which someone was using Lisp. I don’t mean to flame, but I’ve seen roughly a thousand times more talking about Lisp than the using of it.

I’ve always meant to learn Lisp, but I get the sense that I’ll be applying the knowledge more on usenet than on actual programs.

I’ve recently discovered Ruby, and of all its amazing strengths ( I’ll put at the top of the list — at least as far as something that you’ll probably see on any open vim session on my workstation these days — is the enumerable module ) nothing quite matches the friendliness and helpfulness of its community. Ruby seems to be the first language to really get that languages are software and programmers are users.

Programming communities need collective nouns. An obfuscation of Perl hackers. A stalwart of C coders. A furrow of assembly wranglers. An effusing of Ruby hackers. An ulcer of CSS authors. A grumbling of Lisp coders.

Eric wrote on Jun 26, 2006:

I suppose it depends on your geographic location: I’ve spent at least 6 of the last 10 years working with Lisp and Scheme on both academic and commercial projects. But then again, I like to work on cool, offbeat stuff, and I live in the greater Boston area—two factors which, together, correlate highly with Lisp.

If you’ve been working in AI, and you’ve never seen Lisp, the odds are extremely good that your work started after the late 80s. That’s when a combination of Moore’s Law and gross business incompetence killed off half the Lisp companies.

Nonetheless, you should probably learn Lisp if you want to take full advantage of Ruby: The book Practical Common Lisp contains 10 times as much meta-programming advice as all the Ruby documentation I’ve seen, combined.

William James wrote on Aug 25, 2006:

Here are some Ruby equivalents of some Lisp examples:

(iter (for x in '(-1 2 -10 4))
      (finding x maximizing (abs x)))



[-1, 2, -10, 4].inject{|a,b| a.abs > b.abs ? a : b}
# or
[-1, 2, -10, 4].sort_by{|n| n.abs}.last



(iter (for x in '(1 2 3 -4 -10 -43 49 49 8934))
         (until (= (sqrt 7) x))
         (collect x into collection)
         (finding x maximizing (abs x) into y)
         (finally (return (list collection y))))

(By the way, I don’t think this Lisp will work as advertised. (= (sqrt 7) x) seems wrong.)

def maxim a,b; (yield a) > (yield b) ? a : b; end
y = 0
[1,2,3,-4,-10,-43,49,49,8934].inject([]){|a,n|
  break [a,y]  if n==49
  y = maxim(y,n){|x| x.abs}
  a << n }

William James wrote on Aug 26, 2006:

This version will work properly even if the list doesn’t contain 49; also, the variable y has been eliminated.

def maxi a,b; (yield a) > (yield b) ? a : b; end  

[1,2,3,-4,-10,-43,49,49,8934].inject([[],0]){|a,n|  

  break a  if n==49  

  a[1][59] = maxi(a[1][59],n){|x| x.abs}  

  a[0][61] << n ; a }

Miquel wrote on Sep 07, 2006:

I have to say that I work with lisp everyday and it is very powerful. I just haven’t got the motivation to make a decent interface to mysql for my web applications. That is why I chose Rails.

Oh Well..

However, when the db support arrives, I am going back.

Pebblestone wrote on Sep 08, 2006:

Miquel:

Try CLSQL, it supports multiple DBs and on multiple lisp implementations. And I just used it in my last project.

KristofU wrote on Sep 26, 2006:

I’m a C++ programmer, and was looking for a new language to learn and do stuff more elegantly.
Ruby just sucked me right in. There is almost nothing to learn, it’s just there.
You can slice and dice and juggle and the result is always a working program.
You like for-statements? Well, you can use for-statements. You like iterators? Well, you can use iterators. Make object functors or lambda’s, whichever you prefer. Ruby doesn’t force anything upon you, there is time to learn to appreciate the finer features, while still churning out working apps in the meantime.
I’ve also briefly tried Haskell, Scheme and Erlang, but I have to say, I couldn’t get anything done. Compared to Ruby, quite an anti-climax.

taw wrote on Oct 01, 2006:

You might be interested in RLisp, which is Lisp directly connected to Ruby runtime and very tightly integrated with Ruby.

It is very alpha, but can do really cool stuff like HTTP server that uses webrick (from standard Ruby library) and macros.

I think even at this early stage it is probably one of the most practical Lisps for scripting-style of programs, due to access to all Ruby libraries.

The biggest problem right now seems to be culture shock, as RLisp has Ruby/Python-style let (this is the most controversial part, I don’t have any idea why), a “real” object system (based on message passing and everything-is-a-Ruby-object), and lists are arrays (so cdr/cons copy).

Enjoy :-)

Simplicus wrote on Oct 03, 2006:

Why compare Ruby with Lisp and not with Erlang, Haskell or ML or other FP languages?

I find this discussion very fruitful in many aspects. Yet the most interesting part of it for me personally is to what extent Ruby is a functional language? Or using Eric terminology how "functionally dense" Ruby really is?

I have no doubts that Lisp and Scheme in particular can be used for functional programming (FP). Nevertheless both of these languages can be used equally well for non-functional programming also.

Now back to my question:
Why compare Ruby with Lisp and not with Erlang, Haskell or ML, or other "true" (Haskell) or "more functional" (Erlang, ML) FP languages?

I understand that from the above mentioned FP languages only Erlang has no strict types and two others (Haskell and ML) are strict. Nevertheless, I think that we can still compare "functional density" of those with Ruby's.

I have 20+ years of programming experience, starting with DEC PDP-11 Macro Assembler and most of later work in OOP, such as C+ and then Java which I started to play with its first alpha release. That time I liked Java a lot compared to C++. During that "old times" I had also a chance to get to know CLOS and was charmed with its powerful ideas (lists = code = data, generic functions, MOP, to name a few), uniformity and simplicity. Unfortunately, that time (and today as well) I couldn't find a way to do my living working on Lisp or Scheme projects, though I did some work in AI related areas (and learned many other "small" and "big" languages including Prolog) .

About three years ago I started to research FP quite seriously, trying to find programming tools (languages) more powerful then Java and other OOP languages that I could use to implement some of the ideas waiting in silo quite for a long time already.

I started with Scheme, then ML and now Haskell which I like most for its clarity and power. In between I am looking at Erlang and Ruby. Returning back to "FP density" Scheme and Lisp lack some powerful constructs that ML and Haskell have and that I find very important for uniformity, clarity, readability and expressiveness of the language. (Mind you, I love them all Haskell, Scheme and Lisp :)

These things are:

Guards (Haskell)
Pattern matching (Haskell, ML, Erlang)
List comprehensions (Haskell)

Guards example:

max :: Int -> Int -> Int
max x y
	| x >=y 	= x
	| otherwise	= y

Example of both pattern matching and comprehensions – quick sort in Haskell is simple as that:

qSort :: [Int] -> [Int] 
qSort [] = []
qSort (x:xs) = 
  qSort [ y | y <-xs, y<=x] ++ [x] ++ qSort [ y | y<-xs, y > x]

This code is very close to informal definition of quick sort:

"To sort the list take off the head '(x:xs)' and then split the result in two parts: the first containing the elements no larger then 'x' and the second exceeding 'x'. Sort these two parts and then concatenate 'x' with results of these two sorts"

Isn' it clarity and readability?

Isn't purely functional notation is better then a notaion produced by macros?

What Ruby gives as equivalent to Guards, Pattern matching and List comprehensions?

I am not even talking about extremely powerful strategy of lazy evaluation that allows Haskell only evaluate a function argument when it is really needed (on demand) to compute the overall result. This in turn makes computations less expensive and allows language to describe infinite data structures.

As almost everything else, lazy evaluation can be implemented in Lisp and Scheme. Any ideas how to do this in Ruby?

After Haskell, Ruby syntax looks too eclectic for me (foo, @bar, baz :faz => xxx), 'blocks' and 'yields' makes me feel sorry for functions as first class objects, "meta programming" seems for me to be a perfect tool to hide and obscure what is really going on behind the code I see right now (because methods can be added to classes in other source files), etc…

Maybe I am totally wrong about Ruby and will drastically change my opinion after more experience with it, we'll see.

As for this discussion it would be interesting to know what you guys think about:

The question I started this message with
Why Haskell (and FP in general) is not as widespread as Java, Python & Ruby? And all these sad things are happening in spite of:

a. Though not so big but very strong and dedicated community of Haskell developers.

Constantly growing set of open source Haskell applications, tools and libraries.

schemer wrote on Oct 19, 2006:

Simplicus, I would agree that pattern matching and guards are awesome, even in the absense of types.

I mainly code in Scheme, but I was absolutely blown away by Monads and lazy evaluation when I tried Haskell. I can’t belive most programmers consider those aspects of Haskell a turn off, both are techniques which I was earlier trying to reinvent while programming in Scheme while not even realizing it.

However, from personal experience, I would have to note that there is no reason why scheme can’t have those features; (There are libraries for pattern matching as you presumed, and there are

As for what I think makes scheme less “functionally dense” than Haskell are:

Scheme’s mutable define, mutable cons, and non-imperative i/o are what make it less functional than Haskell. Of course, these features that I mentioned aren’t really fundamental properties of the language, both in my opinion are historical cruft that I hope someday will be fixed…

On the other hand, I’d love to see a sexpr Haskell. :)

As for ruby, I don’t consider it to be “functional,” (although I wouldn’t be surpised if ruby programmers were fond of hofs) because as far as I know, it doesn’t have tail-call optimization. With Common Lisp, even though it doesn’t have tco, every major compiler out there (CMUCL, SBCL, probably also CLisp) optimizes tail calls, so I have no particular problem with CL, assuming a very non-idiomatic style to CL.

TCO is a big deal for me, because in Elisp I have to use destructive maps/filters to avoid exceeding the functional call depth, that’s really annoying…

schemer wrote on Oct 19, 2006:

Argh. I made some bad typos to my post: I meant lack of non-imperative i/o in my prior statement.

Also, I forgot to note several prominent scheme hackers have implemented monads in scheme, though unfortunately there’s no standardized example…

Lazy evaluation in scheme is also doable as you mentioned, even though I haven’t yet gottten the srfi for stream-define and friends working properly (apparently the my implementations’ modules are not compatable with it without extra macros)

I also don’t grasp why the srfi for memoized streams uses set! when it could just use a helper function (but then again, I would say that this is a problem of mutable define in scheme)

William James wrote on Oct 24, 2006:

Quicksort in Ruby:

def qsort ary  

  return ary  if ary == []  

  pivot = ary[0][61]  

  left, right = ary[1..-1].partition{|x| x < pivot }  

  qsort( left ) + [pivot] + qsort( right )  

end

memeplex wrote on Oct 24, 2006:

Lack of community and codebase are strong impediments for every but a few language. Also most new languages are missing a standard ffi which let them easily access c/c++ legacy code, preferable via bindings autogenerated by tools like swig. In this regard, Lisp has cffi, which is a good thing. But anyway, you won’t get too far away without people doing the heavy lifting, all that enormous amount of binding code should be generated, patched and tested. It seems like the way to go for small communities is to parasite mature and well established runtimes, and to compile to their intermediate binary representation. Of course I’m talking of java and .net/mono/pnet here. They’re doing their thing pretty well, performance is more than acceptable, and taken as intermediate languages (which msil is more than java bytecode) they can even be compiled to native code (take for example gcj). I love scheme programming but I’m also fond of new oo+fp languages, namely scala or nemerle. Because of the reflection capabilities of this cross-language/cross-platform runtimes, binding is far easier than it is for C/C++ native stuff. Sometimes it’s just a matter of importing modules and start using them. Admittedly, a generic runtime it’s not the same than a standard library tailored to an specific language. But then those standard libraries could be build faster on top of the facilities offered by the runtime, and in case they are still not complete, one can always resort to the generic api, it’s not a cul-de-sac. Also you can easily bind to c/c++ if you really need it: there are mature swig modules for java and c#, while there aren’t others for a lot of cool, tiny languages. I’m currently programming in bigloo scheme targeting the jvm, and in scala too (they both are able to generate code for the clr also, although support isn’t that mature already). I think it’s the way to go for languages with small communities. And in any case, these intermediate platforms promote code reuse, offer advanced interpretation/compilation techniques (while there could be lack of support for specific techniques, like tail-recursion, that’s a price to pay), powerful development environments, etc etc. Just how I’m feeling like lately.

Eric Kidd wrote on Oct 26, 2006:

Oh, I’m a huge fan of Haskell. The syntax is nice and dense (though occasionally a bit alien), and monads are a remarkably powerful abstraction.

There’s a whole class of designs which can be easily expressed with monads (unification, backtracking, transactions, continuations-as-a-library, localized state, modular parsers, and so on). A lot of these designs can’t be implemented cleanly with Lisp macros (unless you write a code-walker, or an entire embedded language).

It also seems that monads are most useful when you can have more than one of them, and when you can overload your monadic operations depending on the return type of your function. This is the biggest barrier I’ve found to using monadic code in Scheme.

Haskell is a fairly hard programming language, and I doubt it will ever go mainstream. But it’s by far one of the most interesting languages on the scene, and I learn a lot programming in it. My biggest complaint is the lack of subtyping, which is fixed by O’Haskell.

If I had to recommend four languages to learn, I’d recommend Ruby (because it’s so fun), Lisp/Scheme (for the macros), Oz (for unification, concurrent logic and constraints) and Haskell (for monads and for the deeply mathematical style).

scripting geek wrote on Dec 13, 2006:

Very interesting and enjoyable discussion!

I did LISP and Scheme for a few years in the early 90s, doing AI at Berkeley. Looking back over that code some years later, just for fun, I was struck by the total impenetrability of my earlier LISP excretions. The resulting tangle of sexp was so nasty, I would have opted for ritual seppuku if required to decipher the stuff. But that’s just me and my substandard LISP code.

On another tangent.. I’m finding the impassioned arguments about language properties very interesting, but ultimately somewhat irrelevant, at least insofar as productivity is concerned.

I think that nowadays the breadth and quality of the available open source libraries is a much larger factor in the appeal of a language as a useful tool than are the theoretical upper limits of its expressive capability.

Learning to live with somewhat clunky metaprogramming facilities as opposed to super-powerful macros is infinitely easier, I think, than lacking libraries for half the stuff you need done on a daily basis.

It sounds as though LISP is still far behind the Python/Ruby camps insofar as that’s concerned, and for this reason alone I wouldn’t give it another look at this time. And, at least in my case, I learned that power of program-as-AST that’s so nice for computers is not so good for human comprehension, as we do not make good recursive tree evaluators. Beyond a modest level of nesting and layering, things start to look very nasty indeed.

Python/Ruby on the other hand are very readable, with plenty of useful idioms captured by grammar or sugar, and have APIs to easily interact with just about everything. With tools such as these, I alone can replace a small department of C programmers, and that’s very appealing.

Chris Rathman wrote on Jan 14, 2007:

Since the topic is a comparison between Ruby and Lisp (and Simplicus expressed an interest in comparing Ruby with Haskell or ML or other FP languages), I thought I’d mention my pet project to translate the SICP examples into these languages.

http://www.codepoetics.com/wiki/index.php?title=Topics:SICP_in_other_languages

I’ve still got a lot to learn about Ruby, but in translating the first chapter and a half the main quirks I had with Ruby were (1) it’s lack of Tail Call Optimization – an attibute shared with Python and JavaScript (many in the Lisp community consider the lack of TCO to be a bug); and (2) the distinction between named and anonymous functions. As far as I can tell, you can curry lambdas, but you can’t curry named functions – Erlang works this way as well.

gar37bic wrote on Jan 14, 2007:

This is a great discussion, and has been very informative for me. I’m thinking about which language I want to tackle next, primarily for research into areas such as neural networks.

In my real life, I spend a lot of time writing heuristic filters based on regular expressions – (typically using PCRE: ‘perl-compatible regex’, though I don’t use Perl) – to do things like parse, filter, sanitize and combine multiple legacy datasets into one ‘canonical’ set. This often requires running other programs to pre-sort incoming data streams, and then maintaining arrays of match-probability scores for possible matches. How would these languages address problems such as these?

gar37bic wrote on Jan 14, 2007:

I also often ‘scrape’ (with permission) HTML, SQL, CVS, PDF, DOC and other files off the net, to generate input data for our company’s data mining operation, for example corporate federal and SEC filings.

And right now I’m scripting OpenOffice to do file format conversions to .DOC …

Eric Kidd wrote on Jan 14, 2007:

Chris Rathman: I thought I'd mention my pet project to translate the SICP examples into these languages.

Oh, that’s really cool. Thanks for the link!

David A. Wheeler wrote on Jan 15, 2007:

One problem with traditional Lisp notation is that it’s fairly hard to read — and as programs get bigger, it gets worse. It’s especially nasty when you use operators that are traditionally infix operators, e.g., (+ 3 (* 2 3)) may be a regular syntax, but even Graham admits it’s awkward.

I’ve developed a notation called “sweet-expressions” which I think are easier to read. A sweet-expression readers can read normal s-expressions, but can also accept other formats. It still works with all the macro constructs, etc., of Lispy languages. More info is here:
http://www.dwheeler.com/readable/

A Total Coward wrote on Jan 17, 2007:

bill a: your links to LoaL are all 404s. Did this stuff every see any daylight?

Timmy Jose wrote on Jan 18, 2007:

For a programmer used to C/C++ and JAVA and having merely studied the rudimentaries of Lisp in college ( as part of A.I ) and having had to content myself by looking at Ruby from the sidelines till now, I found this thread extremely interesting. It was all a bit one-sided though with Lispers outnumbering Rubyists (?) by a huge ratio! In my opinion, the syntax of Lisp seemed to be more in tune with my liking than Ruby’s. I think I’ll take the plunge with Lisp and see where Ruby fits in later. Good job on the discussion people!

Charlie Lindahl wrote on Dec 13, 2007:

I worked for a number of years on LISP machines at Texas Instruments in the 80s

One of my points is : it’s the environment. Not just the IDE; in LISP and Smalltalk environments EVERYTYHING is written in the language in
question/use (with the exception of machine-specific low-level stuff
required to talk to the OS / machine architecture).

What this means (not great for a general business case, but incredible for learning and expressive power) is that I can actually take apart / customize / study the underlying OS and/or tools.

So, for example, say you don’t like the way EMACS works. Go grab the source (a keystroke away in LISP IDEs) and go plowing through it with
the IDEs/debuggers and rewrite it to your specifications.

I learned the Macintosh Toolbox structure by running Lisp on the Mac
and dissecting the window data structures (on the fly / realtime )
without needing to have source code.

Once Ruby gets to the point of this kind of power in the environment,
inspection, and debugging it’ll be amazingly powerful.

Quite good now from what I’ve seen so far.

My $.02 Charlie Lindahl Houston, TX

Random Hacks

Random code snippets, projects and musings about software from Eric Kidd, a developer and entrepreneur.

132 KiB Raw Blame History Unescape Escape

Why Ruby is an acceptable LISP | Random Hacks

Why Ruby is an acceptable LISP

Ruby is a denser functional language than LISP

Ruby gives you about 80% of what you want from macros

Ruby's libraries, community, and momentum are good

More posts

Random Hacks

132 KiB

Raw Blame History