---
created_at: '2014-11-04T21:51:55.000Z'
title: Array languages for Lisp programmers (2010)
url: http://archive.vector.org.uk/art10500180
author: networked
points: 76
story_text: ''
comment_text: 
num_comments: 18
story_id: 
story_title: 
story_url: 
parent_id: 
created_at_i: 1415137915
_tags:
- story
- author_networked
- story_8559275
objectID: '8559275'

---
[Source](http://archive.vector.org.uk/art10500180 "Permalink to Vector, the Journal of the British APL Association")

# Vector, the Journal of the British APL Association

﻿ 

 

[ ![Kenneth E. Iverson][1]  
Kenneth E. Iverson  
1920-2004 ][2]

# Current issue

![Vol.26 No.4][3]

[Vol.26 No.4][4]

[Articles in press][5]   
[Full index][6]

## Volumes

* [26][7]
* [25][8]
* [24][9]
* [23][10]
* [22][11]
* [21][12]
* [20][13]
* [19][14]
* [18][15]
* [17][16]
* [16][17]
* [15][18]
* [14][19]
* [13][20]
* [12][21]
* [11][22]
* [10][23]
* [9][24]
* [8][25]
* [7][26]
* [6][27]
* [5][28]
* [4][29]
* [3][30]
* [2][31]
* [1][32]
* [home][33]

    * [advertise][34]

    * [archive][35]

        * [character-mapped code][36]

    * [contribute][37]

    * [in press][5]

    * [team][38]

    * [subscribe][39]
* [about us][40]
* [blog][41]
* [books][42]
* [community][43]

    * [committee][44]

    * [consultants][45]

    * [events][46]

    * [sponsors][47]
* [contact][48]
* [fonts][49]
* [interpreters][50]
* [video][51]

© 1984-2018  
British APL Association  
All rights reserved.  

Archive articles posted online on request: ask the [archivist][52]. 

[archive][35]/[24][9]/[2][53]

Volume 24, No.2

* Proof for author
* 0.1

# Array languages for Lisp programmers

# by Slobodan Blazeski (slobodan.blazeski@gmail.com)

A language that doesn't affect the way you think about programming is not worth knowing. [[1]][54]

You can post comments on this article at the [Vector blog][55]. _Ed._

During my pilgrimage into the land of programming languages I've stumbled upon quite a lot of them. Most languages are just poorly-integrated duct-tape-and-wire rehashes of principles discovered by others. Few of them are true paradigm-shifting jewels. APL and its offspring are definitely among those. But what's their value for spoiled Lisper brats? What's so great about them that Lisp doesn't already have? During my whirlwind presentation of examples I will try to answer this question. 

The first thing that comes to mind is that array-languages such as APL, J and q operate on whole arrays instead of munching elements one by one. 

By using array functions you can work on whole containers the same way as you work on scalars. It takes only a little tinkering with CLOS[[2]][56] dispatch to make a set of operators that work roughly the same. (Adding ranks will take more time than I currently have.) [[3]][57]

| J | Common Lisp |  
| - | ----------- |  
|  

    
    
       2 + 2 3 4
    4 5 6
    

 | 
    
    
       (j+ 2 #(2 3 4))
    4 5 6
    

 |  
| 
    
    
       2 3 4 + 1 2 3
    3 5 7
    

 | 
    
    
       (j+ #(2 3 4) #(1 2 3))
    3 5 7
    

 | 

Then having a set of operators for making arrays is very handy, so I wrote those quickly in Lisp. 

| J | Common Lisp |  
| - | ----------- |  
|  

    
    
       2 3 $ 'abcde'
    abc
    dea
    

 | 
    
    
       (shape '(2 3) "abcde")
    abc
    dea
    

 |  
| 
    
    
       i. 2 3
    0 1 2
    3 4 5
    

 | 
    
    
       (enum 2 3)
    0 1 2
    3 4 5
    

 | 

Adverbs give great power to J, but Lisp has had higher-order functions since forever. Things like built-in `reduce` already work on vectors and it doesn't take much work to create a version of `reduce` such as `fold` that handles multidimensional arrays. 

| J | Common Lisp |  
| - | ----------- |  
|  

    
    
       +/ 1 2 3 4
    10
    

 | 
    
    
       (reduce #'+ #(1 2 3 4))
    10
    

 |  
| 
    
    
       +/ i. 2 3
    3 5 7
    

 | 
    
    
       (fold #'+ #(1 2 3 4))
    3 5 7
    

 | 

And once you have them it's too easy to write factorial of 6 in array style: 

| J | Common Lisp |  
| - | ----------- |  
|  

    
    
       */ 1+ i. 5
    120
    

 | 
    
    
    (reduce #'+ (j+ 1(enum 5)) _or_
    (fold #'j+ (enum :start 1))
    

 | 

One of q's very cool features allows omitting the first three parameters of a function. That's a feature that I immediately ported to Lisp with a little macro. 

| q | Common Lisp |  
| - | ----------- |  
|  

    
    
    {x+y+z}⇔{[x;y;z] x+y+z}
    

 | 
    
    
    (f + x y z) ⇔
    (lambda (x y z)(+ x y z))
    

 | 

But maybe it's not about single features but the ways to combine them into non-trivial programs and problem solutions. 

Just take a look at the Lisp code answering the task below: [[4]][58]

> We have a list of elements, some are duplicates. We're trying to figure out how to find the duplicate elements and increase a counter value by 1 for each instance of the element found. The list consists of lists with two elements, the first being the incremental counter, the second being the string sought. Example: 
>     
>     
>     ((1 "one") (1 "two") (1 "three") (1 "one") (1 "four") (1 "two"))
> 
> The result should be:
>     
>     
>     ((2 "one") (2 "two") (1 "three") (1 "four"))

q
: 
    
    
    count:flip(count each group v[;1];unique v[;1])
    

Lisp
: 
    
    
    (defun count (list)
      (let ((hash (make-hash-table)))
        (dolist (el list)
          (incf (gethash (cadr el) hash 0) (car el)))
        (let (result)
          (maphash (lambda (key val)
            (push (list val key) result))
          hash)
        result)))
    

The above seems an exemplary case where Lisp loses its magic. But that's only because q has `group` built in. If I write it, or being a lazy person like myself ask c.l.l [[5]][59] denizens to write it for me, the edge is lost. 

q
: 
    
    
    brunberg:flip(count each group v[;1];unique v[;1])
    

Lisp
: 
    
    
    (defun brunberg (l)
      (mapcar (f list (length x) (cadar x))  (group l #'cadr #'string=)))
    

The q solution is still shorter but only due to a Lisp culture of very long and descriptive names. Using a golfing [[6]][60] library with operators' bindings pointing to shorter names will make it look even shorter. 

Another fine example is the problem of parsing identifiers: 

> This piece of [Haskell] code goes through a parse tree of Haskell source code, locates every reference to an identifier that ends with `Widget`, puts it on a list, and removes duplicates so every identifier is represented in the list only once. [[7]][61]

Haskell
: 
    
    
    extractWidgets :: (Data a) => a -> [String]
    extractWidgets = nub . map ((HsIdent a)->a) . listify isWidget
      where isWidget (HsIdent actionName)
          | "Widget" `isSuffixOf` actionName = True
        isWidget _ = False
    

q
: 
    
    
    a:distinct raze over x
    a where a like "*Widget"
    

Lisp
: 
    
    
    (defun widgets (l)
      (unique (keep (f like x "*Widget") (flatten l))))
    

This time the difference is even smaller. I have the same number of tokens with a surplus of parentheses. 

It seems it's always the same old story. As soon as Lisp integrates the needed utility or invests into a domain-specific language, the advantage of the other language is lost. The big ball of mud is like a warlock, able to steal opponents' powers while still staying the same.   

However the features above were easy compared to the work needed to implement my beloved tacit [[8]][62] style. While point-free style isn't a complete stranger to the Lisp community, it remains a rarity [[9]][63]: 

Lisp
: 
    
    
    (reduce  (flip #'cons) #(1 2 3 4) :initial-value '())
    (4 3 2 1)
    

Writing a macro that understands forks and hooks is an exercise for a beginner – if the operators covered are strictly monadic and/or dyadic as are those in J. However in Lisp all those optional auxiliary and keyword arguments immensely complicate matters. In order to make it operational within a month or so of effort a tacit macro could be implemented to work just on a subset of Lisp operators. 

A library consisting of operators with integrated looping, adverbs that understand verb rank, and a macro able to operate on hooks and forks would bridge the gap in Lisp array-processing facilities. But it's not enough to bridge the gap between technology and need. Somebody needs to make humans want to cross that bridge or else it will follow the fate of Richard C. Waters' series [[10].][64]    

So why should Lispers study array programming languages? 

Learning anything of APL, J or q makes programmers aware of opportunities opened by thinking at the higher level of container abstraction. This knowledge pays off well in Lisp, with or without the array sublanguage. Keep in mind too that the difference between an array sublanguage and an array language is like the difference between learning a foreign language at home or among its native speakers. You _can_ learn just the language while remaining clueless about its culture! 

loop
: 
    
    
    (loop for i from 0 to 10
      collect ( loop for j from 0 to 10
        collect (+ i j)))
    
    

functional
: 
    
    
    (flatten (outer #'+ (range 10) (range 10)))
    

loop
: 
    
    
    (defun separate (list)
      (loop for element in list
        if (plusp element)
        collect element into positives
        else if (minusp element)
        collect (abs element) into minuses
        else collect element into zeroes
        finally (return (list minuses zeroes positives))))
    

functional
: 
    
    
    (defun separate (lst)
      (mapcar  (f mapcar #'abs (remove-if-not x lst)))
        (list #'plusp #'zerop #'minusp)))
    

## Notes and references

1. Alan J. Perlis, Epigrams in Programming
2. <http://en.wikipedia.org/wiki/Common_Lisp_Object_System>
3. The Lisp return values were presented in J style for increased readability. 
4. http://www.nsl.com/papers/kisntlisp.htm 
5. comp.lang.lisp  
<http://groups.google.com/group/comp.lang.lisp/>
6. Golfing:  
<http://en.wikipedia.org/wiki/Perl#Perl_golf>  
Some of the functional solutions were developed in the c.l.l. golfing sessions with feedback from many c.l.l. denizens. 
7. <http://www.defmacro.org/ramblings/haskell-productivity.html>
8. <http://en.wikipedia.org/wiki/Tacit_programming>
9. Vassil Nikolov – "Could loop loop backward across vector?" 
10. <http://series.sourceforge.net/>
11. The first three solutions in the last block were written by Kenny Tilton, Matt Pillsbury and Rob St Amant respectively. Though Tilton's and Amant's solutions are far longer they're also more efficient. 

[ ][65]
    
    
    script began 16:21:13
    caching off
    debug mode off
    cache time 3600 sec
    indmtime not found in cache
    cached index is fresh
    recompiling index.xml
    index compiled in 0.2742 secs
    read index
    read issues/index.xml
    identified 26 volumes, 101 issues
    array (
      'id' => '10500180',
    )
    regenerated static HTML
    article source is 'XHTML'
    completed in 0.3107 secs
    

[1]: http://archive.vector.org.uk/images/kei/tn_keicartoon.gif
[2]: http://en.wikipedia.org/wiki/Kenneth_E._Iverson "Wikipedia article"
[3]: http://archive.vector.org.uk/images/covers/32x45/v2604.jpg
[4]: http://archive.vector.org.uk/26/4
[5]: http://archive.vector.org.uk/inpress
[6]: http://archive.vector.org.uk/index
[7]: http://archive.vector.org.uk/26
[8]: http://archive.vector.org.uk/25
[9]: http://archive.vector.org.uk/24
[10]: http://archive.vector.org.uk/23
[11]: http://archive.vector.org.uk/22
[12]: http://archive.vector.org.uk/21
[13]: http://archive.vector.org.uk/20
[14]: http://archive.vector.org.uk/19
[15]: http://archive.vector.org.uk/18
[16]: http://archive.vector.org.uk/17
[17]: http://archive.vector.org.uk/16
[18]: http://archive.vector.org.uk/15
[19]: http://archive.vector.org.uk/14
[20]: http://archive.vector.org.uk/13
[21]: http://archive.vector.org.uk/12
[22]: http://archive.vector.org.uk/11
[23]: http://archive.vector.org.uk/10
[24]: http://archive.vector.org.uk/9
[25]: http://archive.vector.org.uk/8
[26]: http://archive.vector.org.uk/7
[27]: http://archive.vector.org.uk/6
[28]: http://archive.vector.org.uk/5
[29]: http://archive.vector.org.uk/4
[30]: http://archive.vector.org.uk/3
[31]: http://archive.vector.org.uk/2
[32]: http://archive.vector.org.uk/1
[33]: https://sites.google.com/site/baavector/home
[34]: https://sites.google.com/site/baavector/home/advertise
[35]: http://archive.vector.org.uk/
[36]: https://sites.google.com/site/baavector/home/character-mapped-apl
[37]: https://sites.google.com/site/baavector/home/contribute
[38]: https://sites.google.com/site/baavector/home/team
[39]: https://sites.google.com/site/baavector/home/subscribe
[40]: https://sites.google.com/site/baavector/about-us
[41]: https://sites.google.com/site/baavector/blog
[42]: https://sites.google.com/site/baavector/books
[43]: https://sites.google.com/site/baavector/community
[44]: https://sites.google.com/site/baavector/community/committee
[45]: https://sites.google.com/site/baavector/community/consultants
[46]: https://sites.google.com/site/baavector/community/events
[47]: https://sites.google.com/site/baavector/community/sponsors
[48]: https://sites.google.com/site/baavector/contact
[49]: https://sites.google.com/site/baavector/fonts
[50]: https://sites.google.com/site/baavector/interpreters
[51]: https://sites.google.com/site/baavector/video
[52]: mailto:archivist%40vector.org.uk
[53]: http://archive.vector.org.uk/24/2
[54]: http://archive.vector.org.uk#ref1
[55]: http://vectoreditor.blogspot.com/2009/11/j-and-q-for-lispers.html
[56]: http://archive.vector.org.uk#ref2
[57]: http://archive.vector.org.uk#ref3
[58]: http://archive.vector.org.uk#ref4
[59]: http://archive.vector.org.uk#ref5
[60]: http://archive.vector.org.uk#ref6
[61]: http://archive.vector.org.uk#ref7
[62]: http://archive.vector.org.uk#ref8
[63]: http://archive.vector.org.uk#ref9
[64]: http://archive.vector.org.uk#ref10
[65]: http://validator.w3.org/check?uri=referer "Validate the markup"

