233 lines
13 KiB
Markdown
233 lines
13 KiB
Markdown
---
|
||
created_at: '2016-12-25T02:41:26.000Z'
|
||
title: Reclaiming Software Engineering (2010)
|
||
url: http://www.zerobanana.com/essays/reclaiming-software-engineering
|
||
author: akkartik
|
||
points: 62
|
||
story_text:
|
||
comment_text:
|
||
num_comments: 14
|
||
story_id:
|
||
story_title:
|
||
story_url:
|
||
parent_id:
|
||
created_at_i: 1482633686
|
||
_tags:
|
||
- story
|
||
- author_akkartik
|
||
- story_13252608
|
||
objectID: '13252608'
|
||
year: 2010
|
||
|
||
---
|
||
> When I use a word, it means just what I choose it to mean—neither more
|
||
> nor less.
|
||
|
||
[](#p1 "Paragraph 1") It is the fate of every vocation to be
|
||
misunderstood by outsiders, and engineering is by no means an exception:
|
||
‘Most people think engineers do things in a very mechanistic way. … They
|
||
don’t seem to have any idea of the creativity and ingenuity in
|
||
engineering.’[2](#fn2 "Footnote 2") To the public at large the
|
||
engineering method is just following an impenetrable set of
|
||
scientifically predetermined instructions, the engineer a ‘soulless
|
||
apparatchik’[3](#fn3 "Footnote 3") —the sort of person who might regard
|
||
the employee-feeding machine from Chaplin’s Modern Times as a splendid
|
||
labour-saving device. It is as if engineers had created such machines in
|
||
their own image.
|
||
|
||
[](#p2 "Paragraph 2") Software engineering has an even bigger perception
|
||
problem, though: it is widely misunderstood even by insiders. Civil
|
||
engineers who fail to recognise their work in the typical engineering
|
||
mythology may be certain that, by definition, it is the public
|
||
perception at fault. What is engineering if not what civil engineers do?
|
||
By contrast, without some traditional engineering experience for
|
||
guidance a software developer in the same predicament cannot rule out
|
||
the thought that software development is somehow unlike engineering—or,
|
||
worse, that we are doing it improperly.
|
||
|
||
[](#p3 "Paragraph 3") Certainly there are those even within the
|
||
engineering profession who do subscribe to the outsider view. Software
|
||
developers are engaged in the exact same debate about the nature of our
|
||
work as engineers in the more traditional fields are about theirs. The
|
||
difference is that while software developers disagree about how software
|
||
development ought to work, most accept without question the mechanistic
|
||
theory of how engineering works—the engineering method as ‘adherence to
|
||
a specified morphology’[4](#fn4 "Footnote 4") . The morphology with
|
||
which we are all familiar is known as the Waterfall
|
||
Model,[5](#fn5 "Footnote 5") and the idea that all ‘real’ engineers
|
||
follow it is practically an article of faith.[6](#fn6 "Footnote 6") It
|
||
would come as a shock to most in the software community to discover that
|
||
this is not really the way engineering works. Proponents of a
|
||
mechanistic kind of ‘software engineering’ have been allowed to define
|
||
the term in agreement with their own prejudices.
|
||
|
||
[](#p4 "Paragraph 4") At the root of the unwillingness of the software
|
||
community to accept software development as a form of engineering lies a
|
||
single misconception, inexplicably shared across ideological boundaries.
|
||
To see what it is, we must first define a little terminology.
|
||
|
||
[](#p5 "Paragraph 5") The technical term for the output of an
|
||
engineering process is an ‘artefact’. The artefact is only nominally
|
||
what customers pay for. The actual product is some experience that
|
||
occurs between the artefact and the user.[7](#fn7 "Footnote 7") This is
|
||
what Pirsig was referring to when he wrote that ‘Quality is not a thing.
|
||
It is an event.’[8](#fn8 "Footnote 8") A bridge is an artefact but it is
|
||
not the product. The product is not getting wet.
|
||
|
||
[](#p6 "Paragraph 6") A design document—for example, a set of
|
||
engineering drawings—describes how to construct one or more artefacts,
|
||
but is not itself an artefact. A building is an artefact; the product it
|
||
provides, or one of them, is (again) not getting
|
||
wet.[9](#fn9 "Footnote 9") You can get some of the effect by holding a
|
||
set of working drawings above your head, but beyond a certain
|
||
point—probably a stiff breeze, anything more than a light drizzle or
|
||
when your arm gets tired, whichever comes first—you are bound to want
|
||
the actual building.
|
||
|
||
[](#p7 "Paragraph 7") Our problem arises when people mistake software
|
||
source code for an artefact. Source code is **not** an artefact because
|
||
the user does not interact with it; there is no product experience
|
||
created in this way. Source code is a design document because it
|
||
describes how to construct the true artefact, which is the
|
||
machine-executable code.[10](#fn10 "Footnote 10") Source code is the
|
||
map, not the territory.
|
||
|
||
[](#p8 "Paragraph 8") This point is critical to the understanding of
|
||
software development because, as [Jack Reeves
|
||
explained](http://www.developerdotstar.com/mag/articles/reeves_design.html "What is Software Design?")
|
||
in 1992, it identifies our work as designing rather than constructing
|
||
(or, for that matter, manufacturing) software.[11](#fn11 "Footnote 11")
|
||
Construction is an activity performed by compilers and the associated
|
||
infrastructure around them—not for nothing do we call this the ‘build
|
||
system’. We are software engineers, not software construction workers.
|
||
|
||
[](#p9 "Paragraph 9") We should not see engineering as a metaphor for
|
||
software development. Each branch of engineering has its own set of
|
||
heuristics, moulded in part by the nature of the materials with which it
|
||
works, and software engineering is naturally no exception. Chief among
|
||
its unique features is that software construction is both fast and
|
||
cheap—often taking on the order of seconds to minutes, generally with
|
||
zero marginal cost. Most engineers do not have the luxury of such a
|
||
short feedback loop between their designs and the finished artefacts. It
|
||
would be inappropriate for software engineering to simply adopt a
|
||
morphology from another branch of engineering, even if such a thing
|
||
existed. The common thread uniting those branches is not some shared
|
||
morphology but the fact that none is capable of being reduced to such a
|
||
morphology.
|
||
|
||
[](#p10 "Paragraph 10") Those who see software development as
|
||
mechanistic in nature invariably also mistake it for a construction
|
||
activity. Usually these people will say ‘Software engineering is not yet
|
||
a true engineering discipline’,[12](#fn12 "Footnote 12") but that they
|
||
hope it will eventually become
|
||
one.[13](#fn13 "Footnote 13"), [14](#fn14 "Footnote 14") The obstacle
|
||
is always a lack of ‘rigour’, the solution more
|
||
‘discipline’.[15](#fn15 "Footnote 15") Apparently the best way to
|
||
instil this discipline may to threaten the practitioner with
|
||
imprisonment.[16](#fn16 "Footnote 16")
|
||
|
||
[](#p11 "Paragraph 11") The most prominent example of this school of
|
||
thought is the unfortunately-named Software Engineering Institute at
|
||
Carnegie Mellon University. It is funded by the United States Department
|
||
of Defence, whose bizarre procurement policies treat designs and
|
||
artefacts identically. Unsurprisingly, the SEI tends to label ideas that
|
||
fit its patron’s worldview as ‘software engineering’, even when they may
|
||
be the opposite of established engineering practice.
|
||
|
||
[](#p12 "Paragraph 12") One form this takes is the fetishisation of
|
||
[reliability](../is-it-safe "Is it Safe?") over all other properties of
|
||
a system. To the mechanist, the fact that software sometimes crashes is
|
||
damning evidence that software is not yet an engineering discipline. In
|
||
part this is coloured by an unjustifiably narrow view of the range of
|
||
work in which engineers are engaged. Pencils and paperclips break all
|
||
the time.[17](#fn17 "Footnote 17") Even in areas where safety of life is
|
||
an issue—popularly assumed to be the sole province of engineers—failures
|
||
still occur absent any negligence on the part of
|
||
engineers.[18](#fn18 "Footnote 18") To assume that software is unique
|
||
among humanity’s creations in its tendency to fail is to ignore both the
|
||
evidence and the more prosaic explanation: that the cost to society of
|
||
ensuring it never can usually outweighs the cost of taking the risk.
|
||
|
||
[](#p13 "Paragraph 13") The temptation is strong to declare that this
|
||
reliability-obsessed process is not engineering at all, but it must be
|
||
resisted. I do not claim that all software is engineered, at least in
|
||
the modern sense. Yet so long as we are developing software within a
|
||
social context, where the finished artefact has users other than its
|
||
designers, we can scarcely escape the conclusion that what we are doing
|
||
is engineering. Still, I would certainly argue that methodologies that
|
||
disregard the expenditure of resources are the antithesis of good
|
||
engineering.
|
||
|
||
[](#p14 "Paragraph 14") If I wished to be kind to the SEI, I would note
|
||
only that the context in which they operate is not shared by most—or
|
||
even many—software engineers, and that they could do us all a favour by
|
||
choosing a name that does not imply that they represent the only way of
|
||
doing software engineering. If I were feeling less charitable, I might
|
||
add that even in their chosen context their methods have proven
|
||
[conspicuously
|
||
unsuccessful](http://www.baselinemag.com/networking/The-Ugly-History-of-Tool-Development-at-the-FAA/ "The Ugly History of Tool Development at the FAA").
|
||
|
||
[](#p15 "Paragraph 15") On the other side of the fence, many of those
|
||
who reject the mechanistic theory of software development nevertheless
|
||
mistake the source code for an artefact just the same. At the nexus of
|
||
these two ideas lies the concept of software development as ‘craft’.
|
||
Recognition that all software development is unpredictable design work
|
||
and belief that software is constructed by humans can be reconciled only
|
||
in a medieval model[19](#fn19 "Footnote 19") of software development, in
|
||
which the design and construction of artefacts are not separate
|
||
activities:
|
||
|
||
> The technology of the middle ages was the technology of the artisan. …
|
||
> \[In the Renaissance\] the artisan has been split up into his
|
||
> components, the engineer and the worker.[20](#fn20 "Footnote 20")
|
||
|
||
[](#p16 "Paragraph 16") Seen in this light, the epitome of the software
|
||
craftsman is to be found in [the story of
|
||
Mel](http://catb.org/~esr/jargon/html/story-of-mel.html "Hacker’s Dictionary – The Story of Mel"),
|
||
the ‘Real Programmer’, who famously programs directly in machine code.
|
||
Mel’s legendary technical skills have made him a hero to many, yet he
|
||
seems completely isolated from the economic consequences of his
|
||
work.[21](#fn21 "Footnote 21") It exists in a vacuum, outside of any
|
||
social context.
|
||
|
||
[](#p17 "Paragraph 17") Something is always lost when the engineer is
|
||
separated from the worker, but such transitions occur because these
|
||
losses are outweighed by the gains. The combined efforts of all the
|
||
master craftsmen in Florence might never have been sufficient to
|
||
complete the cathedral dome had Filippo Brunelleschi—or some other
|
||
engineer—not devised a way to build it.[22](#fn22 "Footnote 22") In
|
||
software such a split was already in evidence by the 1960s, simply
|
||
because the methodology Mel used to craft his blackjack program imposed
|
||
a very low upper bound on software’s transformative power. In the
|
||
intervening decades the workers—‘programmers’, in the contemporary sense
|
||
of the term—have been automated out of existence. The revolution will
|
||
not be televised; it already happened and you missed it.
|
||
|
||
[](#p18 "Paragraph 18") To regress to a guild of medieval [Real
|
||
Programmers](http://catb.org/~esr/jargon/html/R/Real-Programmer.html "Hacker’s Dictionary – Real Programmer")
|
||
does not strike me as a goal to which we should aspire. We need not do
|
||
so if we recognise that source code is not an artefact and accept our
|
||
role as software designers and not software builders. Engineers, freed
|
||
from the need to personally construct their artefacts, take no less care
|
||
in and derive no less pleasure from their work than did the craftsmen
|
||
who preceded them.
|
||
|
||
[](#p19 "Paragraph 19") In all our discourse about software engineering,
|
||
we have failed to challenge the assumption that it could only be brought
|
||
about through the application of ‘rigour’, ‘discipline’ and
|
||
government-mandated participation in a monoculture that treats every
|
||
line of code as if lives depended on it, on pain of imprisonment,
|
||
regardless of context. This vision may be to some a shining beacon of
|
||
hope, to others a nightmare that makes
|
||
[Kafka](http://www.imdb.com/title/tt0057427/ "IMDB – The Trial (1962)")
|
||
look like Disney. But the essence of engineering is not to be found in
|
||
these things. To be an engineer is to strive for change in a world where
|
||
resources are limited.
|
||
|
||
[](#p20 "Paragraph 20") We are the heirs of an engineering tradition as
|
||
old as civilisation, but Eden is burning. The name of our profession
|
||
itself—the very core of our identity—has been co-opted by some of the
|
||
most dysfunctional organisations in society and redefined in ways that
|
||
conflict with our values. In order to claim our birthright we must first
|
||
recapture what it means to be a software engineer.
|