January 2003
(This article is derived from a keynote talk at the fall 2002 meeting of NEPLS.)
Visitors to this country are often surprised to find that Americans like to begin a conversation by asking "what do you do?" I've never liked this question.
I've rarely had a neat answer to it.
But I think I have finally solved the problem.
Now, when someone asks me what I do, I look them straight in the eye and say "I'm designing a new dialect of Lisp [blocked]." I recommend this answer to anyone who doesn't like being asked what they do.
The conversation will turn immediately to other topics.
I don't consider myself to be doing research on programming languages.
I'm just designing one, in the same way that someone might design a building or a chair or a new typeface.
I'm not trying to discover anything new.
I just want to make a language that will be good to program in.
In some ways, this assumption makes life a lot easier.
Americans open conversations with "what do you do?" I never had a neat answer, but I've solved it: I say "I'm designing a new dialect of Lisp [blocked]," and the conversation turns immediately to other things.
I'm not doing research on programming languages. I'm just designing one, the way someone designs a building or a chair—not trying to discover anything new, just wanting a language that's good to program in. That assumption makes life a lot easier.
When people ask what I do, I say I'm designing a new dialect of Lisp—not researching languages, just designing one, the way you'd design a building or a chair.
The difference between design and research seems to be a question of new versus good.
Design doesn't have to be new, but it has to be good.
Research doesn't have to be good, but it has to be new.
I think these two paths converge at the top: the best design surpasses its predecessors by using new ideas, and the best research solves problems that are not only new, but actually worth solving.
So ultimately we're aiming for the same destination, just approaching it from different directions.
What I'm going to talk about today is what your target looks like from the back.
What do you do differently when you treat programming languages as a design problem instead of a research topic?
The difference is new versus good. Design has to be good but not new; research has to be new but not good. The paths converge at the top: the best design uses new ideas, the best research solves problems worth solving—same destination, different directions.
I want to show what your target looks like from the back: what you do differently treating languages as a design problem instead of research.
Design has to be good but not new; research has to be new but not good. The two paths converge at the top, approached from different directions.
The biggest difference is that you focus more on the user.
Design begins by asking, who is this for and what do they need from it?
A good architect, for example, does not begin by creating a design that he then imposes on the users, but by studying the intended users and figuring out what they need.
Notice I said "what they need," not "what they want." I don't mean to give the impression that working as a designer means working as a sort of short-order cook, making whatever the client tells you to.
This varies from field to field in the arts, but I don't think there is any field in which the best work is done by the people who just make exactly what the customers tell them to.
The customer is always right in the sense that the measure of good design is how well it works for the user.
If you make a novel that bores everyone, or a chair that's horribly uncomfortable to sit in, then you've done a bad job, period.
It's no defense to say that the novel or the chair is designed according to the most advanced theoretical principles.
And yet, making what works for the user doesn't mean simply making what the user tells you to.
Users don't know what all the choices are, and are often mistaken about what they really want.
The answer to the paradox, I think, is that you have to design for the user, but you have to design what the user needs, not simply what he says he wants.
It's much like being a doctor.
You can't just treat a patient's symptoms. When a patient tells you his symptoms, you have to figure out what's actually wrong with him, and treat that.
This focus on the user is a kind of axiom from which most of the practice of good design can be derived, and around which most design issues center.
The biggest difference is that you focus on the user. A good architect doesn't impose a design; he studies the users and figures out what they need. Need, not want—a designer isn't a short-order cook.
The customer is always right in that good design is measured by how well it works for the user—a novel that bores everyone or a chair that's miserable to sit in is a bad job, period. Yet users don't know all the choices, and are often mistaken about what they really want.
So you design what the user needs, not what he says he wants. It's like being a doctor: you don't just treat reported symptoms—you figure out what's actually wrong. This is the axiom from which most of good design can be derived.
Design focuses on the user—but on what he needs, not what he says he wants. Like a doctor, you treat the actual problem, not the reported symptoms.
If good design must do what the user needs, who is the user?
When I say that design must be for users, I don't mean to imply that good design aims at some kind of lowest common denominator.
You can pick any group of users you want.
If you're designing a tool, for example, you can design it for anyone from beginners to experts, and what's good design for one group might be bad for another.
The point is, you have to pick some group of users.
I don't think you can even talk about good or bad design except with reference to some intended user.
You're most likely to get good design if the intended users include the designer himself.
When you design something for a group that doesn't include you, it tends to be for people you consider to be less sophisticated than you, not more sophisticated.
That's a problem, because looking down on the user, however benevolently, seems inevitably to corrupt the designer.
I suspect that very few housing projects in the US were designed by architects who expected to live in them.
You can see the same thing in programming languages.
C, Lisp, and Smalltalk were created for their own designers to use.
Cobol, Ada, and Java, were created for other people to use.
If you think you're designing something for idiots, the odds are that you're not designing something good, even for idiots.
Who is the user? Not some lowest common denominator—you can pick any group, and what's good for one may be bad for another. But you have to pick some group; there's no good or bad design except with reference to a user.
You're most likely to get good design when the users include the designer. Design for a group that doesn't include you and it tends to be for people you consider less sophisticated—and looking down on the user, however benevolently, corrupts the designer. C, Lisp, and Smalltalk were made for their designers; Cobol, Ada, and Java for others.
If you think you're designing something for idiots, the odds are that you're not designing something good, even for idiots.
You can pick any user group, but you get good design when the users include you. Looking down on the user corrupts the designer—C, Lisp, and Smalltalk were built for their makers; Cobol, Ada, and Java for other people.
Even if you're designing something for the most sophisticated users, though, you're still designing for humans.
It's different in research.
In math you don't choose abstractions because they're easy for humans to understand; you choose whichever make the proof shorter.
I think this is true for the sciences generally.
Scientific ideas are not meant to be ergonomic.
Over in the arts, things are very different.
Design is all about people.
The human body is a strange thing, but when you're designing a chair, that's what you're designing for, and there's no way around it.
All the arts have to pander to the interests and limitations of humans.
In painting, for example, all other things being equal a painting with people in it will be more interesting than one without.
It is not merely an accident of history that the great paintings of the Renaissance are all full of people.
If they hadn't been, painting as a medium wouldn't have the prestige that it does.
Like it or not, programming languages are also for people, and I suspect the human brain is just as lumpy and idiosyncratic as the human body.
Some ideas are easy for people to grasp and some aren't.
For example, we seem to have a very limited capacity for dealing with detail.
It's this fact that makes programing languages a good idea in the first place; if we could handle the detail, we could just program in machine language.
Even for the most sophisticated users, you're still designing for humans. In math you choose abstractions that make the proof shorter, not ones easy to understand. Scientific ideas aren't meant to be ergonomic.
The arts are different—design is all about people. When you design a chair, the human body is what you design for. All the arts pander to the limitations of humans: a painting with people in it is more interesting, and it's no accident the great Renaissance paintings are full of people.
Languages are for people too, and the brain is as lumpy as the body. We have a very limited capacity for detail—which is what makes languages a good idea at all; if we could handle it, we'd just program in machine language.
However sophisticated your user, you're still designing for humans—unlike research, where ideas aren't meant to be ergonomic. The arts all pander to human limits, and the brain is as lumpy as the body.
Remember, too, that languages are not primarily a form for finished programs, but something that programs have to be developed in.
Anyone in the arts could tell you that you might want different mediums for the two situations.
Marble, for example, is a nice, durable medium for finished ideas, but a hopelessly inflexible one for developing new ideas.
A program, like a proof, is a pruned version of a tree that in the past has had false starts branching off all over it.
So the test of a language is not simply how clean the finished program looks in it, but how clean the path to the finished program was.
A design choice that gives you elegant finished programs may not give you an elegant design process.
For example, I've written a few macro-defining macros full of nested backquotes that look now like little gems, but writing them took hours of the ugliest trial and error, and frankly, I'm still not entirely sure they're correct.
We often act as if the test of a language were how good finished programs look in it.
It seems so convincing when you see the same program written in two languages, and one version is much shorter.
When you approach the problem from the direction of the arts, you're less likely to depend on this sort of test. You don't want to end up with a programming language like marble.
For example, it is a huge win in developing software to have an interactive toplevel, what in Lisp is called a read-eval-print loop.
And when you have one this has real effects on the design of the language.
It would not work well for a language where you have to declare variables before using them, for example.
When you're just typing expressions into the toplevel, you want to be able to set x to some value and then start doing things to x.
You don't want to have to declare the type of x first. You may dispute either of the premises, but if a language has to have a toplevel to be convenient, and mandatory type declarations are incompatible with a toplevel, then no language that makes type declarations mandatory could be convenient to program in.
Languages aren't primarily a form for finished programs but something programs get developed in. You want different mediums for the two: marble is durable for finished ideas but hopelessly inflexible for new ones.
A program, like a proof, is a pruned tree that once had false starts branching all over it. So the test isn't how clean the finished program looks, but how clean the path to it was.
We act as if the test were how good finished programs look. From the arts you depend on that less; you don't want a language like marble.
It's a huge win to have an interactive toplevel, a read-eval-print loop. There you want to set x and start working on it, not declare its type first. If a language needs a toplevel to be convenient, and mandatory type declarations rule one out, such declarations can't be.
A language isn't just a form for finished programs but the medium you develop them in—marble versus oil. So the test isn't how clean the finished program looks, but how clean the path to it was; a toplevel rules out mandatory type declarations.
In practice, to get good design you have to get close, and stay close, to your users.
You have to calibrate your ideas on actual users constantly, especially in the beginning.
One of the reasons Jane Austen's novels are so good is that she read them out loud to her family.
That's why she never sinks into self-indulgently arty descriptions of landscapes, or pretentious philosophizing. (The philosophy's there, but it's woven into the story instead of being pasted onto it like a label.)
If you open an average "literary" novel and imagine reading it out loud to your friends as something you'd written, you'll feel all too keenly what an imposition that kind of thing is upon the reader.
In the software world, this idea is known as Worse is Better.
Actually, there are several ideas mixed together in the concept of Worse is Better, which is why people are still arguing about whether worse is actually better or not.
But one of the main ideas in that mix is that if you're building something new, you should get a prototype in front of users as soon as possible.
The alternative approach might be called the Hail Mary strategy.
Instead of getting a prototype out quickly and gradually refining it, you try to create the complete, finished, product in one long touchdown pass.
As far as I know, this is a recipe for disaster.
Countless startups destroyed themselves this way during the Internet bubble.
I've never heard of a case where it worked.
To get good design you stay close to your users. One reason Jane Austen's novels are so good is that she read them aloud to her family—that's why she never sinks into arty landscapes or pretentious philosophizing the way an average "literary" novel would.
In software this is Worse is Better. Several ideas are tangled in the phrase, but one main one is: if you're building something new, get a prototype in front of users as soon as you can.
The alternative is the Hail Mary strategy: instead of shipping a prototype and refining it, you create the finished product in one long touchdown pass. This is a recipe for disaster. Countless startups destroyed themselves this way during the Internet bubble. I've never heard of a case where it worked.
To get good design you stay close to actual users—Jane Austen read her novels aloud to her family. This is Worse is Better: get a prototype out fast. The Hail Mary strategy, one finished product in a single pass, is a recipe for disaster.
What people outside the software world may not realize is that Worse is Better is found throughout the arts.
In drawing, for example, the idea was discovered during the Renaissance.
Now almost every drawing teacher will tell you that the right way to get an accurate drawing is not to work your way slowly around the contour of an object, because errors will accumulate and you'll find at the end that the lines don't meet.
Instead you should draw a few quick lines in roughly the right place, and then gradually refine this initial sketch.
In most fields, prototypes have traditionally been made out of different materials.
Typefaces to be cut in metal were initially designed with a brush on paper.
Statues to be cast in bronze were modelled in wax.
Patterns to be embroidered on tapestries were drawn on paper with ink wash.
Buildings to be constructed from stone were tested on a smaller scale in wood.
What made oil paint so exciting, when it first became popular in the fifteenth century, was that you could actually make the finished work from the prototype.
You could make a preliminary drawing if you wanted to, but you weren't held to it; you could work out all the details, and even make major changes, as you finished the painting.
You can do this in software too.
A prototype doesn't have to be just a model; you can refine it into the finished product.
I think you should always do this when you can.
It lets you take advantage of new insights you have along the way.
But perhaps even more important, it's good for morale.
Worse is Better runs throughout the arts. Every drawing teacher tells you not to work slowly around an object's contour—errors accumulate and the lines won't meet—but to lay down a few quick lines roughly right and refine.
Prototypes were traditionally made of different materials: typefaces to be cut in metal were first drawn with a brush on paper, statues to be cast in bronze modelled in wax.
What made oil paint so exciting in the fifteenth century was that you could make the finished work from the prototype, even major changes as you went. You can do this in software too: refine the prototype into the finished product. It lets you exploit new insights, and—more important—it's good for morale.
Worse is Better runs through the arts. Drawing teachers say sketch roughly then refine; prototypes were once made in different materials. What made oil paint exciting was that you could make the finished work from the prototype—and you can do this in software too.
Morale is key in design.
I'm surprised people don't talk more about it.
One of my first drawing teachers told me: if you're bored when you're drawing something, the drawing will look boring.
For example, suppose you have to draw a building, and you decide to draw each brick individually.
You can do this if you want, but if you get bored halfway through and start making the bricks mechanically instead of observing each one, the drawing will look worse than if you had merely suggested the bricks.
Building something by gradually refining a prototype is good for morale because it keeps you engaged.
In software, my rule is: always have working code.
If you're writing something that you'll be able to test in an hour, then you have the prospect of an immediate reward to motivate you.
The same is true in the arts, and particularly in oil painting.
Most painters start with a blurry sketch and gradually refine it.
If you work this way, then in principle you never have to end the day with something that actually looks unfinished.
Indeed, there is even a saying among painters: "A painting is never finished, you just stop working on it."
This idea will be familiar to anyone who has worked on software.
Morale is another reason that it's hard to design something for an unsophisticated user.
It's hard to stay interested in something you don't like yourself.
To make something good, you have to be thinking, "wow, this is really great," not "what a piece of shit; those fools will love it."
Morale is key in design, and people don't talk about it enough. One of my first drawing teachers told me: if you're bored drawing something, the drawing will look boring. Draw a building's bricks mechanically once you're bored and they look worse than if you'd merely suggested them.
Refining a prototype keeps you engaged. In software my rule is: always have working code. If you can test something in an hour, you have an immediate reward in prospect. There's even a saying among painters: "A painting is never finished, you just stop working on it."
Morale is another reason it's hard to design for an unsophisticated user: it's hard to stay interested in something you don't like yourself. To make something good you have to be thinking, "wow, this is really great," not "what a piece of shit; those fools will love it."
Morale is key in design, and rarely discussed. Bored drawing looks boring; in software my rule is always have working code, so a test is an hour away. It's hard to stay interested in something you don't like yourself.
Design means making things for humans.
But it's not just the user who's human.
The designer is human too.
Notice all this time I've been talking about "the designer."
Design usually has to be under the control of a single person to be any good.
And yet it seems to be possible for several people to collaborate on a research project.
This seems to me one of the most interesting differences between research and design.
There have been famous instances of collaboration in the arts, but most of them seem to have been cases of molecular bonding rather than nuclear fusion.
In an opera it's common for one person to write the libretto and another to write the music.
And during the Renaissance, journeymen from northern Europe were often employed to do the landscapes in the backgrounds of Italian paintings.
But these aren't true collaborations.
They're more like examples of Robert Frost's "good fences make good neighbors." You can stick instances of good design together, but within each individual project, one person has to be in control.
I'm not saying that good design requires that one person think of everything.
There's nothing more valuable than the advice of someone whose judgement you trust. But after the talking is done, the decision about what to do has to rest with one person.
Why is it that research can be done by collaborators and design can't?
This is an interesting question.
I don't know the answer.
Perhaps, if design and research converge, the best research is also good design, and in fact can't be done by collaborators.
A lot of the most famous scientists seem to have worked alone.
But I don't know enough to say whether there is a pattern here.
It could be simply that many famous scientists worked when collaboration was less common.
Whatever the story is in the sciences, true collaboration seems to be vanishingly rare in the arts.
Design by committee is a synonym for bad design.
Why is that so?
Is there some way to beat this limitation?
I'm inclined to think there isn't-- that good design requires a dictator.
One reason is that good design has to be all of a piece.
Design is not just for humans, but for individual humans.
If a design represents an idea that fits in one person's head, then the idea will fit in the user's head too.
Related:
Design means making things for humans. But it's not just the user who's human. The designer is human too.
Design has to be under one person's control to be any good, yet research can be done by collaborators—one of the most interesting differences between them. The famous arts "collaborations" are molecular bonding, not nuclear fusion: in opera one writes the libretto, another the music. They're really Frost's "good fences make good neighbors"; within each project one person is in control.
This doesn't mean one person thinks of everything. But after the talking is done, the decision rests with one person.
Why can research be done by collaborators and design can't? I don't know. Perhaps, if design and research converge, the best research is also good design and can't be done by collaborators either—a lot of famous scientists worked alone.
True collaboration is vanishingly rare in the arts, and design by committee is a synonym for bad design. Good design requires a dictator. It has to be all of a piece, for individual humans: if a design fits in one person's head, it will fit in the user's head too.
Design has to be under one person's control to be any good, while research can be done by collaborators. Famous arts "collaborations" are really good fences making good neighbors. Design by committee is a synonym for bad design: good design requires a dictator.