July 2004
(This essay is derived from a talk at Oscon 2004.)
A few months ago I finished a new book, and in reviews I keep noticing words like "provocative'' and "controversial.''
To say nothing of "idiotic.''
I didn't mean to make the book controversial.
I was trying to make it efficient.
I didn't want to waste people's time telling them things they already knew.
It's more efficient just to give them the diffs.
But I suppose that's bound to yield an alarming book.
A few months ago I finished a new book, and in reviews I keep noticing words like "provocative'' and "controversial.'' To say nothing of "idiotic.''
I didn't mean to make it controversial; I was trying to make it efficient, just to give people the diffs. But that's bound to yield an alarming book.
My new book keeps getting called provocative and controversial, but I was just trying to be efficient — to give people the diffs.
Edisons
There's no controversy about which idea is most controversial: the suggestion that variation in wealth might not be as big a problem as we think.
I didn't say in the book that variation in wealth was in itself a good thing.
I said in some situations it might be a sign of good things.
A throbbing headache is not a good thing, but it can be a sign of a good thing-- for example, that you're recovering consciousness after being hit on the head.
Variation in wealth can be a sign of variation in productivity. (In a society of one, they're identical.)
And that is almost certainly a good thing: if your society has no variation in productivity, it's probably not because everyone is Thomas Edison.
It's probably because you have no Thomas Edisons.
In a low-tech society you don't see much variation in productivity.
If you have a tribe of nomads collecting sticks for a fire, how much more productive is the best stick gatherer going to be than the worst?
A factor of two?
Whereas when you hand people a complex tool like a computer, the variation in what they can do with it is enormous.
That's not a new idea.
Fred Brooks wrote about it in 1974, and the study he quoted was published in 1968.
But I think he underestimated the variation between programmers.
He wrote about productivity in lines of code: the best programmers can solve a given problem in a tenth the time.
But what if the problem isn't given?
In programming, as in many fields, the hard part isn't solving problems, but deciding what problems to solve.
Imagination is hard to measure, but in practice it dominates the kind of productivity that's measured in lines of code.
Productivity varies in any field, but there are few in which it varies so much.
The variation between programmers is so great that it becomes a difference in kind.
I don't think this is something intrinsic to programming, though.
In every field, technology magnifies differences in productivity.
I think what's happening in programming is just that we have a lot of technological leverage.
But in every field the lever is getting longer, so the variation we see is something that more and more fields will see as time goes on.
And the success of companies, and countries, will depend increasingly on how they deal with it.
If variation in productivity increases with technology, then the contribution of the most productive individuals will not only be disproportionately large, but will actually grow with time.
When you reach the point where 90% of a group's output is created by 1% of its members, you lose big if something (whether Viking raids, or central planning) drags their productivity down to the average.
If we want to get the most out of them, we need to understand these especially productive people.
What motivates them?
What do they need to do their jobs?
How do you recognize them?
How do you get them to come and work for you?
And then of course there's the question, how do you become one?
The most controversial idea is that variation in wealth might not be as big a problem as we think. I didn't say it was good in itself; I said it can be a sign of good things — a sign of variation in productivity. And that is almost certainly good: if your society has no variation in productivity, it's probably not because everyone is Thomas Edison. It's because you have no Thomas Edisons.
In a low-tech society you see little variation: how much more productive is the best stick-gatherer than the worst, a factor of two? But hand people a computer, and the variation is enormous.
Fred Brooks wrote about this in 1974, measuring productivity in lines of code: the best solve a given problem in a tenth the time. But what if the problem isn't given? The hard part isn't solving problems but deciding what problems to solve, and imagination dominates.
In few fields does productivity vary so much it becomes a difference in kind. This isn't intrinsic to programming; technology magnifies differences in every field, and programming just has a lot of leverage. But everywhere the lever is getting longer, so the success of companies and countries will increasingly depend on how they deal with it.
When 90% of a group's output comes from 1% of its members, you lose big if something drags their productivity down to the average.
So we need to understand these especially productive people. What motivates them? What do they need? How do you recognize them — and become one?
Variation in wealth can signal variation in productivity, and technology magnifies that variation enormously — so the most productive people matter more and more, which is why we need to understand them.
More than Money
I know a handful of super-hackers, so I sat down and thought about what they have in common.
Their defining quality is probably that they really love to program.
Ordinary programmers write code to pay the bills.
Great hackers think of it as something they do for fun, and which they're delighted to find people will pay them for.
Great programmers are sometimes said to be indifferent to money.
This isn't quite true.
It is true that all they really care about is doing interesting work.
But if you make enough money, you get to work on whatever you want, and for that reason hackers are attracted by the idea of making really large amounts of money.
But as long as they still have to show up for work every day, they care more about what they do there than how much they get paid for it.
Economically, this is a fact of the greatest importance, because it means you don't have to pay great hackers anything like what they're worth.
A great programmer might be ten or a hundred times as productive as an ordinary one, but he'll consider himself lucky to get paid three times as much.
As I'll explain later, this is partly because great hackers don't know how good they are.
But it's also because money is not the main thing they want.
What do hackers want?
Like all craftsmen, hackers like good tools.
In fact, that's an understatement.
Good hackers find it unbearable to use bad tools.
They'll simply refuse to work on projects with the wrong infrastructure.
At a startup I once worked for, one of the things pinned up on our bulletin board was an ad from IBM.
It was a picture of an AS400, and the headline read, I think, "hackers despise it.'' [1]
When you decide what infrastructure to use for a project, you're not just making a technical decision.
You're also making a social decision, and this may be the more important of the two.
For example, if your company wants to write some software, it might seem a prudent choice to write it in Java.
But when you choose a language, you're also choosing a community.
The programmers you'll be able to hire to work on a Java project won't be as smart [blocked] as the ones you could get to work on a project written in Python.
And the quality of your hackers probably matters more than the language you choose.
Though, frankly, the fact that good hackers prefer Python to Java should tell you something about the relative merits of those languages.
Business types prefer the most popular languages because they view languages as standards.
They don't want to bet the company on Betamax.
The thing about languages, though, is that they're not just standards.
If you have to move bits over a network, by all means use TCP/IP.
But a programming language isn't just a format.
A programming language is a medium of expression.
I've read that Java has just overtaken Cobol as the most popular language.
As a standard, you couldn't wish for more.
But as a medium of expression, you could do a lot better.
Of all the great programmers I can think of, I know of only one who would voluntarily program in Java.
And of all the great programmers I can think of who don't work for Sun, on Java, I know of zero.
Great hackers also generally insist on using open source software.
Not just because it's better, but because it gives them more control.
Good hackers insist on control.
This is part of what makes them good hackers: when something's broken, they need to fix it.
You want them to feel this way about the software they're writing for you.
You shouldn't be surprised when they feel the same way about the operating system.
A couple years ago a venture capitalist friend told me about a new startup he was involved with.
It sounded promising.
But the next time I talked to him, he said they'd decided to build their software on Windows NT, and had just hired a very experienced NT developer to be their chief technical officer.
When I heard this, I thought, these guys are doomed.
One, the CTO couldn't be a first rate hacker, because to become an eminent NT developer he would have had to use NT voluntarily, multiple times, and I couldn't imagine a great hacker doing that; and two, even if he was good, he'd have a hard time hiring anyone good to work for him if the project had to be built on NT. [2]
The defining quality of the super-hackers I know is that they really love to program. Ordinary programmers write code to pay the bills; great hackers do it for fun, delighted that people will pay them for it.
They're said to be indifferent to money, which isn't quite true: enough of it lets you work on whatever you want, so hackers are attracted by it. But as long as they have to show up every day, they care more about what they do than what they're paid.
So you don't have to pay great hackers anything like what they're worth. One might be a hundred times as productive yet feel lucky to get paid three times as much — partly because they don't know how good they are, partly because money isn't the main thing they want.
What they want is good tools. Good hackers find it unbearable to use bad ones, and will refuse to work on projects with the wrong infrastructure.
Choosing infrastructure isn't just a technical decision but a social one. Choose a language and you're choosing a community: the programmers you can hire for Java won't be as smart [blocked] as the ones for Python. And the quality of your hackers matters more than the language.
Business types prefer popular languages as standards — they don't want to bet the company on Betamax. But a programming language isn't just a format; it's a medium of expression.
Java just overtook Cobol as the most popular language. As a standard you couldn't wish for more; as a medium of expression you could do a lot better. Of all the great programmers I know who don't work for Sun, the number who would voluntarily program in Java is zero.
Great hackers also insist on open source, because it gives them control. When something's broken, they need to fix it — and don't be surprised when they feel that way about the operating system too.
A VC friend told me about a promising startup that decided to build on Windows NT and hired an experienced NT developer as CTO. These guys are doomed, I thought: no first-rate hacker would have used NT enough to become eminent at it, and even a good one couldn't hire anyone good onto an NT project.
Great hackers love to program and care more about doing interesting work than money; what they really demand is good tools — good languages, open source, and the control that comes with it.
The Final Frontier
After software, the most important tool to a hacker is probably his office.
Big companies think the function of office space is to express rank.
But hackers use their offices for more than that: they use their office as a place to think in.
And if you're a technology company, their thoughts are your product.
So making hackers work in a noisy, distracting environment is like having a paint factory where the air is full of soot.
The cartoon strip Dilbert has a lot to say about cubicles, and with good reason.
All the hackers I know despise them.
The mere prospect of being interrupted is enough to prevent hackers from working on hard problems. If you want to get real work done in an office with cubicles, you have two options: work at home, or come in early or late or on a weekend, when no one else is there.
Don't companies realize this is a sign that something is broken?
An office environment is supposed to be something that helps you work, not something you work despite.
Companies like Cisco are proud that everyone there has a cubicle, even the CEO.
But they're not so advanced as they think; obviously they still view office space as a badge of rank.
Note too that Cisco is famous for doing very little product development in house.
They get new technology by buying the startups that created it-- where presumably the hackers did have somewhere quiet to work.
One big company that understands what hackers need is Microsoft.
I once saw a recruiting ad for Microsoft with a big picture of a door.
Work for us, the premise was, and we'll give you a place to work where you can actually get work done.
And you know, Microsoft is remarkable among big companies in that they are able to develop software in house.
Not well, perhaps, but well enough.
If companies want hackers to be productive, they should look at what they do at home.
At home, hackers can arrange things themselves so they can get the most done.
And when they work at home, hackers don't work in noisy, open spaces; they work in rooms with doors.
They work in cosy, neighborhoody places with people around and somewhere to walk when they need to mull something over, instead of in glass boxes set in acres of parking lots.
They have a sofa they can take a nap on when they feel tired, instead of sitting in a coma at their desk, pretending to work.
There's no crew of people with vacuum cleaners that roars through every evening during the prime hacking hours.
There are no meetings or, God forbid, corporate retreats or team-building exercises.
And when you look at what they're doing on that computer, you'll find it reinforces what I said earlier about tools.
They may have to use Java and Windows at work, but at home, where they can choose for themselves, you're more likely to find them using Perl and Linux.
Indeed, these statistics about Cobol or Java being the most popular language can be misleading.
What we ought to look at, if we want to know what tools are best, is what hackers choose when they can choose freely-- that is, in projects of their own.
When you ask that question, you find that open source operating systems already have a dominant market share, and the number one language is probably Perl.
After software, a hacker's most important tool is his office. Big companies think office space expresses rank, but hackers use theirs to think in — and if you're a technology company, their thoughts are your product. Making them work in a noisy environment is like running a paint factory where the air is full of soot.
All the hackers I know despise cubicles. The mere prospect of being interrupted keeps them off hard problems. An office is supposed to be something that helps you work, not something you work despite.
Cisco is proud everyone has a cubicle, even the CEO — which just means they still see office space as a badge of rank. And Cisco does little development in house; they buy the startups that created the technology, where the hackers had somewhere quiet to work.
To learn what hackers need, look at what they do at home. There they work in rooms with doors, with a sofa to nap on, no vacuum-cleaner crew during prime hacking hours, no meetings or corporate retreats. And they choose Perl and Linux, not the Java and Windows they're stuck with at work.
Statistics about Cobol or Java mislead. What we should look at is what hackers choose freely, in projects of their own — and there open source operating systems already dominate, and the number one language is probably Perl.
A hacker's office is where he thinks, and his thoughts are your product — so the cubicle, which hackers despise, is a sign something is broken. Watch what hackers do at home to learn what they really need.
Interesting
Along with good tools, hackers want interesting projects.
What makes a project interesting?
Well, obviously overtly sexy applications like stealth planes or special effects software would be interesting to work on.
But any application can be interesting if it poses novel technical challenges.
So it's hard to predict which problems hackers will like, because some become interesting only when the people working on them discover a new kind of solution.
Before ITA (who wrote the software inside Orbitz), the people working on airline fare searches probably thought it was one of the most boring applications imaginable.
But ITA made it interesting by redefining [blocked] the problem in a more ambitious way.
I think the same thing happened at Google.
When Google was founded, the conventional wisdom among the so-called portals was that search was boring and unimportant.
But the guys at Google didn't think search was boring, and that's why they do it so well.
This is an area where managers can make a difference.
Like a parent saying to a child, I bet you can't clean up your whole room in ten minutes, a good manager can sometimes redefine a problem as a more interesting one.
Steve Jobs seems to be particularly good at this, in part simply by having high standards.
There were a lot of small, inexpensive computers before the Mac.
He redefined the problem as: make one that's beautiful.
And that probably drove the developers harder than any carrot or stick could.
They certainly delivered.
When the Mac first appeared, you didn't even have to turn it on to know it would be good; you could tell from the case.
A few weeks ago I was walking along the street in Cambridge, and in someone's trash I saw what appeared to be a Mac carrying case.
I looked inside, and there was a Mac SE.
I carried it home and plugged it in, and it booted.
The happy Macintosh face, and then the finder.
My God, it was so simple.
It was just like ... Google.
Hackers like to work for people with high standards.
But it's not enough just to be exacting.
You have to insist on the right things.
Which usually means that you have to be a hacker yourself.
I've seen occasional articles about how to manage programmers.
Really there should be two articles: one about what to do if you are yourself a programmer, and one about what to do if you're not.
And the second could probably be condensed into two words: give up.
The problem is not so much the day to day management.
Really good hackers are practically self-managing.
The problem is, if you're not a hacker, you can't tell who the good hackers are.
A similar problem explains why American cars are so ugly.
I call it the design paradox. You might think that you could make your products beautiful just by hiring a great designer to design them.
But if you yourself don't have good taste [blocked], how are you going to recognize a good designer?
By definition you can't tell from his portfolio.
And you can't go by the awards he's won or the jobs he's had, because in design, as in most fields, those tend to be driven by fashion and schmoozing, with actual ability a distant third.
There's no way around it: you can't manage a process intended to produce beautiful things without knowing what beautiful is.
American cars are ugly because American car companies are run by people with bad taste.
Many people in this country think of taste as something elusive, or even frivolous.
It is neither.
To drive design, a manager must be the most demanding user of a company's products.
And if you have really good taste, you can, as Steve Jobs does, make satisfying you the kind of problem that good people like to work on.
Hackers also want interesting projects, and any application can be interesting if it poses novel technical challenges. People working on airline fare searches probably thought it the most boring application imaginable, until ITA made it interesting by redefining [blocked] the problem more ambitiously.
The same happened at Google. The conventional wisdom among the portals was that search was boring. But the guys at Google didn't think so, and that's why they do it so well.
This is where managers matter. A good manager can redefine a problem as a more interesting one — Steve Jobs did this through high standards. There were cheap computers before the Mac, but he redefined the problem as making one that's beautiful.
And they delivered. A few weeks ago I found a Mac SE in someone's trash in Cambridge; I carried it home, plugged it in, and it booted. The happy Macintosh face, then the finder. My God, it was so simple. It was just like ... Google.
Hackers like to work for people with high standards, but you have to insist on the right things, which usually means being a hacker yourself. There should be two articles on managing programmers, one for each case, and the second could be condensed into two words: give up.
The trouble is that if you're not a hacker, you can't tell who the good ones are. A similar problem explains why American cars are ugly; I call it the design paradox. Without good taste [blocked] yourself, how would you recognize a great designer? Not from his portfolio. American cars are ugly because American car companies are run by people with bad taste.
Many think of taste as elusive, even frivolous. It is neither. With really good taste you can make satisfying you the kind of problem good people like to work on.
Hackers want interesting projects, and almost any problem becomes interesting when someone redefines it ambitiously. That's why hackers like managers with real taste — but you can't recognize good people or good design without taste yourself.
Nasty Little Problems
It's pretty easy to say what kinds of problems are not interesting: those where instead of solving a few big, clear, problems, you have to solve a lot of nasty little ones.
One of the worst kinds of projects is writing an interface to a piece of software that's full of bugs.
Another is when you have to customize something for an individual client's complex and ill-defined needs.
To hackers these kinds of projects are the death of a thousand cuts.
The distinguishing feature of nasty little problems is that you don't learn anything from them.
Writing a compiler is interesting because it teaches you what a compiler is.
But writing an interface to a buggy piece of software doesn't teach you anything, because the bugs are random. [3] So it's not just fastidiousness that makes good hackers avoid nasty little problems. It's more a question of self-preservation.
Working on nasty little problems makes you stupid.
Good hackers avoid it for the same reason models avoid cheeseburgers.
Of course some problems inherently have this character.
And because of supply and demand, they pay especially well.
So a company that found a way to get great hackers to work on tedious problems would be very successful.
How would you do it?
One place this happens is in startups.
At our startup we had Robert Morris working as a system administrator.
That's like having the Rolling Stones play at a bar mitzvah.
You can't hire that kind of talent.
But people will do any amount of drudgery for companies of which they're the founders. [4]
Bigger companies solve the problem by partitioning the company.
They get smart people to work for them by establishing a separate R&D department where employees don't have to work directly on customers' nasty little problems. [5] In this model, the research department functions like a mine.
They produce new ideas; maybe the rest of the company will be able to use them.
You may not have to go to this extreme. Bottom-up programming [blocked] suggests another way to partition the company: have the smart people work as toolmakers.
If your company makes software to do x, have one group that builds tools for writing software of that type, and another that uses these tools to write the applications.
This way you might be able to get smart people to write 99% of your code, but still keep them almost as insulated from users as they would be in a traditional research department.
The toolmakers would have users, but they'd only be the company's own developers. [6]
If Microsoft used this approach, their software wouldn't be so full of security holes, because the less smart people writing the actual applications wouldn't be doing low-level stuff like allocating memory.
Instead of writing Word directly in C, they'd be plugging together big Lego blocks of Word-language. (Duplo, I believe, is the technical term.)
The uninteresting projects are the ones where, instead of a few big clear problems, you have a lot of nasty little ones — an interface to buggy software, or customizing for a client's ill-defined needs. To hackers these are the death of a thousand cuts.
What distinguishes nasty little problems is that you learn nothing from them: a buggy interface teaches you nothing, because the bugs are random. Working on nasty little problems makes you stupid — good hackers avoid it for the same reason models avoid cheeseburgers.
Some problems inherently have this character, and by supply and demand they pay especially well. So a company that found a way to get great hackers to do them would be very successful. How would you do it?
One place this happens is startups. At our startup we had Robert Morris as a system administrator — like having the Rolling Stones play at a bar mitzvah. You can't hire that kind of talent, but people will do any amount of drudgery for companies of which they're founders.
Bigger companies partition: they establish a separate R&D department where employees don't work directly on customers' nasty little problems. It functions like a mine, producing new ideas the rest of the company might use.
You may not need that extreme. Bottom-up programming [blocked] suggests another partition: have the smart people work as toolmakers, while others use the tools to write the applications. This way smart people write 99% of your code, plugging together big Lego blocks instead of writing Word directly in C. (Duplo, I believe, is the technical term.)
The dullest projects are heaps of nasty little problems that teach you nothing and make you stupid. Companies can still get great hackers to do them — through founding, R&D departments, or by making the smart people toolmakers.
Clumping
Along with interesting problems, what good hackers like is other good hackers.
Great hackers tend to clump together-- sometimes spectacularly so, as at Xerox Parc.
So you won't attract good hackers in linear proportion to how good an environment you create for them.
The tendency to clump means it's more like the square of the environment.
So it's winner take all.
At any given time, there are only about ten or twenty places where hackers most want to work, and if you aren't one of them, you won't just have fewer great hackers, you'll have zero.
Having great hackers is not, by itself, enough to make a company successful.
It works well for Google and ITA, which are two of the hot spots right now, but it didn't help Thinking Machines or Xerox.
Sun had a good run for a while, but their business model is a down elevator.
In that situation, even the best hackers can't save you.
I think, though, that all other things being equal, a company that can attract great hackers will have a huge advantage.
There are people who would disagree with this.
When we were making the rounds of venture capital firms in the 1990s, several told us that software companies didn't win by writing great software, but through brand, and dominating channels, and doing the right deals.
They really seemed to believe this, and I think I know why.
I think what a lot of VCs are looking for, at least unconsciously, is the next Microsoft.
And of course if Microsoft is your model, you shouldn't be looking for companies that hope to win by writing great software.
But VCs are mistaken to look for the next Microsoft, because no startup can be the next Microsoft unless some other company is prepared to bend over at just the right moment and be the next IBM.
It's a mistake to use Microsoft as a model, because their whole culture derives from that one lucky break.
Microsoft is a bad data point.
If you throw them out, you find that good products do tend to win in the market.
What VCs should be looking for is the next Apple, or the next Google.
I think Bill Gates knows this.
What worries him about Google is not the power of their brand, but the fact that they have better hackers. [7]
What good hackers like, along with interesting problems, is other good hackers, and they clump together. So it's winner take all: at any time there are only ten or twenty places where hackers most want to work, and if you aren't one, you'll have zero.
Having great hackers isn't by itself enough. It works for Google and ITA, but it didn't help Thinking Machines or Xerox. Sun had a good run, but their business model is a down elevator, and there even the best hackers can't save you.
Still, all else equal, a company that can attract great hackers has a huge advantage. Some disagree: in the 1990s several VCs told us software companies won not by writing great software but through brand, dominating channels, and doing the right deals.
A lot of VCs are looking, unconsciously, for the next Microsoft. But no startup can be the next Microsoft unless some other company bends over at the right moment to be the next IBM. Microsoft's culture derives from that one lucky break — a bad data point. Throw it out and good products do tend to win; what VCs should look for is the next Apple, or the next Google.
I think Bill Gates knows this. What worries him about Google is not the power of their brand, but the fact that they have better hackers.
Great hackers clump together, so attracting them is winner-take-all — but they aren't enough to save a doomed business model. VCs chase the next Microsoft when they should chase the next Apple or Google.
Recognition
So who are the great hackers?
How do you know when you meet one?
That turns out to be very hard.
Even hackers can't tell.
I'm pretty sure now that my friend Trevor Blackwell is a great hacker.
The remarkable thing about this project was that he wrote all the software in one day (in Python, incidentally).
For Trevor, that's par for the course.
But when I first met him, I thought he was a complete idiot.
He was standing in Robert Morris's office babbling at him about something or other, and I remember standing behind him making frantic gestures at Robert to shoo this nut out of his office so we could go to lunch.
Robert says he misjudged Trevor at first too.
Apparently when Robert first met him, Trevor had just begun a new scheme that involved writing down everything about every aspect of his life on a stack of index cards, which he carried with him everywhere.
He'd also just arrived from Canada, and had a strong Canadian accent and a mullet.
The problem is compounded by the fact that hackers, despite their reputation for social obliviousness, sometimes put a good deal of effort into seeming smart.
When I was in grad school I used to hang around the MIT AI Lab occasionally.
It was kind of intimidating at first. Everyone there spoke so fast. But after a while I learned the trick of speaking fast. You don't have to think any faster; just use twice as many words to say everything.
With this amount of noise in the signal, it's hard to tell good hackers when you meet them.
I can't tell, even now.
You also can't tell from their resumes.
It seems like the only way to judge a hacker is to work with him on something.
And this is the reason that high-tech areas only happen around universities.
The active ingredient here is not so much the professors as the students.
Startups grow up around universities because universities bring together promising young people and make them work on the same projects.
The smart ones learn who the other smart ones are, and together they cook up new projects of their own.
Because you can't tell a great hacker except by working with him, hackers themselves can't tell how good they are.
This is true to a degree in most fields.
I've found that people who are great at something are not so much convinced of their own greatness as mystified at why everyone else seems so incompetent.
But it's particularly hard for hackers to know how good they are, because it's hard to compare their work.
This is easier in most other fields.
In the hundred meters, you know in 10 seconds who's fastest. Even in math there seems to be a general consensus about which problems are hard to solve, and what constitutes a good solution.
But hacking is like writing.
Who can say which of two novels is better?
Certainly not the authors.
With hackers, at least, other hackers can tell.
That's because, unlike novelists, hackers collaborate on projects.
When you get to hit a few difficult problems over the net at someone, you learn pretty quickly how hard they hit them back.
But hackers can't watch themselves at work.
So if you ask a great hacker how good he is, he's almost certain to reply, I don't know.
He's not just being modest. He really doesn't know.
And none of us know, except about people we've actually worked with.
Which puts us in a weird situation: we don't know who our heroes should be.
The hackers who become famous tend to become famous by random accidents of PR.
Occasionally I need to give an example of a great hacker, and I never know who to use.
The first names that come to mind always tend to be people I know personally, but it seems lame to use them.
So, I think, maybe I should say Richard Stallman, or Linus Torvalds, or Alan Kay, or someone famous like that.
But I have no idea if these guys are great hackers.
I've never worked with them on anything.
If there is a Michael Jordan of hacking, no one knows, including him.
So who are the great hackers, and how do you know when you meet one? Very hard; even hackers can't tell. I'm pretty sure my friend Trevor Blackwell is a great hacker — you may have read how he made his own Segway, writing all the software in one day, in Python.
But when I first met him I thought he was a complete idiot, babbling in Robert Morris's office while I gestured frantically at Robert to shoo this nut out so we could go to lunch. Robert misjudged him too at first.
Hackers also sometimes put real effort into seeming smart. The MIT AI Lab was intimidating — everyone spoke so fast — until I learned the trick. You don't have to think any faster; just use twice as many words to say everything.
With this much noise in the signal, you can't tell good hackers when you meet them, or from their resumes. The only way to judge a hacker seems to be to work with him on something.
This is why high-tech areas only happen around universities: they bring promising young people together on the same projects, so the smart ones learn who the other smart ones are.
Because you can't tell a great hacker except by working with him, hackers can't tell how good they are. It's particularly hard because their work is hard to compare. Hacking is like writing: who can say which of two novels is better? Certainly not the authors.
With hackers, other hackers can tell, because unlike novelists they collaborate. But hackers can't watch themselves work, so ask a great one how good he is and he'll say I don't know. He's not being modest; he really doesn't know.
And none of us know, except about people we've worked with — so we don't know who our heroes should be. I'm tempted to say Richard Stallman, or Linus Torvalds, but I have no idea if these guys are great hackers; I've never worked with them. If there is a Michael Jordan of hacking, no one knows, including him.
You can't tell a great hacker except by working with him — so hackers can't even tell how good they are, and we don't really know who our heroes should be.
Cultivation
Finally, the question the hackers have all been wondering about: how do you become a great hacker?
I don't know if it's possible to make yourself into one.
But it's certainly possible to do things that make you stupid, and if you can make yourself stupid, you can probably make yourself smart too.
The key to being a good hacker may be to work on what you like.
When I think about the great hackers I know, one thing they have in common is the extreme difficulty [blocked] of making them work on anything they don't want to.
I don't know if this is cause or effect; it may be both.
To do something well you have to love [blocked] it.
So to the extent you can preserve hacking as something you love, you're likely to do it well.
Try to keep the sense of wonder you had about programming at age 14.
If you're worried that your current job is rotting your brain, it probably is.
The best hackers tend to be smart, of course, but that's true in a lot of fields.
Is there some quality that's unique to hackers?
I asked some friends, and the number one thing they mentioned was curiosity.
I'd always supposed that all smart people were curious-- that curiosity was simply the first derivative of knowledge.
But apparently hackers are particularly curious, especially about how things work.
That makes sense, because programs are in effect giant descriptions of how things work.
Several friends mentioned hackers' ability to concentrate-- their ability, as one put it, to "tune out everything outside their own heads.'' I've certainly noticed this.
And I've heard several hackers say that after drinking even half a beer they can't program at all.
So maybe hacking does require some special ability to focus.
Perhaps great hackers can load a large amount of context into their head, so that when they look at a line of code, they see not just that line but the whole program around it.
John McPhee wrote that Bill Bradley's success as a basketball player was due partly to his extraordinary peripheral vision.
"Perfect'' eyesight means about 47 degrees of vertical peripheral vision.
Bill Bradley had 70; he could see the basket when he was looking at the floor.
Maybe great hackers have some similar inborn ability. (I cheat by using a very dense [blocked] language, which shrinks the court.)
This could explain the disconnect over cubicles.
Maybe the people in charge of facilities, not having any concentration to shatter, have no idea that working in a cubicle feels to a hacker like having one's brain in a blender. (Whereas Bill, if the rumors of autism are true, knows all too well.)
One difference I've noticed between great hackers and smart people in general is that hackers are more politically incorrect [blocked].
To the extent there is a secret handshake among good hackers, it's when they know one another well enough to express opinions that would get them stoned to death by the general public.
And I can see why political incorrectness would be a useful quality in programming.
Programs are very complex and, at least in the hands of good programmers, very fluid.
In such situations it's helpful to have a habit of questioning assumptions.
Can you cultivate these qualities?
I don't know.
But you can at least not repress them.
So here is my best shot at a recipe.
If it is possible to make yourself into a great hacker, the way to do it may be to make the following deal with yourself: you never have to work on boring projects (unless your family will starve otherwise), and in return, you'll never allow yourself to do a half-assed job.
All the great hackers I know seem to have made that deal, though perhaps none of them had any choice in the matter.
Finally, how do you become a great hacker? I don't know if you can. But it's possible to do things that make you stupid — and if you can make yourself stupid, you can probably make yourself smart too.
The key may be to work on what you like. One thing the great hackers I know have in common is the extreme difficulty [blocked] of making them work on anything they don't want to.
To do something well you have to love [blocked] it, so preserve hacking as something you love, and try to keep the sense of wonder you had about programming at age 14. If you're worried that your current job is rotting your brain, it probably is.
Is there a quality unique to hackers? The number one answer from friends was curiosity — particularly about how things work, which makes sense, since programs are giant descriptions of how things work.
Friends also mentioned hackers' ability to concentrate, to "tune out everything outside their own heads.'' Perhaps great hackers can load a large amount of context into their head, seeing not just a line of code but the whole program around it. (I cheat by using a very dense [blocked] language, which shrinks the court.)
This could explain the disconnect over cubicles: the people in charge of facilities, having no concentration to shatter, have no idea that a cubicle feels to a hacker like having one's brain in a blender.
Hackers are also more politically incorrect [blocked] than smart people in general. And I can see why this would help in programming: programs are complex and, in good hands, very fluid, so it helps to have a habit of questioning assumptions.
Can you cultivate these qualities? I don't know, but here's my best shot at a recipe: make this deal with yourself — you never have to work on boring projects, unless your family will starve otherwise, and in return you'll never allow yourself to do a half-assed job. All the great hackers I know seem to have made that deal, though perhaps none had any choice in the matter.
You may not be able to make yourself a great hacker, but you can work on what you love, stay curious, and protect your ability to concentrate — and never allow yourself a half-assed job.
Notes
[1] In fairness, I have to say that IBM makes decent hardware. I wrote this on an IBM laptop.
[2] They did turn out to be doomed. They shut down a few months later.
[3] I think this is what people mean when they talk about the "meaning of life." On the face of it, this seems an odd idea. Life isn't an expression; how could it have meaning? But it can have a quality that feels a lot like meaning. In a project like a compiler, you have to solve a lot of problems, but the problems all fall into a pattern, as in a signal. Whereas when the problems you have to solve are random, they seem like noise.
[4] Einstein at one point worked designing refrigerators. (He had equity.)
[5] It's hard to say exactly what constitutes research in the computer world, but as a first approximation, it's software that doesn't have users.
I don't think it's publication that makes the best hackers want to work in research departments.
I think it's mainly not having to have a three hour meeting with a product manager about problems integrating the Korean version of Word 13.27 with the talking paperclip.
[6] Something similar has been happening for a long time in the construction industry. When you had a house built a couple hundred years ago, the local builders built everything in it. But increasingly what builders do is assemble components designed and manufactured by someone else. This has, like the arrival of desktop publishing, given people the freedom to experiment in disastrous ways, but it is certainly more efficient.
[7] Google is much more dangerous to Microsoft than Netscape was. Probably more dangerous than any other company has ever been. Not least because they're determined to fight. On their job listing page, they say that one of their "core values'' is "Don't be evil.'' From a company selling soybean oil or mining equipment, such a statement would merely be eccentric. But I think all of us in the computer world recognize who that is a declaration of war on.
Thanks to Jessica Livingston, Robert Morris, and Sarah Harlin for reading earlier versions of this talk.
In fairness, IBM makes decent hardware — I wrote this on an IBM laptop. And the doomed NT startup did shut down a few months later.
The "meaning of life'' may be this: in a compiler the problems fall into a pattern, like a signal; when the problems are random, they seem like noise.
Research, as a first approximation, is software that doesn't have users — and what draws the best hackers there isn't publication but not having to sit through a three-hour meeting about integrating the Korean version of Word with the talking paperclip.
Google is much more dangerous to Microsoft than Netscape was, partly because they're determined to fight. From a company selling soybean oil, "Don't be evil'' would merely be eccentric; in the computer world we all recognize who that is a declaration of war on.
Notes on the talk: IBM hardware, the doomed NT startup, the "meaning of life" as solving problems that fall into a pattern, and why Google's "Don't be evil" reads as a declaration of war.