Seibel: As a programmer, do you consider yourself a scientist, an engineer,an artist, or a craftsman?

Norvig: Well, I know when you compare the various titles of books and soon, I always thought the “craft” was the right answer. So I thought art was alittle pretentious because the purpose of art is to be beautiful or to have anemotional contact or emotional impact, and I don’t feel like that’s anythingthat I try to do. Certainly I want programs to be pretty in some ways, andsometimes I feel like I spend too much time doing that. I’ve been in aposition where I’ve had the luxury to say, “Gee, I have time to go back andpretty this up a little bit.” And places where I’ve been able to write for apublication, you spend more time doing that than you would if it was justfor your own professional growth.

But I don’t think of that as art. I think craft is really the right word for it.You can make a chair, and it’s good looking, but it’s mostly functional—it’s achair.

Seibel: How do you recognize a great programmer, particularly whenyou’re hiring? You guys have hired a lot of programmers and you obviouslytry to hire really good programmers. How do you do it?

Norvig: We still don’t know.

Seibel: Google is somewhat famous for asking puzzle questions ininterviews. Do you think that’s a good approach?

Norvig: I don’t think it’s important whether people can solve the puzzlesor not. I don’t like the trick puzzle questions. I think it’s important to putthem in a technical situation and not just chitchat and get a feeling if they’rea nice guy. Though it is important to have someone that you can get alongwith. But you really have to see, can they technically do what they said theycan do. And there are a lot of different ways to demonstrate that. Andmany times you can see it from the résumé. I think our best signal is ifsomebody has worked with one of our employees before and the employeecan vouch for them. But we still try to draw it out on-site during theinterview. It’s more you want to get a feeling for how this person thinks andhow they work together, so do they know the basic ideas? Can they say,“Well, in order to solve this, I need to know A, B, and C,” and they startputting it together. And I think you can demonstrate that while still failing ona puzzle. You can say, “Well, here’s how I attack this puzzle. Well, I firstthink about this. Then I do that. Then I do that, but geez, here’s this part Idon’t quite understand.” For some people that little part clicks and for someit doesn’t. And you can do fine if it doesn’t click as long as you’vedemonstrated the basic competency and fluency in how you think about it.And then you really want to have people write code on the board if you’reinterviewing them for a coding job. Because some people have forgotten ordidn’t quite know and you could see that pretty quickly.

Seibel: So is that just a negative indicator? If they can’t write reasonablecode, that’s a bad sign. But if they don’t stumble over that hurdle, it’s hardto tell whether they’re actually going to write really good code in a largercontext.

Norvig: Right. You could tell to a certain degree but at other levels youcan’t. And we’ve studied this pretty carefully because we’ve gotten a lot ofapplications and we look at it at two levels. One, we say, from all therésumés we get, are we interviewing the right set of people? And then, fromthe interviews we get, are we hiring the right set of people?

Seibel: So how do you measure that? You don’t know about the one youdidn’t talk to or didn’t hire.

Norvig: Yeah, so that’s hard. At both levels, you’ve only got half yoursample, so it’s this biased problem, but I guess basically what we’re doing issaying, “Of the people that we interviewed and did well, what did theirrésumé look like,” and try to find more of those. Is having so many years ofexperience important? Is working on an open-source programming projectimportant? How does that compare to winning a programming contest?

Seibel: Do you really take all these things and shove them into a database?

Norvig: Yes, we do, and when we’re doing the hiring, we have these scoresthat come up that say, “The résumé predictor says such and such, and theinterview predictor says such and such.” We don’t take them as gospel, butthey’re just another piece of input along with all the other feedback wehave.

Seibel: Do the people who are doing the interviews have those numbersbeforehand?

Norvig: No, we only see that when they’re in the hiring committee, oncewe’ve gathered all the feedback. One of the interesting things we found,when trying to predict how well somebody we’ve hired is going to performwhen we evaluate them a year or two later, is one of the best indicators ofsuccess within the company was getting the worst possible score on one ofyour interviews. We rank people from one to four, and if you got a one onone of your interviews, that was a really good indicator of success.

Seibel: But you had to do well enough on something else that you actuallygot hired?

Norvig: Right, so that’s the thing. Ninety-nine percent of the people whogot a one in one of their interviews we didn’t hire. But the rest of them, inorder for us to hire them somebody else had to be so passionate that theypounded on the table and said, “I have to hire this person because I seesomething in him that’s so great, and this guy who thought he was no goodis wrong, and I’ve got to stand up for him and put my reputation on theline.”

Seibel: So you’re surrounded by top-notch programmers here at Google.Given how pervasive computers and software are in our society, do youthink everybody needs to understand a bit about programming just to getalong in or understand the world they live in?

Norvig: You probably want an educated person to understand howsoftware is made to the same degree they understand how a car is made.The other interesting thing is, how much does an informed citizen have tobe a programmer? Certainly the average person now can do wordprocessing and many of them can do spreadsheets, and so if you’re a littlebit experienced with spreadsheets, you’re starting to be a programmer.

Lots of attempts at end-user programming, and programming for everyone,haven’t been very successful. I don’t know how easy it is. Is there a way ofthinking that people have that we’ve gotten all the people that it’s easy toteach, and the other ones are going to be really hard, or is it just that we’vemissed the model and there’s some simple model in place of many peoplethat could influence programming if we created it?

Seibel: A lot of people that I have talked to for this book, and elsewhere,got into computers both because they just enjoyed it and because they feltlike it would change the world. Some of the folks I talked to for this bookdid that in the past and now are depressed by how little they feel the worldhas changed as a result. How do you feel about that?

Norvig: Well, I’m in the right place. We have hundreds of millions of usersand we can make a difference for them, and we can launch new services forthem quickly. I think that’s great. I can’t imagine anything else I could bedoing to have that level of impact.