Guerrilla Guide To Interviewing

http://www.joelonsoftware.com/articles/fog0000000073.html

Introduces one manager's techniques for job interviews. Includes the following list of question categories to use:

I believe you can find out more about a person by letting them tell their story than by asking them to answer your questions. Being good at discerning the subtle can not be replaced with a formulaic set of test questions. Hiring a programmer is a serious business. That person generally must not only be technically adept, but they must also work well in a team environment. You can train a curious, sincere and friendly person to write good code better than you can train a highly-strung hypergeek to get along with people (I would prefer a well-trained and talented friendly person). Believe me, I have seen many egotistical geniuses foul up working relationships through ridiculous hubris. I look more for curiosity than passion. Passion burns out - creates conflict. Curiosity is relaxed - it sustains and perseveres. If I had to ask one question, it would be: "O.K. Could you ask me some questions about this company?" From this, I am quite confident that I could find all the information I needed to begin the decision process. If what I heard sounded intriguing, I might hire the candidate on the spot. If what I heard sounded promising, I might call them in for a second interview. With application of this general approach, I have never failed to hire good programmers. Letting a person tell his story in a relaxed atmosphere is also a surefire way to spot a potential wacko. When they get that special gleam in the eye and talk about how they decided to join the militia, better look at your watch and remind them you're late for a dentist appointment. ;=) -- KirkKitchen.

Amen to that. Everyone seems to be looking for a magic set of questions, or types of questions that will shortcircuit the need to listen to the people. I find that I get good results by asking people their opinions about software-related topics - Java, Patterns, etc. I don't care what those opinions are, but I expect good people to be aware of recent trends, and to have thought about them. -- PeteHardie

In other words, you have a standard set of questions around asking what their opinions are, and you have an idea on what kinds of answers you would like to hear. How is that different? (I don't think "everyone" is trying to avoid listening. Having standard questions helps in structuring the interview, and helps arrange the right kind of opportunities to listen. -- PaulHudson

Having standard questions leads to having standard answers, which leads to interviewers not listening except for the expected correct one. Like MicrosoftsManholeCoverQuestion, there can be lots of answers that are as good, and many that are trite, but usefully correct that might be taken as wrong. I prefer to ask questions that let the interviewee explain how they think - not ones that make them sweat to find a clever correct answer to show off their creativity, nor ones that force them to risk alienating a prospective boss by contradicting him/her. -- PeteHardie

Having standard questions leads to having standard answers, which leads to interviewers not listening except for the expected correct one.. I'd be interested in your justification for this statement, since I can't see why it holds, and also why it doesn't apply to the questions you ask.

I worked for a company which always asked standard questions. Their legal department had told them that unless everyone was asked exactly the same questions, this could lead to claims of unfair treatment. It was possible to set out procedures like "if the candidate has C++ experience ask them questions A B and C" but these had to be followed to the letter. This meant, for example that when one candidate said they had extensive Smalltalk experience, but no Java, and this was not anticipated, they had to be asked Java questions but the interviewer could not expand on their smaltalk experience. Since no other company has done this, I can only assume their legal department was wrong!

I have a standard question ("please attempt to write a program that reverses an array in a language of your choice"). I point out this is under specified, that I'm less interested in the actual code than in how they got there, then I listen to them as they work. In what way is that looking for a "standard answer" and "not listening"?

I agree you want the interviewee to explain how they think. I just don't see why that cannot happen if you use standard questions. I use a reasonable standard set of questions. I get a whole range of answers, and I listen to them all.

I wouldn't do the contradict question in the form expressed above - but I might ask them to justify an answer further by asking them what they would do if someone - perhaps a customer - expressed the opposite point of view. No need to be antagonistic, but also no need to avoid asking them why they hold a particular view.

I see nothing wrong with asking them how they would approach answering a question like the gas stations one. The point isn't to get a "correct" answer in terms of an actual number. It's exactly to let them explain how they think. -- PaulHudson

I've been trying to resist stirring trouble, but this really gets right up my nose. The way to get to know me and my skills is certainly not to put me in a room and see how I react to trick questions (see below), or ask me to do strange things and compare my reactions to a "what creative people usually do" checklist. How patronizing! "Do they use pointer arithmetic? This is a good sign." I say this personally as JustaProgrammer, but I don't presume to know whether this technique would "work". -- LukeGorrie

I have to disagree with Luke here. This is, in fact, one of the best guides to interviewing that I've seen - even if you think pointer arithmetic is generally a bad thing (I am with Luke on that one point). -- CurtisBartley

From my experience, an interviewee is never going to produce the best code they could. I code in an environment - man pages, my reference books, other programmers - asking me to code anything not absolutely trivial in less than 1 hour is guaranteed to result in quality lower than my normal mode. Ditto creativity - I'm creative when comfortable, not when in a suit being grilled by HR droids. And for asking trick questions (see below) and being deliberately argumentative, just to test my reaction', I would suggest that verges on ambush, and would make me question the trust I could place in such an employer -- Pete Hardie

I have to agree with Luke. The questions sound horrible, at least for most jobs. At an interview, you want to find out how well the person will work in your job environment, with your people, your customers, your product, etc. Asking trick questions (see below) doesn't shed much light on that, unless figuring out how many gas stations are in Los Angeles is part of the job. And the implicit premises sound simplistic and wrong. For example, often a very wise way to deal with someone who is arguing against truth is to give in. This doesn't show that you're weak, it shows that you know better than to argue with a bully. -- BenKovitz

An interviewer using these techniques on me would fail MY test. I have worked for unstable people in the past, and will never knowingly do so again. So when my potential future boss starts acting goofy, I "no hire" myself. I have had seemingly rational people turn out to be wacko but I have never yet had a seemingly wacko person turn out to be rational, and I do not plan to try further experiments -- KenMegill

I agree with Ken. In a job interview, the candidate is also interviewing the company too. If I am asked impossible questions or the challenge, how can I know if you are just testing me, or if that is how you work day to day? I will mostly likely assume that if I joined the company, you will be asking more impossible questions and argue simple facts with me. How can I work with someone like this? I think I will probably reject any offer from such company. -- OliverChung

In our case, I don't think we'd be making you an offer. If you can't tell the difference between an interview situation and a working situation (or can't see why there has to be a difference), you're not the kind of person we'd want

Interesting... I believed the best way to show that you can do the job is to do it. See this column by Nick Corcodilos in EE Times http://eet.com/story/ask_the_headhunter/OEG20010702S0083. I have tried this "Do the job in the interview" approach once in a interview with developers, and it worked amazingly well. By the end of the interview, I already feel like I am part of the team, and I know I will have an offer (and I did get one). My point is, artificially distinguishing between interview situation and work situation is not a good way to find good candidate, there are a lot of people (salesmen?) who are good at interviews but bad at work, and vice versa. Making the interview more like work will let people who are good at work show you how good they are. -- OliverChung

I've been using this as a guide to interviewing and, actually, I like it.

We've never done "the challenge," not because we don't want to, but because I think we're afraid to.

Of course, no-one is going to write their best code on an interview. But that's why you interview more than one person, so you can compare. And you make the question fairly straightforward. When you point out a bug, do they fix the algorithm or patch in a special case? When they write their code do they agonize over variable names? Do they ask for help (it's very important to know when to do that)?

The creative/impossible questions may sound silly, but it's amazing to me what you find out about people when you ask them. Do they think it might be fun? A challenge? Do they query for information? Again, do they ask for help? Or do they give up right away?

The people who do the interviews with me aren't HR droids, they're either other developers or client solutions managers that development has to work closely with.

Another thing I like about the Guerrilla guide is the idea of getting people to boil down their thoughts into hire/no-hire, and to write them down without consulting anyone else. Groupthink/political pressure is a huge risk with interviewing, and I think this is a good approach.

I never ever try to pressure people. If they seem nervous, I try to get them to relax. I tell them the important thing is the process, not the answer. (And that's true.) -- StephenNg

trick questions?

In response to the angst above over trick questions, ... go read the link, and you'll see that Joel says "avoid brain teaser questions like the one where you have to arrange 6 equal length matches to make exactly 4 identical perfect triangles", those which involve a high factor of ah-hah!. He recommends simple logic/math/procedural problems. He also totally disdains questions which are best answered by references (eg. "What's the difference between varchar and varchar2 in Oracle 8i?).

[Hmm. The matches question took me 10 - 15 seconds. I guess I fail... especially after pointing out that the 'difference' is '2'! -- vk]

A contrasting style of "Guerrilla Interviewing," from an article titled "Nerd Herding" on http://freshmeat.net/ by Cal Evans:

Pre-Screen 
  :
Sometimes over the phone, sometimes over lunch (especially if there was a recruiting agency involved), I would meet with each potential candidate. I wasn't looking at skills at this point; I wanted to get to know the candidate. Does he have the personality that would mesh with the rest of the herd? Can she communicate ideas clearly and concisely? Can she talk fluently about the technologies we are deploying? Most candidates made it through my initial pre-screen. It was just my one and only shot at filtering out candidates by myself that I didn't think fit the mold. From this point on, I was just another vote. From here on out, it was a herd effort.
The Interview from Hell
One of my team members came to me one day and thanked me for hiring her early in the herd building process. She said she was sure that she wouldn't have survived the current interview process! The main interview was with the entire herd. It was as 'no-holds-barred' as Nerds get. We had some pretty heavy-hitting senior Nerds on our herd, so if you put something on your resume, you had better know it. There was only one rule (aside from the 100 HR rules for interviewing): no trick questions.
As the herd grew in size, my active role in the interview diminished. I spent more time watching how the candidate handled the pressure of the interview and trying to get a feel for the candidate's personality and how it fit in the herd. Assessing these points was my assignment, while the rest of the herd picked them apart technically. It was very important to me that the candidate's personality was a good fit. In any herd, and especially those assigned to high-visibility or high-pressure projects, the members must be comfortable with each other, must be able to learn to trust each other, and yes, must like each other. Personality was 50% of the grade of any interview.
The De-Briefing
Once the interview was over, usually within 30-40 minutes, we would all get back together to discuss what we saw. Instead of going around the room in clockwise fashion (way too regimented), I tried to get the most junior members to speak first. This helped avoid the 'me-too' syndrome from the juniors and forced them to think through their opinions.
The most important aspect of the entire interview process was that everyone in the herd got a veto. If anybody gave a thumbs down, the candidate was no longer viable. Yes, this meant that sometimes we passed on a talented candidate. It pained me to let this happen, but to override the process and take the candidate anyhow would have damaged my credibility, and who said I was a god at hiring anyhow? If someone saw something I didn't, they deserved to be taken seriously. The damage I would have done to herd morale by overriding the process would outweigh any benefit that the new herd member would have brought. Trust me, once your herd understands you are serious about this rule, they will think long and hard about any decision to veto. I never had a situation in which a single person vetoed a candidate.
The outcome of this process was two tight-knit Nerd herds. They worked well together, they played together, they understood each other, and they respected each other. Of course, they fought with each other, called each other names, and played jokes on each other, too. It was grand!

The entire article, which has some interesting views on software management, is located at http://freshmeat.net/articles/view/157/.

Oh boy, playing together, being gang-raped by 20 people at an interview, hey, this sounds like a great place if you are one year out of school, otherwise forget it.

I don't play the tricks, but I do do some of the other things. It's worked pretty well for me Exposing the - very large - number of candidates who couldn't program in C if they were paid to - which is of course what we are proposing to do :-), discovering whether people will listen to, and think about, what they're being asked to do, or whether they just implement without thought whatever they're told to do (lots of those, too).

These are largely recent graduates, often from IT conversion courses, but the tests have value for developers with several years experience.

I don't care about good, efficient, code. I do care that the interviewee asks for the additional information needed to specify the problem better (I deliberately under-specify it, and tell them so), that they have some idea how to test whether their produced code works. I ask them to verbalize their current thinking to give some idea of their approach to problems.

Good candidates produce code that would work. Great candidates can do it with pointers and arrays, and can test it, and have ideas on how to make it faster. Many candidates can't get anywhere, especially with pointers. About 10% of people refuse to make an attempt, saying it's too difficult.

So, for the nay-sayers, what would people suggest as alternative interviewing techniques? I do (did) want reasonable C programmers, who can solve problems, who can clarify ambiguous requests and so on, and time doesn't permit us to spend a day on each candidate.

-- PaulHudson

Much as I personally don't like trivia questions, like the "how many gas stations" question, I think all the questions are good, except "The Challenge," which is clearly quite worthless:

How is a candidate who doesn't know you supposed to know if the ridiculous stupid obviously false statement you just made is an invitation to argue or one of your most deeply held beliefs?
Don't laugh: I'm serious. If you use "The Challenge", you're taking a big risk. -- JeffGrigg

I'd suggest that trick questions and the "challenge phase" of the interview be prefaced by some statement like "OK, now we're going to ask you some questions to see how you handle them." Then the interviewer is more likely to see the candidate use problem-solving skills, rather than getting-along-with-a-clueless-idiot skills. -- KrisJohnson

The intelligent answer to the gas station question is to call the appropriate regulating authority and ask. But I guess whoever gives the intelligent answer would fail.

Why would that be a failure? I'd like to get that answer

If you ask "how many cars did GM sell last year," and the candidate thinks to call GM and ask them, then you're looking at a person who knows how to get correct answers rather than guess at them. I'd find it hard to call that a failure. We call this kind of person "resourceful." Of course, I can always ask them another question.

Another tack to take is to reply with a question: "How accurate do you need the answer to be?" or "Why do you need to know?" - both indicate that the candidate is thinking of the reasons for the question. In other words, there are some good stock responses that will invalidate the usefulness of such an open-ended question -- PeteHardie

That doesn't invalidate the usefulness at all. If the candidate is thinking of why I'm asking, asking for clarification in the way you describe, that's great. I'll reply with "let's try for plus or minus 50%", or "because I want to see how you'd go about finding the answer".

Interviewing isn't a win-lose game; the aim is for win-win, and both sides should be trying for that. "Win-win" includes ending with a job offer, but it also includes ending with a joint realization that there's an inadequate fit between person and position.

There's no gain in the interviewer trying to trick the interviewee, and there's no gain in the interviewee doing this either. So, querying why a question is being asked is fine, but at some point you're going to have to answer something. You're being given the opportunity to demonstrate what you can do, so do it.

When I ask a question, I may have an answer in mind - the answer I would give to that question. People need to realize that if you give a different answer than what I expect, your answer isn't automatically "wrong." You may have given a very good answer. Maybe both answers are good answers to the question.

-- I expected somebody to have said this already, but since no one did, here it is: "There are 479 gas stations in LA. If you don't believe me, prove me wrong."

Re-read two paragraphs up. There's no gain in attempted trickery, on either side

The thing to remember is that 85% of the various tricks that are used to cause somebody to expose their personality are known by most of the candidates. Especially logic puzzles. People trade stories of logic puzzles and weird interview tricks. I know I've done it. Or your candidate could have recently taken a class on the area of math that your logic problem covers. Or they could like logic puzzles and work on them on their own.

I will say that every person has a different, perhaps valid, way of determining if they would want to work with somebody. One person may want somebody who's not condescending, so they make them explain something in painstaking detail. One interviewer I chatted with will ask you deep, complex questions that cut to the core of one skill or technology you have on your resume. There is no right way to interview. -- KenWronkiewicz

Having just last week taken a short class on interviewing (from the hiring side), I'll take exception to '''The Challenge" above - the one thing that was stressed in the class was that you want the candidate to be relaxed, because people who are not relaxed do not give you their best response. Presenting a confrontational situation will cause the candidate to tense up, and then they are likely to either lose concentration, or go into "tell them what they want to hear" mode. -- PeteHardie

Maybe what you should do is keep them relaxed for the first interview, and then stress them in the second one. I'd sure like to know that my co-workers can handle a stressful situation if it comes up, and I'm getting sick of the guy next to me yelling every time someone mentions there's a bug in his code.

I have become more demanding in interviews just because I am tired of people trying putting bs on their resumes that they really don't know. I'm not trying to jerk people around, but I don't appreciate being lied to either. -- DavidHurt

I have a standard question ("please attempt to write a program that reverses an array in a language of your choice").

So I answer 

  (setq reversed-array (reverse array))
And proceed to tell the interviewer that I'm interested in writing large, complex, robust systems, and not in micro-efficiency. Do I immediately get hired? Or rejected? And why? -- AlainPicard

I hope you don't immediately get either hired or rejected. If I were interviewing you and you did that, I'd ask you (1) "so what do you do if this is in an inner loop and you discover that the implementation you're using conses on (reverse array)?" and (2) "please implement such-and-such another thing instead", choosing some other thing that's not just a single function call away in CL. ... Well, actually, if I were interviewing you I might hire you immediately on the grounds that people with enough sense to prefer CL are rare. :-)

I should darned well hope that the implementation I'm using conses in (reverse array) - it's defined to return a new array. nreverse might be less noisy, though. -- DanBarlow

As the original contributor of the "reverse array" test, I'd have to say that the anonymous comment above is spot on. I've been waiting for the array.reverse() (and the CL equivalent would be fine) as an answer for a long time, but I've not had it to date. I'd regard that kind of reply as a positive sign, but certainly not enough for an immediate hire. I'd then follow up with option 2 (something that wasn't just a function call away), since one of the reasons for asking is to see if they do know how to do "micro-efficiency" if required. -- PaulHudson

[My first thought was (and would be) array.reverse() but in an interview context I might assume you were looking for something more "technical" and write one by hand. As a lot of stuff on this page points out, it's pretty hard for the interviewer and the interviewee to be going in with the same expectations. That's one reason I really detest trick questions or confrontational arguing as an interview technique.]

In a recent interview, I had a development manager argue with me about whether it was better to use a "for" or a "while" in a particular situation - he thought the "for" might produce less efficient code for some compilers. I didn't argue, but it did make me wonder whether I wanted to work for someone who was that fixated on such low-level matters. There were no questions at all about high-level design or work habits, only little exercises that could be solved with a dozen or fewer lines of code, followed by commentary from him on how various compilers would generate object code. I played along, but the bozo bit was flipping. -- KrisJohnson

I tend to ask people to self assess, then find something they claim to know well that I understand. I'll then try to ask a few questions appropriate to their claimed level of ability. Eg, for a self-assessed expert in C - please contrast static and file scope variables.

If they can't keep up, I downgrade their self-assessment across the board.

Think of it as statistical sampling.

Statistical sampling is a great idea, but one could easily make mistakes. For example, the above question basically boils down to arcana of different kinds of global variables. If someone used C, but never globals, they'd be downgraded unnecessarily.

This could work to the benefit of someone who is conservative in what he claims about himself. For example I once went to an interviewer and claimed that I knew more than 90% of people about C and 75% about using UN*X. To my angst, they had me take a computerized test in both of those subjects, on which I scored in the 99th and 98th percentiles respectively. I got that job. Comments on this page suggest that many of these techniques are useful for smoking out candidates who are, er, blowing smoke, but how about those who either are too modest or don't really know enough of the universe to compare themselves to the market? -- MichaelDavis

"Impossible Question" to me is a big warning that the company is dangerous. They follow trendy interview techniques. They probably do similarly stupid things with HR and happy-clappy meaningless meetings. They think "Microsoft does it!! It MUST be a good thing!" It's cargo cult HR. It really puts me right off companies. You do this, you better mentally add 20% to the salary offer.

"The Challenge". That just says I don't want to work with that person. "weak people fold". Yeah. Or the ones who just don't feel like having childish arguments. If I want a stupid childish argument, I know where the nearest infants school is. I don't go to work to have pissing contests with people.

I know plenty of quiet, pleasant people who are excellent developers who'd simply leave the interview at that point. Whether you want them hired or not kind of depends on whether you want a development team that makes software or punch-ups I guess. Far too many places seem to want to hire shouters; The sort of people who will have fights over indenting styles rather than be rational.

I have worked in too many places with too many people who will spend half a day berating you for not believing they can design a perpetual motion machine. I'm just not working with those people anymore, and asking "The Challenge" in an interview tells me you want to hire those sorts of people. Or already have them.

--KatieLucas

CategoryEmployment

EditText of this page (last edited August 21, 2007) or FindPage with title or text search