Interviewing Strategy

Problem: You want to hire a software engineer. Most of them are babes in the wood or bullshit artists.

You ask them the following questions

These questions will tell you a lot. A babe in the wood won't understand many of the questions. A bullshit artist will answer "It depends," and grin "knowingly". Another kind of bad prospect will be opinionated but can't defend their arguments cogently. A good prospect might say, "It depends," and then set up contrasting scenarios.

If they have answers, and good reasons, for each question, then, even if each answer is exactly the opposite of what you believe, you've probably found a great prospect.

I'm sort of wondering how an interviewer can go into an interview and expect to figure out the value of a candidate based on simple rules on how questions are answered. Like trying to assign correct answers to a Rorschach test. --JeanPhilippeBelanger

The followup Why and How questions are the key. If an engineer cannot answer most of the Why or How questions, it usually means he rarely ask Why or How himself, and that kind of engineer is best avoided.

But of course this assumes you want to hire a good engineer instead of one that simply do what was told.

Which is a question that I find corporations are struggling with. A lot of companies want a flock of sheep behind a few people who drive the company. They actually put value on "won't cause any trouble"--JeanPhilippeBelanger

An approach I've used with success is to have a four-hour PairProgrammingTestDrive, on production code, as the final interview. The two times I've done this have been very successful at predicting how the person performed on the job. --JimLittle

Did you ever reject anybody ? I'm wondering about the ways people fail this. --JeanPhilippeBelanger

Yes, the first time I did it, there were two interviews for one position. The first guy didn't show strong knowledge of Java, but a good grasp of OOP. The second guy showed strong Java, but not strong OOP. So I hired the first guy. It's not so much about pass or fail; it's about getting a feel for what the person can do in a real-world environment. --JimLittle

Candidate One

I agree with JimLittle that PairProgramming exercises tell more than interview questions. Being articulate about coding issues definitely correlates with the ability to write software, but it's not as perfect as just giving the person a test drive. But I'll try anyway. -- SteveHowell

I think that this is a good question for a language designer, but for an applications programmer, you should allow him to redirect a bit, and talk about the pitfalls and advantages of the type systems in the particular languages that he works in.

I like the idea that objects either support methods or they don't. You don't care about the object's type, as long as it supports the operation you want to do on it. Python is good for this. You don't declare the type of a variable, but if you call an unknown method on it, Python throws an exception. You lose a bit of compile-time checking in Python, which annoyed me on a recent project, but the run-time checking is fine. In practice, if you mistakenly pass in an object of the wrong type to a function, you almost always end up trying to do something on the object that leads to a run-time exception, so subtle type-safety bugs are rare as long as you test your code.

Commenting should be minimal. Always say things in code, if you can. Code should clearly convey the rational structure of your software, but comments can convey emotional things. My favorite comment: 

  // XXX - I am tired, there's gotta be a better way.
Of course, that kind of comment should be short-lived. :)

None. They create nothing but misinformation. People convey architectures better than documents. People train users better than documents do.

After I wrote code, I invest all my efforts in getting other people to understand what I've done. I do this face-to-face whenever possible. If you are releasing a product to the public, you want to support users with mailing lists and encourage responsible evangelism, so that your product eventually can get an O'Reilly book.

The best document artifact I've ever encountered, actually, is a video. Video is a great way to bring a piece of software to life. Turn on the camera, and let the engineer explain what he did.

If you create a text document, it should be a LivingDocument like a Wiki.

Exceptions are good. Java, C++, Perl, and Python all use them for a reason. They separate error handling from normal processing. Clean-looking code is important. I have written identical programs in both Perl and Python, where the cleanliness of the Python code created real maintenance advantages, and exceptions provide the same sort of thing. Exceptions allow you to clearly document where you are handling errors, and where you are deferring error-handler to functions higher on the stack.

Follow Up

Would I Hire You? 

  Yes [3]
  No  [1]
Good idea. No's should explain why they wouldn't hire me. :) -- SteveHowell

The no was because the dogmatic position against commenting/documentation is not acceptable outside an XP context which is really very constrained and makes a lot of assumptions that are not applicable to most projects. Thinking people are available and even remember what they did strikes me as very naive. If the documentation is disinformation it should be a bug and be fixed. No different than any other flaw. If you had said that i am pro this kind of naming, etc then we could have got some where. But your attitude otherwise makes it unlikely you could fit in a project that wasn't run exactly to your taste.

The question wasn't, "What would you document?", it was "What do you like to document?". If interviewers pay this little attention to the way they word things and then are this critical -- yeesh. Let's just hire people by the haircut they have. -- NotThePersonWhoAnsweredTheQuestions

I guess my answer of "None" seems pretty dogmatic, but then I offer three alternative solutions to standard documentation procedures, which are to do more pure person-to-person technology transfer, use video, and use Wikis. You're right, though, about working outside of an XP context; the more I learn about XP, the more sense it makes to me work within the context of XP, not outside it.

When you use living software and don't understand it, you can almost always track down people who know the software better. If some one didn't introduce you to the software, how would you be using it? If a project was completely abandoned, shouldn't you just kill off the software, or study the software at that point to create the documentation?

-- SteveHowell

Candidate Two

Hey, can other people play too?

Manifest versus latent types

Latent, for preference. The big advantage is agility; it's much easier to change things in a latently-typed language. A smaller but non-trivial advantage is that latent typing makes for shorter programs because you don't have to keep typing type names. That one mostly goes away, though, if you compare with languages like Haskell and ML (with sophisticated type inferencing) rather than with languages like C and Ada (with no type inferencing at all).

The obvious drawbacks are: (1) type checks happening at run time lead to slower programs, and (2) manifest typing lets the compiler catch your errors. My experience is that (1a) slower programs often don't matter if you get to develop them faster, (1b) with type inference (especially if you have optional manifest types) the compiler can often do away with many of the type checks and give you C-like speed, (1c) on the rare occasions when you need speed and the language can't give it to you, most languages have a decent FFI that will let you call things written in C; and (2) I don't seem to spend more time debugging when I write in latently typed languages.

Commenting style

Lots of block comments in the nearest I can manage at the time to clear prose. Few in-line comments; they generally serve as reminders rather than explanations. Particularly important things to document: entry and exit conditions ("contracts", if you like) for functions/methods, loop invariants, intended use of classes and other data structures, sophisticated algorithms.

Usually, the hardest things to find just by reading the code are the highest-level ones. Therefore, higher-level information is usually better than lower-level information.

Documentation artifacts

Whatever the end-users need. This varies enormously between applications. About the only things that can be said consistently: this documentation should be clear, approachable, and complete.

Documentation for other developers: as much as possible should be in the source code - implicitly in the code itself, or explicitly in the comments. It may be sensible to have a higher-level design overview in an entirely separate file. (Or, for large projects, a hierarchy of such documents, which should not be too deep.)

I like to generate documentation automatically from the code, so that the same files generate nicely printed documents and hypertext and the executable itself.

Exceptions

Exceptions are good to have in your language and often bad to have in your code. They're like democracy: the worst solution, apart from all the others.

I think exceptions work better in languages with more automatic resource management. Python or Common Lisp rather than C++; C++ rather than C (of course C doesn't have exceptions as such, but you can cobble together a simple exception system out of macros and longjmp). The reason is obvious: in the presence of exceptions most guarantees about control flow go out of the window and you don't want to be depending on having resources deallocated explicitly.

But the alternatives to exceptions are (1) managing what's effectively the same non-local control flow built piece by piece out of local constructs - which really hurts; or (2) pretending exceptional situations will never arise - which is the industry-standard approach, alas.

Follow Up

[2] Hire
[0] No hire

Follow-up questions (even though I'd probably already hire you):

I like your exceptions-are-like-democracy analogy.

-- SteveHowell

Followup answers:

What if users want a cumbersome document?

Ideally: ask them why, discuss what they actually need from it, and find something I can do that will fit their real needs. (Maybe that would be writing the cumbersome document, alas.) If I'm not allowed to talk to the users, then the chances are that I'm also not allowed to make the decision anyway :-).

How do exceptions make your code bad?

Confusing the flow of control. You look at a function call and can't tell without digging into the function whether it's going to send the processor off to Honolulu next. This is a little better in Java, where at least you get a list of all exceptions that can arise in a particular method, but that has its own well-known drawbacks.

Introducing resource-leakage bugs. (I mentioned this before.) This is partly a special case of the "confused flow of control" problem, but there's more to it than that; sometimes it may simply be impossible to make sure that everything gets deallocated that ought to. It's especially evil in C++. (I think quite a lot of Herb Sutter's book "Exceptional C++", which I haven't read but would if I had to write a big hairy exceptionful thing in C++, is dedicated to this problem.)

[FWIW, this answer would lead me to recommend no-hire. The primary benefit of the way C++ handles exceptions is that they can automatically clean up resources, as long as you follow the ResourceAcquisitionIsInitialization idiom, and if you don't do that, or even mention it, then I wouldn't want to hire you as a C++ programmer. (I don't want to start a C++ argument here; I'm just trying to provide honest and helpful feedback.) --KrisJohnson]

Should every function in a C module have a block-level comment?

No. What would a block-level comment add to 

 extern int clockTicksSinceReset(void) {
   return clockCurrentTick() - clockLastResetTime();
 }
I would ask: 1. what unit are ticks? 2. what do you do when the clock gets reset and your ticks are no longer valid?

(Good questions, which indicate that I chose a bad example. (1) I intended that clockLastResetTime returns a number of ticks; but I couldn't, off the top of my head, think of a good name for it that indicated this and wasn't intolerably verbose. (2) I intended that the clock doesn't get reset. As I say, it was a bad example. I felt bad about it as I was writing it but was too lazy to improve it. Sorry.)

or 

 void drawCircle(double x, double y, double radius) {
   drawEllipse(x, y, radius, radius);
 }
that isn't there already? This sort of little function shouldn't be discouraged.

What languages do you actually use in practice?

Python and C++, mostly. Perl for little sysadminny things. Common Lisp at home for fun. (I would like to use Common Lisp at work too, but it's a bit too weird for my colleagues.) I recently had to do some Visual Basic at work ("had to" in the sense of "it was the only way to do what I wanted", not "my boss forced me"), but I very much hope that will never be a large part of my life. Assembly language for some bits of embedded firmware.

Careful with Terminology

On the whole I agree that questions like these sort out the right people however you need to be careful with terminology. For instance, although I've been a SE for 12 years and am very devoted to the cause, I've never heard the term "latent types". However if you had said weak typing vs strong typing then I would have understood the question immediately. I guess it depends which languages you are familiar with. I've also noticed that even concepts such as "coupling" and "cohesion", some of the most basic and important principles aren't understood by candidates I have had the (dis)pleasure to interview.

I agree; it's not wise to disqualify candidates based on their thesaurus knowledge. If you're going to ask a question like that, take into account their claimed background. If they've worked with more than a couple languages, you might expect them to be able to supply a good answer, but maybe only if you give them hints. Once they understand how to relate to your preferred terms, then you can find out how well they distinguish between them.

Weak/strong, latent/manifest, static/dynamic discussion

"Weak typing" and "strong typing" don't mean the same as "latent types" and "manifest types". My preferred terms are "dynamic typing" and "static typing", for reasons I'll mention in a moment.

C is weakly typed but statically typed. Common Lisp is strongly typed but dynamically typed. Weak/strong is about whether you can circumvent the type system; dynamic/static is about whether the type system (notionally) works at run time or compile time.

I don't like "latent" and "manifest" because something grates about describing a system where most types are arrived at by type inference (by the compiler) as having "manifest" types. -- AnonymousDonor

But TypeInference systems don't have manifest types. The distinction between dynamic vs. static and latent vs. manifest is that static types must be inferrable at compile-time by the compiler, while ManifestTyping must be specified at edit-time by the programmer. Haskell and ML are strongly, statically, latently typed, while C++ and Pascal are mostly-strongly, statically, manifestly typed.

I think the original question would be better phrased as "static vs. dynamic", because most LatentTyping systems let you attach optional type declarations to variables, which is the only non-hardcore-CS reason that manifest types might be superior to latent types. -- JonathanTang

MetaComment?

As the author of this page, I'm fascinated by the turns taken in the discussion. But, really, I think just about everyone on this Wiki would be in the [hire] category. The fraction of developers who send me a CV who are in the league of WikiAuthors? is, I estimate, 5%. The intent of the questions at the top was to easily get rid of the other 95%.

Does anyone agree that these questions (or other, feel free to add them) would help in making this first cut?

I thought they were good questions. -- SteveHowell

See also: HiringPatterns GuerrillaGuideToInterviewing, http://www.possibility.com/epowiki/?page=InterviewQuestions

CategoryEmployment

EditText of this page (last edited March 3, 2013) or FindPage with title or text search