Responding To Readers — A Better Way To Interview Software Engineers?
A summary and discussion of Khun Yee Fung’s thoughtful comment
I wrote this blog post a little while ago, and received a very thorough and thoughtful comment from a user named Khun Yee Fung. Here is the comment, if you would like to read it in its entirety.
The comment is long enough to be its own blog post, and I do not want to copy it verbatim because that would make this blog post too long, but here is a short summary:
- What I wrote was about the FizzBuzz test, even though at this point it is rather dated
- Fung wrote that tests do not assess what we know, but what we do not know. The danger with tests is that any standardized test can be studied for (memorization vs. true comprehension), and that what we want to assess is what they know, not what they don’t know
- What he does instead is asks candidates to bring their own code (as long as it is not proprietary), and explain it. Fung goes on to say:
First, I ask only questions on what the candidate says they know very well. So, if a candidate says they know TCP/IP, I will first ask how much they think they actually know. They tell me how much they know. And then I verify if that is true or not. I don’t ask questions that you can study easily. I ask questions that you will know if you are around something very closely and long enough. Since to become an expert on TCP/IP, for instance, you will get to know the RFCs, I would ask for the RFC number of TELNET, for instance. I don’t expect the correct answer. I don’t care about the RFC number. But I want to know if you are familiar with RFCs. If you are an expert in TCP/IP, you will start talking about all sorts of RFCs. Your eyes will sparkle. You will tell me when you started knowing the existence of RFCs, how you read them, etc.
I will ask you how two connections from the same machine to the same server port get recognized on the server side as two distinct connections. If you know TCP/IP properly, you will know. This is a very easy question. But it is also a very god gateway into protocol design, etc. I will find out the edge of your knowledge on TCP/IP.
And finally, I will ask what interface you use for TCP/IP, if you use Linux or Unix. and If you said sockets, then I will ask you the difference between TCP/IP and sockets.
Easy questions. I want to know how you answer them. How you answer them gives me all the ideas about what you actually know
A Few Initial Thoughts
First of all, this is a really good blog and I think it deserves more attention. This is someone with 40 years of experience, and his blog manages to explain programming (like here) in a completely simple and non-condescending way.
Second, I think that everyone who stumbled on my blog recently has also seen the similar, much more popular post on the same topic…it is called Another Idiot Boldly Proclaims That They Can’t Solve A Basic Programming Problem, by Andrew Zuo. Like a minority of other readers, I find the tone of “Another Idiot” a little off-putting. The “idiot” in question is Max Howell, and I think commenters were right to push back on calling him an idiot, or stating that “package managers are not the most complicated things to write. They just store different versions of packages” (a direct quote from Andrew’s post).
That being said, it makes for an interesting discussion.
To make this more interesting, you can actually find a Quora discussion on the topic by Max Howell. To make this even more interesting, Gayle McDowell (author of Cracking The Coding Interview) joined it.
Max Howell wrote:
I wrote a simple package manager. Anyone could write one. And in fact mine is pretty bad. It doesn’t do dependency management properly. It doesn’t handle edge case behavior well. It isn’t well tested. It’s shit frankly.
Is it any surprise I couldn’t answer their heavily computer-science questions well?
On the other hand, my software was insanely successful. Why is that? Well the answer is not in the realm of computer science. I have always had a user-experience focus to my software. Homebrew cares about the user. When things go wrong with Homebrew it tries as hard as it can to tell you why, it searches GitHub for similar issues and points you to them. It cares about *you*. Most tools don’t give a shit about you. If they go wrong, well screw you. Homebrew helps you.
…
Google in fact gave me seven interviews and I did well in the software engineering ones, because that is actually my talent. I feel bad about my tweet, I don’t feel it was fair, and it fed the current era of outragism-driven-reading that is the modern Internet, and thus went viral, and for that I am truly sorry.
But ultimately, should Google have hired me? Yes, absolutely yes. I am often a dick, I am often difficult, I often don’t know computer science, but. BUT. I make really good things, maybe they aren’t perfect, but people really like them. Surely, surely Google could have used that.
— Max Howell
That Howell proclaimed that his own software was bad is surprising to me, and may even give credence to Andrew’s claim — but at the same time, it simultaneously argues for why Homebrew deserved its success. To paraphrase his words, Max Howell argues that he could have brought great value to Google with his user-focused skill set, even if he did not have god-tier programming ability to ultra-optimize his code.
Gayle McDowell, in the exact same thread, wrote
I suspect that you are awesome as a developer in a certain place, but that Google — at least as a software engineer — wouldn’t be the right fit for you. They really do want computer scientists. The ability to figure out what to build isn’t valued all that highly, because that’s not fundamentally the responsibility of the software engineer.
It sounds like you believe your skill is making great products — not so much as the technical level, but at the product design level. That would suggest a product management role (at google or somewhere else) could be a great fit, if you wanted to be at a big company.
The Take-Home Test
It is always useful to hear accounts from people on the opposite side of the coding interview table. Fung’s suggestion that we ask candidates for code samples reminds me of a very common interview practice: The “take-home test.”
Is it common? Yes. Is it good? Debatable. I will state a few issues I have with them, although these are strictly anecdotal:
- Sometimes take-home coding challenges are essentially just LeetCode quizzes with more time — I have had challenges that were straight out of Project Euler and straight out of LeetCode. At the risk of starting an argument, I think this makes them functionally equivalent to LeetCode interview questions. They are not very realistic (I once had this question). In contrast, another company had me make a frontend web application for a frontend web application role, so no issues there
- It goes without saying, but coding take-home challenges are rather time-consuming to candidates
Above is a podcast episode from “Tech Team Weekly,” which has 235 views and was really hard for me to find again. Like Fung, it comes from the point of view of interviewer, and I think the most entertaining response is at 18:33. One of them says that he views interviews as trust exercises — is their resume really 100% the truth? The best interviews are conversational in nature and the worst are like interrogations on how truthful they are, but at 24:33 he shares this very entertaining story:
- He put out a coding test on GitHub when they were hiring for a developer
- For some reason he checked Google to find his test again, and realized someone on a freelance site had posted his tech test. They wanted to hire someone to take the test for them
- His company had to bid for and win the right to solve their own coding challenge, because otherwise they would not be able to trust any candidate
Further Discussion
No one has really figured this out. Some companies hone in on domain knowledge, like Fung suggested, some companies use take-home challenges, and the majority of them (anecdotally) seem to use some variation of the LeetCode interview. For better or worse, this is usually just one step of many, from behavioral interview to system design interview to automated coding test.
I actually think Meta’s is one of the best — they required two coding questions in 45 minutes to an hour, but cared a lot more about approach and communication than if we actually finished.
In all likelihood, this debate on the ideal software engineer interview will continue for quite some time.
