I had a great conversation with a colleague of mine recently about the challenges of interviewing technical candidates and truly understanding their motivation and capabilities. What had started this discussion was a great article at the NY Times about how Google is dropping the use of their infamous “Brainteasers” as part of the interview process (Microsoft famously did this as well for many years). Here is a link to the Article:
The reality is that the job of being a Technologist is partially about being a specialist and also part of being a generalist. The tough part is understanding the balance and knowing what is the right mix is. Sometimes what a project/team really needs is a Specialist.. They need someone who understands a particularly obscure or arcane technology inside and out and just throw the person at the problem and let them grind away at it. At other times what a project/team needs is someone who has good skills across a number of technologies and can easily move from one particular role to another and, more importantly, can decide what is the best tool / technology to use to solve a particular challenge. It turns out that finding these latter people is often very hard.
I will talk about the challenges of finding Specialists in another post but I wanted to focus on the challenges of identifying a Generalist here.
You see there is a fine line between being a Generalist and being perceived as not having sufficient technical depth. It is the old allegory of “Jack of all trades and Master of none”. The true hallmark of the the Generalist is that they have abstracted their skills to a level beyond any one tool or technology and can quickly learn a new way to express that understanding. And that turns out is very hard to test for.
Google and Microsoft were trying to use Brainteasers to evaluate a candidates analytic skills by throwing an abstract puzzle at them and seeing how they tried to solve it. But you are asking the candidates to demonstrate abstraction skills on an abstraction. It reminds me of watching folks at the gym that are doing yoga poses on a Bosu-ball.. Here is a visual:
The challenge is that it does not seem to correlate with real performance. Abstraction all by itself is hard. Abstracting the Abstract is that challenge squared! It is a skill in and of itself!
So what are we to do? How can we evaluate the capabilities of candidates to see if they have the ability to really move their skills across several different tools and technologies?
The simplest answer is frankly to get them in front of someone with those very same skills. It turns out that if you are really a Generalist then you typically are very good at recognizing the common patterns that almost every information system ever written has to one degree or another. The great dirty secret of computing is that almost every “new” trend has a prior analog. “Cloud Computing” is hot and new right? Except many of the concepts were well explored when we were all using Mainframes. “Mobile Computing” is new and hot right? We learned a lot about the challenges of Distributed Computing during the “Client/Server” revolution. Understanding the patterns is the key to identifying the parts of the problem that already have solutions and the parts that represent truly new challenges. Having a dialog about those prior lessons and new challenges is a sure fire way to identify if the candidate really has the abilities you are looking for.
So forget brain-teasers. Get qualified and skilled individuals to have a discussion about the projects and technologies the candidate has worked with. Let them explain how they decomposed the problem and leveraged previous knowledge and also get them to articulate what new technologies they needed to learn and adapt to solve it. If they can tell you all that and seem to have a gleam in their eye at the end of the discussion you probably have a good hire on your hands.