Interviewing Testing Individuals

Finding good testers is like prospecting for gold. They take patience and skill to find. Many times during an interview I find that most testers have memorized the basic definitions of the field; usually from Google. So, in addition to standard interview questions I ask a scenario question in order to guage their knowledge and understanding. This is akin to a tester who tests a tester, if you will. To do this, I usually set up a simple scenario, such as what kinds of tests would you run if you were asked to test a toaster? Sometimes I would give them requirements, such as below, and sometimes I don't.

1. It is an electric two slice toaster
2. It has a single push down lever that control both sides
3. It automatically pops up, and shuts off when the desired darkness is achieved
4. It has 3 darkness settings: light, medium, dark that are triggered based on heat build up in the toaster

I like this scenario because it has proven, for me, to be a pretty good indicator of the type of testing a candidate has been exposed to as well as an indicator of their ability to decompose of requirements (direct and/or inferred).

Since I do not inform them of this question prior to an interview, I can surmise, based on their responses, what type of testing they favor. Below are some sample responses I tend to get and how I would categorize that response.

Category 1: Performance based

1. Testing two slices on each setting to see how long it takes for each setting
2. Testing multiple slices per setting to arrive at an average time for each setting
3. Repeating 1 and 2 for a single slice
4. Testing the coloration consistency over multiple toasts on the same setting
5. Testing that the toast time remains within tolerance over lots of toast cycles

Category 2: Aesthetics and User Interface

1. Testing to ensure the settings and labels marking the settings in alignment
2. Testing to ensure the lever works when pushed down
3. Testing to ensure the lever does not catch either going up or down
4. Testing that the finish is not tarnished or changed as a result of the heat produced when using the toaster
5. Testing that the slots are big enough for standard size toast

Category 3: Functional - General

1. Testing to ensure toast darkness matches darkness selected
2. Testing a variety of bread: wheat, white, bagels, pop tarts
3. Testing that the lever and toast pops up at the end of the test cycle
4. Testing that the toast can not be re-toasted until the toaster cools down
5. Testing that the toaster still shuts off if the lever is stuck in the down position

Category 4: Functional - Boundary

1. Testing to ensure the toaster is not damaged with no slices (or does not allow the lever to be pushed down)
2. Testing with a single slice (tested twice, once for each slice)
3. Testing to see that any single slice slot is consistent with the other as well as with two slices
4. Testing to see that it works with both thin and thick bread types
5. Testing to see what that it works with oversized bread for the slots

Category 5: Exploratory

1. Testing to see what happens if you push the lever while it is unplugged
2. Testing to see what happens to a variety of bread: wheat, white, bagels, pop tarts
3. Testing to see what happens if you test nothing.
4. Testing to see what happens if you manually hold the lever down.
5. Testing a two slices in one slot with cheese in between sandwich

This is not an end all list of possible tests, but rather a compilation of answers I have received over the years. I have found that people usually cling to a couple of categories, which is usually indicative of their experience. I usually ask the candidate which tests are more important, or which tests can be combined into a single execution. To make things interesting, and to see if people really understand regression testing, I often expand the toaster to a four slice toaster that has two setting knobs. This is where you start to see peoples understanding of testing. Often they will mention “I’d repeat the same tests.” What I look for is for testers who under new combinations, expansion of requirements, important versus non-important re-tests such as:

1. Tests for independent settings between the slot sets
2. Tests for a single slice in one of the two slots in each slot set

Again, this is not a definitive evaluative technique for testers, but I have found that it is quite beneficial and accurate in categorizing a tester’s knowledge and experience.

One thing I haven't tried yet, is bring a physical object to an interview and asking a candidate to test it. The Easy Button from Staples might be a great option. Then I can observe their behavior instead of analyzing their thoughts.