Certification - A money maker, but not for you

I received this advertisement via email. It is funny, in their misconceptions, but it is more sad that so many people buy into it (and by buy, I mean spend $$$).
Okay, so let's

start with the second line of this advertisement,

"If your team is conducting ad hoc, informal tests with little guidance or planning, the quality of the end product can be severely jeopardized—negatively affecting your bottom line"

This nearly represents everything that is wrong with the ISTQB certification. The quality of the end product is not

jeopardized with informal testing, a lack of test planning, or a lack of guidance. In reality, quality is a relationship, the simplest of which being the value of the product to the stakeholders (those who matter).
But this next statement in their advertisement does represents everything that is wrong with the ISTQB certification

"The best way to be certain that you are providing customers with quality software is to make sure your team of testers is certified."
Really? I thought it was by providing something they value, usually something built to meet their wants and/or needs? If I all need to do is put a gold embossed sticker on it, then so be it. Here you go.

All your software is now of high quality. Oh by the way ISTQB, that will be $1,995 + $250 ($1,995 for training so that you know how to use the sticker, and $250 for the right to use the sticker. Let's throw in $9.95 for Shipping and Handling as well). So, they don't have a clue about the testing industry, nor about quality. But hey, let's see what their mission statement says. Maybe that will shed some light on what they are trying to do.

It is the ISTQB's role to support a single, universally accepted, international qualification scheme, aimed at software and system testing professionals, by providing the core syllabi and by setting guidelines for accreditation and examination for national boards

So, their mission is to create a certification scheme (their word, not mine) and provide the materials and exams for that certification. It is nothing less than a money making scheme, and it is right there in their mission statement. They do not care about quality or testing.

I do not accept the ISTQB as a part of any community of software testers. The ISTQB is a business, pursuing their own agenda. I know that may sound a bit harsh, but consider this. As Michael Bolton pointed out, In Oct 2008, ISTQB announced 100,000 certified testers. Each of these testers had to pay a fee to take the exam. For the U.S, this fee is $250 (entry level) and I think $100 in India. That means they have made between $10 million to $25 million in revenue on certifications alone in the past 5 years. So far, they are succeeding at their mission statement.

Interviewing Testing Individuals

Finding good testers is like prospecting for gold. They take patience and skill to find. Many times during an interview I find that most testers have memorized the basic definitions of the field; usually from Google. So, in addition to standard interview questions I ask a scenario question in order to guage their knowledge and understanding. This is akin to a tester who tests a tester, if you will. To do this, I usually set up a simple scenario, such as what kinds of tests would you run if you were asked to test a toaster? Sometimes I would give them requirements, such as below, and sometimes I don't.

1. It is an electric two slice toaster
2. It has a single push down lever that control both sides
3. It automatically pops up, and shuts off when the desired darkness is achieved
4. It has 3 darkness settings: light, medium, dark that are triggered based on heat build up in the toaster

I like this scenario because it has proven, for me, to be a pretty good indicator of the type of testing a candidate has been exposed to as well as an indicator of their ability to decompose of requirements (direct and/or inferred).

Since I do not inform them of this question prior to an interview, I can surmise, based on their responses, what type of testing they favor. Below are some sample responses I tend to get and how I would categorize that response.

Category 1: Performance based

1. Testing two slices on each setting to see how long it takes for each setting
2. Testing multiple slices per setting to arrive at an average time for each setting
3. Repeating 1 and 2 for a single slice
4. Testing the coloration consistency over multiple toasts on the same setting
5. Testing that the toast time remains within tolerance over lots of toast cycles

Category 2: Aesthetics and User Interface

1. Testing to ensure the settings and labels marking the settings in alignment
2. Testing to ensure the lever works when pushed down
3. Testing to ensure the lever does not catch either going up or down
4. Testing that the finish is not tarnished or changed as a result of the heat produced when using the toaster
5. Testing that the slots are big enough for standard size toast

Category 3: Functional - General

1. Testing to ensure toast darkness matches darkness selected
2. Testing a variety of bread: wheat, white, bagels, pop tarts
3. Testing that the lever and toast pops up at the end of the test cycle
4. Testing that the toast can not be re-toasted until the toaster cools down
5. Testing that the toaster still shuts off if the lever is stuck in the down position

Category 4: Functional - Boundary

1. Testing to ensure the toaster is not damaged with no slices (or does not allow the lever to be pushed down)
2. Testing with a single slice (tested twice, once for each slice)
3. Testing to see that any single slice slot is consistent with the other as well as with two slices
4. Testing to see that it works with both thin and thick bread types
5. Testing to see what that it works with oversized bread for the slots

Category 5: Exploratory

1. Testing to see what happens if you push the lever while it is unplugged
2. Testing to see what happens to a variety of bread: wheat, white, bagels, pop tarts
3. Testing to see what happens if you test nothing.
4. Testing to see what happens if you manually hold the lever down.
5. Testing a two slices in one slot with cheese in between sandwich

This is not an end all list of possible tests, but rather a compilation of answers I have received over the years. I have found that people usually cling to a couple of categories, which is usually indicative of their experience. I usually ask the candidate which tests are more important, or which tests can be combined into a single execution. To make things interesting, and to see if people really understand regression testing, I often expand the toaster to a four slice toaster that has two setting knobs. This is where you start to see peoples understanding of testing. Often they will mention “I’d repeat the same tests.” What I look for is for testers who under new combinations, expansion of requirements, important versus non-important re-tests such as:

1. Tests for independent settings between the slot sets
2. Tests for a single slice in one of the two slots in each slot set

Again, this is not a definitive evaluative technique for testers, but I have found that it is quite beneficial and accurate in categorizing a tester’s knowledge and experience.

One thing I haven't tried yet, is bring a physical object to an interview and asking a candidate to test it. The Easy Button from Staples might be a great option. Then I can observe their behavior instead of analyzing their thoughts.