Jackalopes and certification questions

Most folks do not know about Jackalopes. They are rabbits, usually Cotton Tails, with antlers from some sort of deer. The one pictured is from the Austin Museum of the Weird. Jackalopes, like unicorns, are fabled creatures may be based on real creatures with injuries, growths, or encounters with taxidermists with too much spare time. What does that have to do with MySQL Certification you ask?

I updating the MySQL Cluster DBA certification exam from the 5.1 to 7.0/7.1 product level. The current exam has not kept pace with the product and the question bank needs updates. The question bank has roughly 180 questions of which we use 130. The extra 50 are the questions that were either not fully developed or did not make it to the exam for a variety of reasons.

Of the 130 that are used, about ten percent are passed by all who take the exam. Roughly half of the questions are passed by at least seventy percent of the certification candidates. And 20% of the questions are getting very low correct rates.


So now I dig into those 20% to see if the questions are wrong, badly phrased, or if I have the question equivalent of a Jackalope. On occasion, despite the best efforts of all involved, a question that will slip through that is just the right mix of obscurity, mangled intention, and linguistic lethargy that combines to make a question that does not test or measure knowledge.

Sometime the questions can be salvaged. Other times they are replaced with new questions, hopefully concerning some stunning new feature in the cluster software. And hopefully we can vanish the Jackalopes from the MySQL exams.