Withington Pub Quiz League - perfect quiz from Quizbiz 04/02/04

Following the quiz papers we've tackled over the last few weeks of 2003 and the first few of 2004, the debate as to what makes a perfect quiz question has been re-ignited.

Below I publish some comments from Gerry of the Brains, my own thoughts in response to Gerry and some interesting comments from Kieran which rather support Gerry's views. I have also republished the 10 points for a perfect paper that I put on the website last season. Comments (which, of course, I'll publish) very welcome. I'll leave links to this page from the 'Home' page for the next few weeks.

(19/01/04)

Mike ,

I read with interest your feedback on our quizset of last week. Let us for a moment resurrect your old "perfect question" debate. But no shouting. I've just got Fr. Megson down for the night. He looks ever so peaceful clutching his comfort surplice and with his little toe stuck in his mouth. Don't priests look ever so sweet at that age?

Where better to begin than at the end? You conclude by asking, nay beseeching, next week's setters to ask easier questions. Leaving aside the subjectivity of the word for the moment, why do you want "easier" questions? To win and be safely tucked up in bed for 9pm? Surely not. Nobody needs a win more desperately at the moment than the Persistent Vegetative State Brains Of Oak and the, as yet unsparked, Piggies but I'm sure that Gary and Co. would join us in saying that we would rather lose an interesting "hard" quiz than win a soporific "easy" one. We want to win, of course we do, but more than winning I want the questions to entertain me. I want them to tease me and make me bite my nails wondering if I should go for it or chicken out and confer. I want them to cover a wide range of subjects not all of which will be close to my heart but that's democracy for you. Sententiously perhaps, I want to stagger home with a few more irrelevant but vaguely interesting facts tucked under my nerdish anorak so that I can wake my wife and kids up and slurringly inform them that Co. Tipperary used to be divided into ridings and isn't that absolutely amazing and would you like a cup of tea now that you're awake anyway.

So how does a setter achieve this Utopia? I don't know. I suppose we are all biased by our own knowledge. If I know it then so should everyone else. Obviously not true but I suspect it's a trap we all fall into. To judge by your feedback we obviously failed last week and we obviously failed to fulfil Albert Park's requirements by quite a margin for which I am genuinely sorry. And yet we laboured over it and genuinely felt that it was diverse and achievable. It was deliberately "quirky" I grant you, but in most cases the quirkiness was in the form of asides that did not actually influence the toughness of the question and were merely meant to lighten the occasion. Your Plato/Aristotle criticism is, on reflection, valid. I tried to revive an old chestnut by dressing it up and it didn't work. It does sound whimsical as you said (or even "crap" as you probably meant) though in fairness the Geneva Convention clearly states that the setter shall not be held responsible for the acoustics of the quiz venue.

Here is how we arrived at some of the questions. Follow us for a moment into the canyons of our minds. Pull up an armchair - there's plenty of space - and tell us what you think:

1: Make a question hard but compensate by putting extra information into the question to make it more guessable. For example, what is the capital of the Republic of Ireland would be a boring question (too easy even for Kieran's lot!). What is Dublin's most famous shopping street is too hard for most. OK, so ask them which British PM this street is named after. So now you may still not know the answer but at least you are able to use other bits of your knowledge to help you make an educated guess. The PM must have been before Irish independence - Dublin is largely a Georgian city so the PM probably comes from that era. It is still not easy but at least your team now have a fighting chance of scoring. Maybe not a "perfect question" but it sure beats the Weakest Link's "what B. is the capital of Northern Ireland?"

2: I love Round Britain Quiz on Radio 4 and I try to make questions up and use their cryptic formula and phraseology. For example, Q15 from the paper in question (14/01/04) with the answer of Palatine. I sense that our cryptic questions get up people's noses but I think this is unfair. Being cryptic is not the same as being quirky. The setter has every right to "tease" you and to make you approach the question from a different angle. Stumped are (rightly) lauded for their cryptic crossword questions. Why is it any less correct to encrypt the question without actually using a crossword formula.

3: Take an old chestnut and extrapolate on it. I got so fed up being asked to name the city served by Dyce airport that I actually got to thinking "who is this Dyce bugger?". So I looked him up and Q37 (14/01/04) was thus spawned. I agree this is a very hard question but the phrase "one of Aberdeen's most famous sons" might just serve to ring an old chestnut bell from the dim and distant past. And if it doesn't at least you have learned something. Though you'll probably stagger home wishing that Aberdeen had called their airport after Denis bloody Law.

Have to go now 'cos Fr. M. is stirring. Oh dear! I should never have let him eat that boiled turnip before he went to bed.

Can we have a debate on quiz-setting please all you priests out there in cyberspace. Some time ago Gary urged you all to use the website more. Now is your chance to give Mike some relief (steady, Ethel!) and better still, to knock Fr. M. off his pulpit. It's a shame we don't have a judge in the league and then we could have a definitive ruling on legal and illegal quiz-setting and how to punish offenders.

I won't be in Manchester for the quiz on Wed. 21st. January but please don't make it too easy, Dummy. (Don't be intimidated by Bath, he's not as brutal as he looks).

Incidentally the answer to the Weakest Link question above is, of course, Belmopan.

Regards,
Gerry.

(19/01/04)

Gerry,

Excellent note. Worthy of Aristotle or even Plato.

I, too, love Round Britain Quiz and have even been known to listen to past editions on my computer via the internet.

The whole idea of approaching a question from two different directions, giving the questionee the chance to work out the answer to something fairly obscure, fills me with delight. Indeed it's how I learn. The converse, that is to be challenged to remember things that I used to know but, due to old age, or just disinterest, I have forgotten doesn't excite (other than the possible instant triumph of gaining 2 points). It's like the difference between a good meal at The Lime Tree and a quick burger at McDonalds. One lasts and can be relished for some time. The other pleasure fades within minutes and just leaves an emptiness.

However..... many of our good friends and Wednesday drinking colleagues have a different view of life.

They want a test of memory, a bit of a chinwag and then off (after a ritualistic moan about the setters, of which I am as guilty as anybody). On this last point I feel rather strongly (having too often been in the position you are at present after last week's minor mauling). I very much want to prevent our league opting for standard quiz papers from a 'professional source' since the great pleasure each Wednesday is the unpredictability along with the hidden gem (such as the 'Tony Blair, Lord Salisbury, William Pitt the younger' question in your own paper). The weekly joy depends on a few of us (let's be honest it's only about 15/20 of us that share the burden of paper setting) rolling up our sleeves and devoting the occasional weekend to sweating over a computer and a load of reference books. For such selfless effort to be rewarded (sometimes) by brickbats can be a bit galling.

Feedback though is the lifeblood of improvement so I will continue to offer constructive criticism as well as reflect the views I hear, on the website.

As a general rule I very much enjoy the papers you offer because they give tangential views on knowledge allied to a whimsical approach to life. Following this line however is more risky than just going for the quiz book approach. If you overdo the whimsy or make the question really hard because you are giving 2 ways in, and then make both ways in pretty hard as well (I'd never heard of Grafton being a PM), then you can lose your audience. It's significant that the Park and the Opsimaths are 2 of the most ardent fans of the Brains papers usually (and we both like the Bingo format to boot) yet we both struggled last week.

MIKE

(21/01/04)

Mike,

I've been reading the debate about setting question papers, so I thought I'd stick my oar in (how unusual is that?).

I don't think a question paper should be judged by the aggregate score. Our paper from earlier this season had a very high average score but I know it wasn't well received in some quarters. What really pleased me about it was that four of the five games were won by a margin of five points or less. This suggests that we balanced it well and as a result there should have been some close, even exciting games.

This can also happen where the overall scores are much lower, provided the setters give some thought to how the questions are distributed throughout the paper. The way it works with our team is that we all set questions individually and then I amalgamate them into the final paper. Thus Martin will deliver questions with a maths / science, literature and sport predominance. Antony will typically set Greek / Roman history, more literature, music and some quirky subjects like canals. Barry weighs in with a picture round, natural history, geography and perhaps films. They haven't been asked to set on these subjects, it's just

their own particular interests. When I've got all the questions I see what areas aren't covered, say current affairs, cookery, pop music and perhaps more sport. Once there are 64 questions I try to divide them up so that the same person doesn't get all the film questions and if there are some unbalanced pairs then I try to ensure that each team gets the harder questions as fairly as I can.

I'm a fan of Brains' papers. The best match we ever played was last season's cup final against St. Cath's. Big Mike's lot won 36 - 34, great game, went to the last pair and Gerry's penchant for Irish geography had us all thinking about every question. In fact both teams were pulling answers out of half remembered pieces of information and inspired collective working out of answers, Steve Winwood - Higher Love was my personal high point.

Gerry's argument is quite right. I couldn't care less who manufactures the Yaris and I'm sick of bloody Dyce airport, but when we were told that answer last week we all went "oh the airport so that's where it comes from." Good question and I'll remember it.

Questions can be set on any subject, well I did once play in a league where Ceefax page number questions were banned, quite rightly I think. The way it worked in that league was that the team setting the paper went individually to each venue to ask the questions. Thus one Tuesday we all set off with portable cassette recorders to ask, among other things, the teams to identify the sounds we had recorded. I still maintain that "Curly pond weed photosynthesising" from the BBC sound effects lab was a gettable answer, but I don't think I've ever convinced anyone. We've also included herbs and spices to be tasted as questions and other teams have done similar things.

It's the way the question is phrased that can give it the extra interest or extra possibility of working out the answer, witness the Grafton question. We didn't get it, nor did St. Cath's but we could have done if we'd been thinking properly and thus it ranks as a good question.

My beat the intro round last year, eight first lines of songs, answer required just the title, didn't seem

to go down well. I suspect if I'd spead the questions throughout the paper there would have been less

comment. It was worth trying and I won't do it that way again.

We're setting on the 28th and we've already got a few ideas, so I hope I've not put myself up to be

shot at too much with this.

I've given up asking where's Dick Cheney 'cos nobody seemed to care, shame I'll miss him. Working on a new running theme for the next couple of years though.

Kieran

(30/01/04)

(Kieran - referring to the Fifth Finger paper played on 28th January)

Me again fuelling the debate on question setting.

Accepted our paper was not the easiest ever. This was a deliberate decision. We were criticised in some quarters for making it too easy last time and, particularly after Dummy's points fest of the previous week, we thought we'd make it a tougher test this time around. Even given that the average score falls within your range for indicators of the perfect paper.

As regards the round of Se7en. I debated long over whether or not I should indicate that the round was themed. If I had done this, it would quickly have become a simple test of memory, writing down the deadly sins and then crossing them off as the answers were given. I don't think that's what we should be aiming for when setting questions.

You note that Tony from Snoopy's said there wasn't a pair for 'sloth'. Quite right and he should have been asking himself "why not?". Gerry Collins looked stunned when he didn't get an animal pair to sloth but got the N.V. question instead. Again, given that the whole quiz was paired up till that point (bar the seaside pictures) he should have been thinking "something's going on here".

The next two questions were equally easy, I think, and after them, the answers 'Sloth', 'N.V.', 'Grapes of Wrath'and 'Lust for Life' had all been given.

Question 5, "Pride goeth before a fall" was clearly ungettable unless there was some other clue somewhere. This is exactly what The Historymen said. OK so why would a team which has a reputation for straightforward papers, erring on the easy side, suddenly drop in an impossible question? Well, I should say I was trying to make the teams think "we're missing something here", while still giving them a chance of getting to the right answer - but not making it too easy. Otherwise I might just as well have asked for a list of the seven deadly sins, or Santa's eight reindeer (I considered that, but couldn't come up with a question to which the answer was 'Blitzen').

Overall half the questions, Numbers 3, 4, 7 and 8, were pretty easy, and, evenly paired between the teams, so some points should still have been scored and the round shouldn't have affected the overall result too much. Unless, of course, one team was alert to what was going on and the other wasn't. If that was the case then good luck to the alert team. That's part of the skill of answering questions as well. It's not just about a memory test to recite obscure facts learned some years ago.

I accept it didn't work perfectly this time, but I still think it was worth trying. I intend to do something similar next time we set. Not necessarily a theme again, but there will be something else going on apart from the bald question, so you have been warned.

There will be detention for anyone not alert next time.

Cheers

Kieran

(31/01/04)

(Gerry - referring to the Fifth Finger paper played on 28th January)

Mike,

Read the Website last night and was a bit disappointed by the apparently consensual headmasterly "nice try boys but don't dare try pulling a stunt like that again" reaction to the rapidly becoming infamous Deadly Sins round.

Kieran eloquently proved in his Apologia that he doesn't need a defence brief but I think Finger V deserve a bit of moral support.

Talking sense is well outside the remit of Fr. Megson but he was briefly guilty of doing so this week when he succinctly summed up round 7 with the words "fair cop". And it was a "cop" fair and square. Most of us swam blindly, open-mouthedly and without the use of a lateral fin into a cunningly and, I think, brilliantly placed net. We slept on our roses, didn't we, Jitka? Pooped on them and then slept on them. And, aren't mixed metaphors brilliant? If you are caught with your trousers around your blushing ankles just remember that you will look even sillier if you then stick your head in the sand. So pick up your kecks and run with them. Join the Lateral revolution (I think the Vatican already is a member). You have nothing to lose but your hoary old chestnuts.

We shall now sing our new anthem "Sidewards Christian Soldiers" and then you can go home and start plotting revenge on Kieran's lot next time you set. But remember to play hard but logical like they did.
Incidentally, am I the only one who feels he might have done better on this round (and all the others ) 10 years ago. Increasingly I feel knackered after work and, even without alcohol, my brain waves "nighty night" at 8.30pm sharp. We can't turn the clock back alas so how about having the quiz BEFORE work. I can be quite bright some mornings.

See you outside the Red Lion at 7am next Wednesday, Ethel.

Gerry.

The Perfect Quiz Paper (as per Mike Bath 04/04/03)

After sticking my neck out for most of this quiz season on the website and offering views about the quality of the quiz papers set, I have been challenged to come up with what I think is the formula for a perfect quiz paper.

Well here goes:

1.    An aggregate score of between 65 and 80 points. If possible this would be made up of at least 16 x 2 points, 40 x 1 points (conferred or passed over) and a few no scores. Many more than 80 means the quiz was too easy, and less than 65 means it was too hard.

2.    A fair crack of the whip for both teams. Unless you've gone for the Bingo format, this probably means paired questions (one of each pair going to each team). Preferably the second half of the pair should be some distance from the first half so that the likely answer for the second question of the pair has not been discussed just a few seconds before it was asked as part of the process of answering the first question in the pair. It also means avoiding questions where the answer is one of a few possibilities explicitly mentioned in the question - so that a wrong answer by team one leaves team two with a much better chance of guessing right. An example of this would be the recent question asking 3 of Dickens’ books to be listed in the order in which they were written. A wrong answer by team one considerably simplified team two’s task.

3.    Avoiding overpopulation by one sort of question. For instance 6 questions in one paper about football seem too many. 4 might just be OK. Themes are OK as long as they are used as a lead into what is regarded as a pot pourri of general knowledge questions. A whole round where the answer is the name of a film or film actor is not OK, but a round on November 5th that leads to questions on history, chemistry, religion and questions whose answers contain the name of a firework is fine. However 64 questions derived from a November 5th theme is almost certainly 48 too many.

4.    Not veering too far from general knowledge. 'General Knowledge' is notoriously hard to define but most people have a good feel for what it is and what it isn't. Specialist knowledge acquired at work (whether in the labs at Christie's, on the bench in Crown Square or in some software company) probably isn't General Knowledge.

5.    No questions that depend on a subjective judgement for getting the correct answer. For instance ‘Who is the most famous….?’ or ‘Who is the best……?’

6.    Up to 4 questions in a quiz with the ‘well I never’ factor. Too many of these indicate a question setter who is trying to show off arcane knowledge. But a few sprinkled around can really enhance an evening. These are usually the questions that get mentioned the next day at work, I find.

7.    Up to 8 questions which tempt the whole team to confer for a while debating the subject matter and eliminating less likely options to arrive at a preferred answer. A good example this season was ‘How many of the original founding football league clubs are in the Premiership this season?’. Too many of these sorts of questions, however, can lead to 'quiz constipation' and very late finishes which can be pretty antisocial.

8.    Make the wording clear and capable of being answered correctly by just one answer. Incidentally an answer that has to be qualified by a statement such as: ‘only accept ….’ usually means the question wasn’t clearly enough worded.

9.    Keep the wording of the question to as few words as possible but, more importantly, make sure the answer is short. The best questions usually have one word answers.

10. A bit of innovation. However sound a format we have developed for our papers over the years we are all going to get bored by it sooner or later. Each season it is essential to have a few new ideas for the format. The SWMCC crossword round this season was a welcome addition. Last season the Brains floated the 'most northernmost, southernmost, etc' questions.