[eDebate] Fwd: Re: Response to Kloster

[Jean-Paul Lacy]

>As a final caveat, none of us are as smart as we think we are in our
>rankings.  It is still the case that judge rankings are a very poor
>predictor of who wins a debate, being slightly worse than chance.
>
>GARY


If rankings are slightly worse than chance:

Are we collectively bad at picking judges? Or, is does this statistic prove 
that we can collectively pick good judges?

If we're only slightly off pure chance, maybe mutual preference is becoming 
strong enough that we can pick fair judges.

Maybe debaters and coaches are getting smart enough to pick the judges who 
will do their best to determine the fair winner. A mutual 100 judge can 
only pick one winner.

The bottom line is the holy grail--every team in the tournament gets the 
judge who they think can fairly judge their debate.

The real question is: How much lack of mutuality is a predictor of who 
wins? Or, when does the difference predict an outcome?

The point where it becomes a significant difference should be the cut-off 
for mutuality in the whole preference vs mutuality mess.

--JP "still learning statistics" Lacy

ps--While I agree in principle with having a "bright line" or "cap" for 
strikes, shouldn't people be able to figure this out for themselves if they 
filled out a sheet that made their Z-score of a 0 LESS than -1? The numbers 
are on the sheet as you submit them.

pps--Given an unfettered 0-100 system, I disagree with translating things 
into ordinals for an additional comparison point for the tab room. Ordinals 
are useful, but they don't reflect how teams fill out a sheet in an 
unfettered 0-100 system. People are counting on the Z score to reflect 
differences between clusters when they fill out an unfettered 0-100 sheet. 
Ordinals can't reflect that.

ppps--The 9-0 system isn't good enough. Has any system beat ordinals in 
terms of overall preference? Despite whines to the contrary,ordinal ranking 
is the easiest way to fill out a preference sheet. Get a stack of 4x6 cards 
and put them in order if you can't figure out how to do it on a computer. 
Honestly, it is much easier to figure out if X judge is better than Y than 
if X judge should be deemed equal to your A+ judges. If sheet gaming, (as 
reflected in categorical 9-0 prefs,) is valued by the community, it is 
still preserved in an ordinal system. Add a guaranteed strike cap or 
"cut-off" to that system and you have the best we can do for the time being.