The Wisdom of Crowds
Posted: Mon Feb 11, 2008 11:43 am
Has anybody read this?
A colleague of mine let me borrow it based on the job that I do (he had no idea about my interest in BAM). Anyway, the very first chapter starts with the author's opinions on Who Wants To Be A Millionaire. I'm paraphrasing because I don't have the book with me right now, but the gist of what he was saying is that a group can get to the "right" answer of something better than any individual member of that group or individual experts in that particular field. I'm three pages in, so he hasn't convinced me either way on that hypothesis, but his BAM comparison was suspect, to be kind.
He (or somebody) calculated the success rate of PAFs and ATAs. Apparently PAFs are successful around 60% of the time while ATAs are successful 91% of the time. While he admits that they aren't the same questions being answered, the one set of questions isn't appreciably more or less difficult than the other, so clearly group think is more desirable than individual input, even if that individual is hand picked as an expert in a particular field.
Is he crazy? Clearly, he hasn't watched the show near enough to know that ATA is used before PAF in the vast majority of cases. ATA works very well at 8K and below, but once you hit 25K, it's almost always a crapshoot. PAF is rarely used below that 8K marker. This guy really needs to do his homework. A better comparison (albeit one that wouldn't have very many instances) would be to compare how many times a contestant has used both ATA and PAF on the same question and see what the results are. Of course, if that happened, the ATA results would have to have been ambiguous enough to use a second lifeline, but still the conclusions drawn from that comparison would be much more valid than taking the entire process as a whole.
OK, that's our on-topic post of the day....
A colleague of mine let me borrow it based on the job that I do (he had no idea about my interest in BAM). Anyway, the very first chapter starts with the author's opinions on Who Wants To Be A Millionaire. I'm paraphrasing because I don't have the book with me right now, but the gist of what he was saying is that a group can get to the "right" answer of something better than any individual member of that group or individual experts in that particular field. I'm three pages in, so he hasn't convinced me either way on that hypothesis, but his BAM comparison was suspect, to be kind.
He (or somebody) calculated the success rate of PAFs and ATAs. Apparently PAFs are successful around 60% of the time while ATAs are successful 91% of the time. While he admits that they aren't the same questions being answered, the one set of questions isn't appreciably more or less difficult than the other, so clearly group think is more desirable than individual input, even if that individual is hand picked as an expert in a particular field.
Is he crazy? Clearly, he hasn't watched the show near enough to know that ATA is used before PAF in the vast majority of cases. ATA works very well at 8K and below, but once you hit 25K, it's almost always a crapshoot. PAF is rarely used below that 8K marker. This guy really needs to do his homework. A better comparison (albeit one that wouldn't have very many instances) would be to compare how many times a contestant has used both ATA and PAF on the same question and see what the results are. Of course, if that happened, the ATA results would have to have been ambiguous enough to use a second lifeline, but still the conclusions drawn from that comparison would be much more valid than taking the entire process as a whole.
OK, that's our on-topic post of the day....