The Wisdom of Crowds

On my flight to Baltimore about two months ago I read The Wisdom of Crowds by James Surowiecki. I actually didn’t plan to buy the book–I just saw an autographed copy at Kepler’s and picked it up on impulse.

The first page of the introduction sucked me into wild intellectual romp from which I’m still recovering.

Since that flight I’ve recommended it to dozens of people and purchased it for two (to whom I owed a book). I’ve been meaning to write about it ever since, but I kept getting distracted. Plus I saw that two of the bloggers I read commented on it: Jordon Cooper and Todd Hunter (who commented not once, not twice, but thrice), so I knew the book was getting the buzz it deserved.

So what’s the big deal? What’s the idea that is still rocking my world? Simply this: given the right conditions, diverse groups of people collectively solve certain types of problems better than experts.

This isn’t a bolt from the blue: the basic idea has been kicking around for a long time, but the book is magnificent nonetheless. The anecdotes are precise and illuminating, the data is detailed, documented, and convincing, and the writing sparkles.

What Problems Do Groups Solve Better?
There are some problems you need experts to handle (problems of skill are the most important kind: landing a plane or operating on the brain are good examples), but there are several broad types of problems that groups tend to outperform experts on:

  1. Cognition Problems: questions with factual answers
    How many jelly beans are in a jar?
    Where is a sunken submarine?
  2. Coordination Problems: how do we all work together when it’s in our best interest to do so?
    How can we drive safely in heavy traffic?
    How should we deliver this product to market?
  3. Cooperation Problems: how do we work together when we have divergent goals and values?
    How can we control pollution while promoting industry?
    How can borrowers get money from lenders at the best rate for each?

Under What Conditions Do Groups Solve These Problems Better?

There are four key qualities that make a crowd smart. It needs to be diverse, so that people are bringing different pieces of information to the table. It needs to be decentralized, so that no one at the top is dictating the crowd’s answer. It needs a way of summarizing people’s opinions into one collective verdict. And the people in the crowd need to be independent, so that they pay attention mostly to their own information, and not worrying about what everyone around them thinks.
from The Wisdom of Crowds Q &A, emphasis added

Note that these criteria (diversity, decentralization, aggregation, and independence) often tend to move us towards a solution that not everyone is happy with. In Surowiecki’s own words:

The wisdom of crowds isn’t about consensus. It really emerges from disagreement and even conflict. It’s what you might call the average opinion of the group, but it’s not an opinion that every one in the group can agree on. So that means you can’t find collective wisdom via compromise.
from The Wisdom of Crowds Q &A

What Can Go Wrong?
When any of the above criteria are not met, groups often perform abysmally worse than experts or even isolated idiots. Some specific challenges:

  1. Cascades (p 40f and throughout the book): people imitate each other without understanding and everybody jumps off a cliff because all their friends did. Think about the stock market in the late 90s.
  2. Groupthink (p 36): people don’t feel free to disagree and groups reach suboptimal decisions that almost everyone can see a problem with but no one is willing to comment on. This is one of the cardinal sins of the Assemblies of God, by the way.
  3. Polarization (p184-190): people egg one another on until the entire group adopts a more radical view than any of the members would have advocated going in.

Summary Thoughts
Surowiecki’s real contribution, in my estimation, is detailing the criteria under which groups outperform experts and the conditions under which groups fail catastropically.

Also, his endnotes rocked–they’re as good as the footnotes in Gordon Fee’s commentary on 1st Corinthians. If you read this book and didn’t read the notes, go back and read them right now!

The most stimulating idea in the entire book for me was using of artificial markets to predict future events (pages 17, 79, 103, 220-221, especially 278-280, and 285). I have no idea how it applies to my context, but it was a fascinating concept.

Learn More
You can read an excerpt from the book, read an article by the author or hear him discuss the book on NPR.

Cardinal, Silver, Bronze

42 current or former Cardinals are competing in this year’s Olympics.

True story: last night Andrew was over at our house and mentioned that one of his advisees (Dana Kirk) is competing in the 200m women’s butterfly tomorrow. “I tell her what classes to take.”

That’s kind of wild.

It reminds me of one of my all-time favorite Stanford stories.

Some athletes are in a study group discussing their plans over break. One says he’s heading to Cancun. Another mentions that he’s going hiking in the mountains. The last guy says he’s going to the Masters.

One of the other atheletes knows how hard it is to get tickets, so he asks, “Man, how did you score tickets to the Masters? That’s hard to do.”

The last guy: “Dude, I’m playing.”

“Oh. I knew you were good, Tiger. I didn’t know you were that good.”

note: that’s a true story, but I can’t find the exact reference and so I made up some details.

Stone Jars to Beer Kegs Convertor

I was reflecting on the story of Jesus turning water into wine and began to think about the amount of wine involved. I was trying to figure out how to describe it to college students in a way they could visualize easily.

So what represents lots of alcohol to a student?

All in unison–“a keg!”

So what is the equivalent number of kegs to the six stone jars Christ transformed (or as a tongue-twister, how many kegs could Christ convert)?

A simple internet search helped me discover that Jesus created about 10 kegs of alcohol.

Just in case you ever need to know that.

Best quote about the story:

Nympha pudica Deum vidit, et erubuitRichard Crashaw
The modest water, seeing its God, blushed.

Best joke I know about turning water into wine:

Did you hear about the Baptist deacon who was driving along when he was stopped by the local police. Seeing an empty bottle on the floor, the officer said, “Sir, have you been drinking?”

“No officer, just water,” replied the smiling deacon.

“Then why do I smell alcohol on your breath?” asked the policeman.

“Praise the Lord!” shouted the deacon. “He’s done it again!”