Glen Davis

On my flight to Bal­ti­more about two months ago I read The Wis­dom of Crowds by James Surowiec­ki. I actu­al­ly did­n’t plan to buy the book–I just saw an auto­graphed copy at Kepler’s and picked it up on impulse.

The first page of the intro­duc­tion sucked me into wild intel­lec­tu­al romp from which I’m still recov­er­ing.

Since that flight I’ve rec­om­mend­ed it to dozens of peo­ple and pur­chased it for two (to whom I owed a book). I’ve been mean­ing to write about it ever since, but I kept get­ting dis­tract­ed. Plus I saw that two of the blog­gers I read com­ment­ed on it: Jor­don Coop­er and Todd Hunter (who com­ment­ed not once, not twice, but thrice), so I knew the book was get­ting the buzz it deserved.

So what’s the big deal? What’s the idea that is still rock­ing my world? Sim­ply this: giv­en the right con­di­tions, diverse groups of peo­ple col­lec­tive­ly solve cer­tain types of prob­lems bet­ter than experts.

This isn’t a bolt from the blue: the basic idea has been kick­ing around for a long time, but the book is mag­nif­i­cent nonethe­less. The anec­dotes are pre­cise and illu­mi­nat­ing, the data is detailed, doc­u­ment­ed, and con­vinc­ing, and the writ­ing sparkles.

What Prob­lems Do Groups Solve Bet­ter?
There are some prob­lems you need experts to han­dle (prob­lems of skill are the most impor­tant kind: land­ing a plane or oper­at­ing on the brain are good exam­ples), but there are sev­er­al broad types of prob­lems that groups tend to out­per­form experts on:

1. Cog­ni­tion Prob­lems: ques­tions with fac­tu­al answers
   How many jel­ly beans are in a jar?
   Where is a sunken sub­ma­rine?
2. Coor­di­na­tion Prob­lems: how do we all work togeth­er when it’s in our best inter­est to do so?
   How can we dri­ve safe­ly in heavy traf­fic?
   How should we deliv­er this prod­uct to mar­ket?
3. Coop­er­a­tion Prob­lems: how do we work togeth­er when we have diver­gent goals and val­ues?
   How can we con­trol pol­lu­tion while pro­mot­ing indus­try?
   How can bor­row­ers get mon­ey from lenders at the best rate for each?

Under What Con­di­tions Do Groups Solve These Prob­lems Bet­ter?

There are four key qual­i­ties that make a crowd smart. It needs to be diverse, so that peo­ple are bring­ing dif­fer­ent pieces of infor­ma­tion to the table. It needs to be decen­tral­ized, so that no one at the top is dic­tat­ing the crowd’s answer. It needs a way of sum­ma­riz­ing peo­ple’s opin­ions into one col­lec­tive ver­dict. And the peo­ple in the crowd need to be inde­pen­dent, so that they pay atten­tion most­ly to their own infor­ma­tion, and not wor­ry­ing about what every­one around them thinks.
from The Wis­dom of Crowds Q &A, empha­sis added

Note that these cri­te­ria (diver­si­ty, decen­tral­iza­tion, aggre­ga­tion, and inde­pen­dence) often tend to move us towards a solu­tion that not every­one is hap­py with. In Surowieck­i’s own words:

The wis­dom of crowds isn’t about con­sen­sus. It real­ly emerges from dis­agree­ment and even con­flict. It’s what you might call the aver­age opin­ion of the group, but it’s not an opin­ion that every one in the group can agree on. So that means you can’t find col­lec­tive wis­dom via com­pro­mise.
from The Wis­dom of Crowds Q &A

What Can Go Wrong?
When any of the above cri­te­ria are not met, groups often per­form abysmal­ly worse than experts or even iso­lat­ed idiots. Some spe­cif­ic chal­lenges:

1. Cas­cades (p 40f and through­out the book): peo­ple imi­tate each oth­er with­out under­stand­ing and every­body jumps off a cliff because all their friends did. Think about the stock mar­ket in the late 90s.
2. Group­think (p 36): peo­ple don’t feel free to dis­agree and groups reach sub­op­ti­mal deci­sions that almost every­one can see a prob­lem with but no one is will­ing to com­ment on. This is one of the car­di­nal sins of the Assem­blies of God, by the way.
3. Polar­iza­tion (p184-190): peo­ple egg one anoth­er on until the entire group adopts a more rad­i­cal view than any of the mem­bers would have advo­cat­ed going in.

Sum­ma­ry Thoughts
Surowieck­i’s real con­tri­bu­tion, in my esti­ma­tion, is detail­ing the cri­te­ria under which groups out­per­form experts and the con­di­tions under which groups fail cat­a­strop­i­cal­ly.

Also, his end­notes rocked–they’re as good as the foot­notes in Gor­don Fee’s com­men­tary on 1st Corinthi­ans. If you read this book and did­n’t read the notes, go back and read them right now!

The most stim­u­lat­ing idea in the entire book for me was using of arti­fi­cial mar­kets to pre­dict future events (pages 17, 79, 103, 220–221, espe­cial­ly 278–280, and 285). I have no idea how it applies to my con­text, but it was a fas­ci­nat­ing con­cept.

Learn More
You can read an excerpt from the book, read an arti­cle by the author or hear him dis­cuss the book on NPR.

I just read a web page that rang true with my expe­ri­ence: it’s Cam­pus Cru­sade’s Youth and Tech­nol­o­gy Forum.

If you work with col­le­gians (or youth), give it a read. Hat tip: Andrew.