Monday, January 11, 2010

If you don't know me or like my comments...

I may have remarkably sound judgment.

Not really, but I have struggled through the intensely cerebral and math-strewn "Expert Political Judgment" by Philip Tetlock, it would seem fame and acclaim are no proof of accuracy.

I had seen Tetlock's work cited by many political writers and my son gave me the book for Father's Day.  (Yup - just now reaching the end.)


Long story short(er):  Tetlock measures with scrupulous efforts for objectivity, whether "talking heads" add value to political discourse.  His meticulously (the book actually ends at p. 238 with another 80 pages of appendices, bibliography, equations, etc!) foot-noted answer:
Indeed, one of the more disconcerting results of this project has been the discovery of an inverse relationship between how well experts do on our scientific indicators of good judgment and how attractive these experts are to the media and other consumers of expertise.
Another reviewer points out this is hardly a surprise - at least to psychologists and political scientists. He also points out clearly how adding TV and an entertainment aspect to the business of punditry can be unhelpful.
The expert-prediction game is not much different. When television pundits make predictions, the more ingenious their forecasts the greater their cachet. An arresting new prediction means that the expert has discovered a set of interlocking causes that no one else has spotted, and that could lead to an outcome that the conventional wisdom is ignoring. On shows like “The McLaughlin Group,” these experts never lose their reputations, or their jobs, because long shots are their business. More serious commentators differ from the pundits only in the degree of showmanship. These serious experts—the think tankers and area-studies professors—are not entirely out to entertain, but they are a little out to entertain, and both their status as experts and their appeal as performers require them to predict futures that are not obvious to the viewer. The producer of the show does not want you and me to sit there listening to an expert and thinking, I could have said that. The expert also suffers from knowing too much: the more facts an expert has, the more information is available to be enlisted in support of his or her pet theories, and the more chains of causation he or she can find beguiling. This helps explain why specialists fail to outguess non-specialists. The odds tend to be with the obvious. [More]
This struck home.  At USFR or at live FJ events, we smile inwardly when an interviewee or speaker issues a startling prediction or explanation.  It's good TV.  But is is good for our viewers/attendees?  Often I would say it adds little, based on Tetlock's work. But until information consumers express their desires for different or more useful communications, this is pretty much what we have to deliver to stay in business.

I have noticed in my own information-filtering, I am discounting more familiar opinion-vendors and actually constructing predictions and world-views based slightly more on contrary evidence and/or less analyzed data.  If experts really aren't, then we all can be.

No comments: