Andrew McAfee wrote an interesting blog post on HBR titled” “Big Data’s Biggest Challenge? Convincing People NOT to Trust Their Judgment“. I have picked a few of the main points below, but the entire post is worth a read. McAfee concludes that we should turn many of our decisions, predictions, diagnoses, and judgments—both the trivial and the consequential—over to the algorithms. When presented with this evidence, a contemporary expert’s typical response is something like “I know how important data and analysis are. That’s why I take them into account when I’m making my decisions.”
This sounds right, but it’s actually just about 180 degrees wrong. McAfee supports his statements with research on cancer screening, Supreme court cases, and parole data. When experts apply their judgment to the output of a data-driven algorithm or mathematical model (in other words, when they second-guess it), they generally do worse than the algorithm alone would.
As sociologist Chris Snijders puts it, “What you usually see is the judgment of the aided experts is somewhere in between the model and the unaided expert. So the experts get better if you give them the model. But still the model by itself performs better.”
Things get a lot better when we flip this sequence around and have the expert provide input to the model, instead of vice versa. When experts’ subjective opinions are quantified and added to an algorithm, its quality usually goes up.
As Ian Ayres puts it in his great book Supercrunchers, “Instead of having the statistics as a servant to expert choice, the expert becomes a servant of the statistical machine.”
So how, if at all, will this great inversion of experts and algorithms come about? How will our organizations, economies, and societies get better results by being more truly data-driven? It’s going to take transparency, time, and consequences:
– Transparency to make clear how much worse “expert” judgment is
– Time to let this news diffuse and sink in
– Consequences so that we care enough about bad decisions to go through the wrenching change needed to make better ones