The Devil's in the Data and Won't Get Out

Print

He may risk becoming known as Dances with Data at some point.  For now, statistician, data-set analyst, and New York Times blogger Nate Silver has been all but inducted into the Pocket Protector Set's Hall of Fame, and been crowned a rock star for good measure.

Silver's achievements create an understandable draw for the populace:  call it the popularity of prediction wed to perfection.  How does one improve on 100% accuracy in calling the electoral nature of all 50 states, sometimes down to the same fraction of a finish?

Such is the power of crowd-wowing feats in creating a perfect score, so to say, in any area of human enterprise -- especially in dry-seeming areas having few memorably-high scores, and where the possibility or probability of perfection seems an impossible, unknowable dream.

Any acts of acing cryptic endeavors therefore become legend, and are entered into the League and Lore of Really Big Deals.  Such victories become talismans and touchstones for most people -- those who recognize their status as mere mortals and coin-flippers, and who contend with lifetime success rates that waver in the low to middle 50% range.

En route to his eye-opening results, Silver threw off the howls of many critics to his system: Those who were honestly unsure, all the way up (or down) to GOP operatives who accused him of all but being in league with the devil.

In an appearance on the Stephen Colbert show, Silver maintained a mountain of modesty and aplomb while his host provided humorous sniper fire.  Silver said there are, essentially, far more complex tasks than looking at poll numbers, histories, and counting to 270 -- nothing heretical, and nothing as complex as Galileo's accomplishments.

In the end, the telling algorithms are on the wall, or whiteboard, or wherever:  Pundits who share their instincts, from the gut, as it's said, now have exceptionally accurate competition from those using their heads.

Perhaps we can blend art with science, heart with head, human with machine, and get a prediction on the long-term viability of gut instinct -- providing Silver, sleek supercomputers, and America's bookies are all up to the challenge.

* * * * *

Using one's head may now seem an after-the-fact luxury in South Carolina, where pursuit of a supposed industry standard led to the release, via foreign hacking, of 3.6 million tax returns dating back to 1998.

The added salt in the wound is that all of the Social Security numbers -- and about 16,000 debit and credit card numbers on the returns -- were unencrypted.

Why was all that information sitting out in the clear -- clear as a sitting duck, sporting a sign that virtually bellowed, electronically, STEAL ME! to thieves?

It's an old story of cost versus risk.  The story being fronted is that encryption adds layers of cost, although some computer scientists and others would take no small exception.

At any rate, it appears South Carolina officials thought they were gambling on Cheap being relatively safe -- and on their number not coming up -- and they lost.  Everyone did.

The state has set aside $12 million to pay credit-monitoring firm Experian to handle problems that come up for victims of credit card or identity fraud.

Whether encryption adds unacceptable costs or not can be debated, along with whether or not the state of South Carolina dropped the ball in protecting its residents.

In any event, this isn't the last data hacking tale you'll ever read in your lifetime regarding local and national businesses and governments illegally penetrated by the electronic intrusions of individual criminals, groups, and foreign governments.

Suspicious cynics and alternate theorists might even suggest that the data security companies are themselves behind some of the cyber break-ins, generating fear -- and additional, lucrative contracts for their own companies.  No way to really know, one supposes, which only tosses gas on any smoldering fires of paranoia.

In the end, we can call for Security all we like, but that doesn't mean anyone will come when we holler.  Such is the nature of each new modern blessing, packing a hidden new curse.

The Devil's in the data -- sure enough seems to like it in 'ere, and ain't seemin' very likely to come out anytime soon.


Perfecting predictions [story [plus video]:  http://readwrite.com/2012/11/07/nate-silvers-model-proves-to-be-stunning-portrait-of-logic-over-punditry

Behold, the power of logic:  http://readwrite.com/2012/11/07/why-nate-silver-won-and-why-it-matters?&_suid=135240283210205827140416949987

South Carolina's hack job:  http://readwrite.com/2012/11/05/massive-south-carolina-data-breach-shows-your-data-is-always-at-risk?&_suid=135240322374908513519549742341