What data doesn’t understand

It’s true data, and our new found ability to sift through large volumes of it, has come with many benefits: fraud detection, genomics, natural language processing to name a few. But, data doesn’t get humanity. It’s just a reflector, not the director. As a tool it has certain biasses built into it. One of which is its ability to take the wide, and make it narrow. It’s also great at finding correlation between the disparate. You know data what it isn’t good at? Detecting boredom.

We humans are weird beings and right at the point when data might tell us something is heading a certain way, we about face, and go in the exact opposite direction, often quicker than anyone expects. Probably because we love variety, nuance and something a little different.

It turns out that computers don’t actually understand – they calculate. The word computer itself used to be a job title of people who literally added things up. The large majority of algorithms we employ calculate the probability of something. That probability calculation will be based on the stack of code it feeds from. And the larger that stack, the deeper and more hidden the bias will be inside it. What this means for us, is that when we change our mind, on a whim, ‘the system’ won’t see it coming.

The stimulus we get as humans comes from the real and messy world we we live in. So much of which still sits outside of the data economy, even with all the tracking we do these days. So what does this mean for us? It means that unexpected change is inevitable, and the data wont tells us it’s coming. We need to look for it ourselves and measure it from personal human experience. Variety is one of the great human desires, and just when something is peaking in popularity, we decide to leave the building for no real reason other than the fact we are human.

Get my blog posts by email here.