Expertise during a data explosion

No one really knows how much data is being created in the world. We know that most of the data that exists was created in the past couple of years. Some people say it is doubling every year. The reason that this is even possible comes back to lower barriers of entry. Until we had low cost computing, internet connectivity, and more recently the smart phone, data was isolated, segmented and verified by institutions who were given the authority to create, curate and store it. Authority in this instance was a function of finance. The cost of creating permanent information was expensive – print materials, broadcast hardware, costs of distribution all limited the ability for information to be created and shared. It meant there was far less data, but it also meant we knew where to look to find what was available.

Data has moved from being something which was structured, in know-able places, to something which is unstructured, distributed and without authority. It’s now organic, alive and rapidly evolving. Authority and tools go hand in hand. Now that the tools of creating and storing data are omnipresent and almost free, their is no authority governing it. This means two important things:

  • It will continue to increase exponentially
  • Knowledge no longer has a boundary

So how can anyone be an expert on anything?

In this environment expertise has no choice but to change. No one can know everything, even in the most niche of subjects. If we add to this the idea that the major factors of production are shared – that being 1’s and 0’s – then the potential for cross fertilisation of ideas is infinite. What is true today might be kiboshed tomorrow by new inventions, ideas and collaborations.

The new art of expertise has to become this – knowing where to look and who to rely on.

While we’ll never know everything that has ‘just happened’ and we’ll never be able to predict exactly what is next, we can study the trajectory. Pattern recognition, is quickly becoming more important than knowing. What experts will need to be able to determine in the future is how likely something is, how to assess the sentiment of future behaviour and how to be able to verify what just happened. Expertise is becoming a weird kind of reverse archaeology.

Increasingly what we need to know is how to work with the tools to uncover knowledge as it is created.  The age of memorising things for future reference is quickly becoming obsolete.

– – –

Come join me Tuesday Night in Melbourne to dig into your future and some of The Lessons School Forgot – register here. See you then, Steve.