Must read: Behemothic mathematics
IT jargon may come and go, but the authors of Big Data explain why this term is a big deal with the potential to touch all aspects of our lives...
Big Data: A Revolution that will Transform how we Live, Work and Think
Viktor Mayer-Schönberger and Kenneth Cukier
£8.99, John Murray
Everyone has heard of the term “big data”, but no one really knows what it means. That isn’t such a bad thing – it is a new trend. But it will define how society makes decisions concerning everything from healthcare to education to financial management, so it will be crucial for us to understand it.
A good starting point for this is the idea that we can do things with a large body of data that we simply can’t manage with smaller amounts.
We can extract new value from this wealth of material in order to improve what we already do, or to unleash new forms of product or service. In this way, data has become a new vital resource and economic input – the “oil” of the information age. In our book we identify a number of other concepts, including messiness and correlation.
Messiness refers to the idea that we don’t need to persist with our time-honoured interest in clean, highly curated data.
We can instead embrace material that is of varying quality, since the benefits of having a vast amount of data outweighs working with far smaller amounts of more accurate data.
Correlation is the notion that we can put our trust in what the data reveals, even if it’s only an association and we don’t know the causal mechanism. (Of course, when we do take the time and trouble to try to understand what lies behind an association, correlations often point the way towards causality.)
For instance, Walmart, like all retailers, keeps a record of its sales transactions for accounting purposes. A few years ago it reviewed its huge database of old receipts and spotted that, each time a hurricane had struck the US east coast, not only did its sales of storm supplies shoot up beforehand; so did its sales of Pop-Tarts.
It duly placed these sugary snacks next to the torches and batteries at the front of its stores to save time for shoppers and maximise its sales whenever a hurricane was imminent. The company benefited because it was able to crunch a lot of numbers.
Using just a little data, or a sample, may not have revealed the crucial information.
The data may also have been messy, not the pristine material so beloved of statisticians. And the finding is only a correlation – Walmart doesn’t know why Pop-Tarts are such a popular pre-storm purchase.
But big data enabled it to extract new value from old data that had been collected for a different purpose.
As it goes with Walmart, so it goes with every other dimension of society.
Big data is poised to reshape not only business but how we see the world and even ourselves.
Gone are the days when we make decisions on instinct alone. Now we’ll have to base everything on evidence.
We wrote this book to unveil what’s new and explain why it matters – and to provide an urgently needed map of the hazards, as well as the benefits, that lie ahead.