Small data, huh? The words don’t make for good marketing copy. Certainly the term lacks pizzazz.
On the other hand, “big data” does appear to hold much more promise. 2014 has been proclaimed by some to be the year of big data. IDC has predicted that the market for big data will reach $16.1 billion this year.
But how many organizations have the capability to make the most of big data? GEs of the world, perhaps. Finding patterns and trends in big data likely calls for expertise in data science, a skill not easy to come by.
What about smaller to midsized enterprises? How many of them have been able to learn all there is to learn about data lying in their transactional systems? Throw in unstructured data and the number probably dwindles rapidly.
Perhaps there is a lot more that can be done with the smaller data sets you would typically see in these organizations.
Easier to manage. Better contained. Less noise. Less challenging to put a context around it, test hypotheses and so on.
On the other hand, big data clearly suffers from a noise problem. Possible to mistake noise for the signal and make erroneous conclusions.
With the world growing increasingly digital, volumes of big data are rapidly growing. Which means even more noise, making it even harder to do meaningful analysis. Evidently there are some applications where mining big data can reap huge dividends. However it is a specialized activity that calls for specialized skills. Should everyone be jumping onto this bandwagon? Or is it something to be left to the select few?
Especially for the smaller to midsized organizations, perhaps there should be the equivalent of a small data movement. For example, identify all key business metrics using a methodology like the balanced scorecard. Capture these metrics on a regular basis. Monitor these metrics to stay on top of process and organizational health.
There is a lot more that can be done. Lots of cost savings and process improvement opportunities yet to be discovered.
Small data is the low hanging fruit. Let’s go get it first. Let’s go small.