Thursday, December 22, 2016
Better Data not Bigger Data
Big Data is Big News. Big consultancies and senior managers are all using the term "Big Data" these days. The words are the "Information Superhighway" of this decade*. Over-used, over-hyped and mis-understood.
In practice I tend to refer as something as "Big Data" if I can't inset it & work with it in an Excel spreadsheet... even though we've has databases for half a century that can deal with more than 1,048,576 rows (which is actually the maximum rows you can get in an Excel spreadsheet).
However there can be a lot of insight that can be gleaned from much smaller data sources. You don't need to have access to every single customer record in a database to analyse most trends about your users. You only need to examine a smaller accurate and representative data set. It can't be incomplete, out of date or incorrect.
So shouldn't we really use the term "Better Data"?
*BTW: What is this decade actually called? Sure we had "the Eighties", "the nineties" and even the "noughties"... but are these "the teenies"?