Matt Aslett, research director at 451 Research, wrote the foreword for the new book Harness the Power of Big Data, and he shares here some of his thoughts on the topic - and the book.
‘Big Data’ is a curious phrase. Since I first encountered it some three and a half years ago, it has come to be one of the IT industry’s dominant trends, and yet I don’t believe I have come across a single person in that time who will admit to liking the term.
While the term Big Data might be almost universally unloved, it is also now almost universally understood to refer to the realization of greater business intelligence by storing, processing, and analyzing data that was previously ignored due to the limitations of traditional data management technologies.
Those limitations have come to be defined by a combination of words beginning with the letter ‘V’: typically volume, variety, and velocity, but with others increasingly added to the mix (more about which in a moment).
The increased use of interactive applications and websites – as well as sensors, meters and other data-generating machines – means that an increasing volume of data is being produced. At the same time, the cost of storage, processing, and bandwidth has dropped enormously, while network access has increased significantly.
The result is that it is now more economically feasible to store, process, and analyze data that was previously ignored or under-utilized – such as Web and network log data, or social media content – to ensure that the business is performing adequately and efficiently, and to track customer sentiment and generate new revenue-generation opportunities.
Much of the data that is being produced by these interactive applications and data-generating machines lacks the structure to make it suitable for storage and analysis in traditional relational databases and data warehouses. In addition to this variety, the data is also being produced at a velocity that is often beyond the performance limits of traditional systems.
It is interesting to note that since the first edition of this book the authors have added a fourth ‘V’: veracity. This succinctly highlights the fact that tried and tested data management principles – data quality, data cleansing, master data management, and data governance – are not completely redundant in a world of Big Data.
The ongoing importance of existing data management techniques also illustrates another important point about Big Data: that new data management and analytic technologies are being implemented to complement rather than replace traditional approaches to data management and analytics.
Big Data is about adopting new technologies that enable the storage, processing, and analysis of data that was previously ignored. It is also about the adoption of new business processes and analytics approaches that take advantage of that data.
451 Research uses the term ‘Total Data’ to describe the management approach being embraced by early adopters to process and explore Big Data using the range of new technologies available. ‘Total Data’ is not an alternative to ‘Big Data.’ We believe that if your data is ‘big,’ the way you manage it should be ‘total.’
While the authors of this book may not use the term ‘Total Data,’ we see evidence of similar thinking in the numerous Big Data success stories contained within these pages, as well as the portfolio of technologies being assembled by IBM to address its customers’ Big Data needs.
Which brings us back to definitions, and the futility of getting hung up on the meaning of Big Data rather than embracing the associated new data management technologies and approaches. IBM’s research has highlighted that companies that invest in analytics and understanding their business and their customers are significantly more likely to achieve competitive advantage over their rivals.
Whatever you might think of the term Big Data, the adoption of new data processing technologies and techniques that complement established analytics approaches promises to deliver new levels of business intelligence, customer service and revenue opportunities. Many companies are finding that is not something they can afford to ignore.
-- Matt Aslett, research manager, data management and analytics, 451 Research