A report from IDG.net recently discussed “big data,” large-scale data sets, and the tools to manage it, stating that enterprises need planning and preparation in order to gain the most advantage from their information. Described as data “bigger than previous-generation, non-parallel infrastructure could handle,” there exists huge opportunities for developing new tools that manage and exploit these new large-scale data sets.
While traditional business intelligence and ERP (enterprise resource planning) platforms are being adapted to deal with big data, the report opines, native applications developed specifically to handle and analyze this new data are critical. Not only is there more data to analyze, but it also comes from entirely new sources, says the report. Some of this unstructured data (in the form of blogs, Web comments and other information) can’t be crunched the way relational databases are, leading to new data analysis platforms which mine the different types of increasing content.
“New networked devices and applications are collecting more data than ever and more organizations are holding on to it, creating huge demands for storage. In the second quarter of this year, storage companies shipped 5,429 petabytes of disk capacity, up 30.7 percent from last year’s second quarter,” cites the report.
Among the innovative new tools being developed include applications that collect and analyze information from sources such as Twitter and Facebook, gauging trends and individual consumer preferences.
This new proliferation of big data is changing not only business, but earth sciences, biology, psychology and other fields. Enterprises need to better prepare by knowing what they want to find out from their data and then figuring out how to exploit their data.