"Shrink data sets" to get grip on rocketing data creation

Businesses are being urged to move to storing smaller “essential” data sets instead of storing everything together as global data creation spirals, as a result of new technologies built around the Internet of Things, artificial intelligence, drones, driver-less cars and other data platforms.

In a new study, Data Age 2025, analyst house IDC is forecasting a 10-fold rise in worldwide data by 2025, swelling to a total of 163 zettabytes (ZB) - equivalent to watching the Netflix catalogue 489 million times. IDC says consumers and businesses creating, sharing and accessing data between any device and the cloud will continue to grow well beyond previous expectations.

And whereas consumers were once the primary creators of the bulk of the world’s data, Data Age 2025 predicts this will change, with enterprises creating 60 percent of the world’s data in 2025. “Business leaders will have the opportunity to embrace new and unique business opportunities powered by this wealth of data and the insight it provides, but they will also need to make strategic choices on data collection, utilisation and location,” said IDC.

There is also the evolution of data from being used for business background needs to becoming “life-critical”. By 2025, said IDC, nearly 20 percent of the data in the “global datasphere” will be “critical” to our daily lives and nearly 10 percent of that will be “hypercritical”. IDC said embedded systems and the Internet of Things will mean that by 2025, an average connected person anywhere in the world will interact with connected devices nearly 4,800 times per day – equivalent to one interaction every 18 seconds.

IDC says automation and machine-to-machine technologies are shifting the bulk of data creation away from traditional sources. While data creation in the previous ten years has been characterised primarily by an increase in entertainment content, “the coming decade will reflect the shift to productivity-driven and embedded data, as well as non-entertainment images and video such as surveillance and advertising”, the analyst said.

IDC analyst Dave Reinsel said: “From autonomous cars to intelligent personal assistants, data is the lifeblood of a rapidly growing digital existence – opening up opportunities previously unimagined by businesses. Technology innovation will be vitally important to evaluate and fully activate the intricacies of what’s contained within this large volume of data – and storage in particular will continue to grow in importance, as it provides the foundation from which so many of these emerging technologies will be served.”

The IDC research was sponsored by data storage company Seagate. Seagate says that if not managed correctly, the data deluge could cause businesses to lose revenue, deliver poor customer experiences and lead to operational inefficiencies. Seagate is urging CEOs to “act now” to make sure their data strategy is focused on storing small sets of data that have “significant business impact” – rather than just storing “everything and anything in the same way”.

Seagate CEO Steve Luczo said: “The era of Big Data is upon us, the value of data is really not in the ‘known’, but in the ‘unknown’ where we are vastly underestimating the potentials today. What is really exciting are the analytics, the new businesses, the new thinking and new ecosystems from industries like robotics and machine-to-machine learning, and their profound social and economic impact on our society.”