Current Filter: Document>>>>>>

   

Big data has become a reality. But it is not the same reality for every company, or every user. The explosion of data is creating different problems and opportunities. The medical provider required to store scanned images for each patient's lifetime faces a very different challenge to the FMCG brand now offered an unprecedented depth of customer purchasing behaviour data. The end user despairing over the time taken to locate a file or email has a different set of challenges to the legal team struggling with new, big data inspired compliance demands.

A recent Gartner survey of 720 companies asked about their plans to invest in big data gathering and analysis, and revealed that almost two-thirds are funding projects or plan to this year, with media/communications and banking firms leading the way. The research firm insists 2013 is the year of experimentation and early deployment for big data.

Adoption is still at the early stages with less than 8% of all respondents indicating their organisation has deployed big data solutions. 20% are piloting and experimenting, 18% are developing a strategy, 19% are knowledge gathering, while the remainder have no plans or don't know.

This is, therefore, a critical phase in the big data evolution. While storage costs have come down in recent years, organisations cannot possibly take a 'store everything' approach to big data and hope to realise the full long term benefit. The issue is not only what data to retain and where but how to extract value from that data - not just now but in the future as big data technologies, including analytics, become increasingly sophisticated.

In addition to the huge expansion in data volumes, organisations also now have access to new content types. While this depth of data offers exciting opportunities to gain commercial value, it also creates significant management challenges. How should the business protect, organise and access this diverse yet critical information that increasingly includes not only emails and documents but also rich media files and huge repositories of transaction level data?

At the heart of a successful big data strategy is the ability to manage the diverse retention and access requirements associated with both different data sources and end user groups. While today a large portion of the data in a typical enterprise does not get regularly accessed for a year or more, this is definitely set to increase as big data strategies evolve. Many organisations are gleefully embarking upon a 'collect everything' policy on the basis that storage is cheap and the data will have long-term value.

Certainly inexpensive cloud-based storage is enabling big data strategies. But the reality is that while it is feasible to store all the data in the cloud, even with fast connections retrieving that 5Tb of data from the cloud back into the organisation would take an unfeasibly long time. Furthermore, cloud costs are increasing, especially as organisations add more data; and even cheaper outsourced tape backup options still incur escalating power and IT management costs.

In addition, the impact of unused data sitting on primary storage extends far beyond higher backup costs; time consuming end user access leads to operational inefficiency and raises the risk of non-compliance.

Organisations cannot take a short term approach to managing the volumes of big data and hope to realise long term benefits. There is a clear need to take a far more intelligent approach to how, where and what data is stored. Is it really practical to take a backup of an entire file server simply because some of the documents need to be retained for several years to meet compliance requirements? Or is there a better way that extracts the relevant information and stores that in a cheaper location, such as the cloud?

To retain information and avoid a cataclysmic explosion in data volumes, organisations need to take a far more strategic approach to data archive and backup. What information must be kept on expensive local storage, and what can be sent to the cloud or another location? And what policies will be put in place to take data ownership away from end user control? By taking a strategic approach to archiving data, based on the property of each data object, organisations avoid the problems caused by end users applying their own 'retain everything' policies.

By deleting the local data source and moving it to a virtual data repository, an organisation avoids duplication and inconsistency whilst still ensuring information can be retrieved in a timely and simple fashion. Policy driven rules for data retention can be based on criteria such as file name, type user, keyword, tagging or Exchange classifications, while tiering can be applied based on content rules to any target, including tape or cloud.

Page   1  2