As data continues to grow exponentially, data center managers, globally and in India, are facing huge challenges to efficiently manage their data and storage. According to a study by IDC in association with EMC, digital information in India will grow from 40,000 petabytes to 2.3 million petabytes over the next decade (2010 to 2020) — twice as fast as the worldwide rate. As a result, enterprises of all sizes — small, medium and large — will face an increasing challenge to store, protect and manage the rapidly-growing digital information, and comply with backup requirements. This explosion of data, referred to as Big Data in contemporary lingo, has created both the need and appetite for innovation in the storage arena. And so you have technologies such as multi-tiered storage, fluid data and storage virtualization. The other need for efficient data management stems from the very nature of any data, which tends to lose its importance over time. There is a constant need to archive the old and keep the latest or most important data readily available. It is believed that data access follows the 80/20 rule — the most recent 20 percent of the data attracts 80 percent of the access.

“One of the most serious concerns of data center administrators is how they can take care of the growing size of data, the rack space, floor space, and power & cooling requirements. The storage component, which has the maximum mechanical parts, requires maximum power and cooling,” says Rajesh Awasthi, Director - Cloud and Telco, NetApp India.

According to most authorities InformationWeek spoke to, the average utilization rate of storage in most data centers is only about 40-45 percent. This indicates that much of the storage capacity in data centers is underutilized.

Awasthi of NetApp defines storage efficiency as the ability to store maximum amount of data in the smallest possible space and at the lowest possible cost. “It is nearly impossible to predict how long any data file will be retained on the disk. All data begins its life on primary storage. Whether it’s a database entry, user file, software source code file, or an e-mail attachment, this data consumes physical space on a disk  drive somewhere within your primary storage environment. The creation of data on primary storage begins a chain of events that lead to storage inefficiencies,” he says.

Vendor offerings

Traditionally, there has been a gap between management of storage and management of data. Data management which includes management of files, file systems and structured data, has often been a separate discipline from the management of the underlying storage infrastructure. Data administrators have historically concerned themselves with the redundancy, performance, persistence and availability of their data. On the other hand, the storage administrators have focused on delivering physical infrastructure that satisfies the data’s requirements. Typically, the storage is configured first and then within the constraints of the configured storage, data management takes place. If the storage requirements of the data change, the data must be migrated to different storage or the underlying storage must be reconfigured. Either process is disruptive and requires multiple domain-specific administrators to work closely together.

Database giant Oracle attempts to address such gaps with solutions like the Oracle Enterprise Manager. “Offering a single console to manage multiple server architectures and myriad operating systems, Oracle Enterprise Manager’s capabilities include asset discovery, provisioning of firmware and operating systems, automated patch management, patch and configuration management, virtualization management, and comprehensive compliance reporting. An open, extensible system that can be integrated with existing data center management tools, Oracle Enterprise Manager manages across the entire infrastructure stack — from firmware, operating systems and virtual machines, to servers, storage, and network fabrics,” reports an Oracle whitepaper.

According to the report, Oracle Enterprise Manager allows data center staff to observe and take action against energy misuse, and supports viewing energy consumption in terms of real dollars.

The performance of spinning disk-based storage device (which showed no significant improvement) was another challenge that remained unaddressed for many years. That situation has changed dramatically thanks to Flash-based storage devices. For instance, 4 milliseconds can be considered a representative response time for small spinning-disk reads. Flash-based devices can deliver the same read in 0.4 milliseconds, an order of magnitude improvement in the response time.

Oracle’s new Database Smart Flash Cache feature leverages this I/O breakthrough offered by Flash-based storage devices. “Flash cache is a technology available inside the servers and it’s like bringing storage inside the servers,” says Mitesh Agarwal, CTO & Director – Systems, Oracle India.