What is your take on Big Data analytics?
I think the term Big Data is not apt as bigness is the least interesting part. Volume of the data is important but volume is not new; diversity of data is. Big Data is not just the ERP and billing systems — it is also the clickstream data from the Internet and microblog entries on Facebook. For instance, customers blog about their experiences with various banks and companies. To understand what the customers are saying on social media platforms, companies and banks do a lot of analysis, such as natural language processing and sentiment analysis, to collect the data. The complexity of such data is much higher.
Big Data also includes sensor and rich media data like voice recording, videos and so on. In London, for example, while travelling from work to home, a person’s video is captured at least 150 times. This data is used for analyzing crime and homeland security. Analysis of this data is Big Data analytics, which cannot be done with traditional database analytics.
How should CIOs handle the tremendous growth of unstructured data?
I don’t believe that any data is unstructured. We have to overcome this myth that anything that is not in rows or columns is unstructured. The blogs and videos are structured, but non-traditional data. It is difficult for CIOs to handle the non-traditional data as it does not match the traditional data, which existed in the form of rows and columns.
CIOs are struggling to get value from non-traditional data. The key point for CIOs is to use new capabilities like the non-SQL methods of accessing the data. In Teradata we use both — the open source implementation (Hadoop) and the Aster Data technology (that uses MapReduce as an extension of SQL). This technology develops polymorphic file systems that can change shapes according to the data.
How can BI/ analytics help businesses to grow?
Organizations are getting smarter by not only using BI/analytics for collecting information, but also by using it for analyzing customers’ experiences and preferences. They further use this information to create new products.
For example, an automobile insurance company in North America created a new product that helped it in doing customized pricing based on the driving habits of customers. Typically, a traditional automobile insurance is charged based on the customer’s age, address and gender. But this is completely unfair, as a good driver is also charged more. I have faced this myself — the cost of my automobile insurance was more than my car, just because I was a 21-yearold, unmarried male who lived in Boston (the highest risk driving place in the US).
After analyzing several such customer experiences, the automobile insurance company (Teradata customer) decided to allow its customers to get discounts on their automobile insurance if they put a black box in their car. The black box monitors customers’ driving habits. One sample of data is collected every second that includes location and velocity (speed and directions). For one year, the insurance company just collects the data and analyzes it to figure out different types of insurance charges based on driving habits of customers.
For example, if a data scientist finds out that a particular customer never drives above the speed limit, then the customer gets a discount. On the other hand, if a customer shows risky driving habits, then the company charges him more as high car speed can lead to accidents, and in turn lead to more cost to the automobile insurance company.
How would Teradata differentiate itself from its competitors?
We are growing in double digits. We are taking market share away from our competitors because we are not confused; we only focus on analytics. Our competitors are still using old technologies to solve analytics problems, but it doesn’t work pretty well. It is like using a hammer to put a screw in the wall. A case in point is, Bank of America, which was not able to scale up with our competitor’s solution. On the other hand, with our solution the bank was able to save USD 80 million in two years. Leading companies like Walmart, eBay, Vodafone, AT&T, British Airways and United Airlines are also our customers.
Some of our competitors, who talk about in-memory analytics in India, do not understand analytics because the cost per terabyte of in-memory is at least 50 times the cost of mechanical disk drives. The Indian market cannot tolerate that kind of cost and more importantly we do not access all data equally and frequently. It is economically irrational to store all the data at memory prices.
From the massive data available, we frequently access only 20 percent of the data. So, customers want that 20 percent of data to be in high-performance storage and the remaining 80 percent of the data to be in low-cost storage. CIOs want an environment that allows both — optimization for price and performance and optimization for price and storage. In the Indian market, it is very important as the market consumes high volumes of data and is very sensitive towards price. We measure the relevance of data and automatically migrate hot data in the high-performance storage and cold data in low-cost storage. It is completely automatic as today’s hot data will be tomorrow’s cold data. Other database vendors have to manually do this and thus require an army of database managers, which in turn increases the TCO of the company.
What according to you are the upcoming analytics trends for 2012?
Consumer intelligence is emerging as a new trend and the banking sector is the most aggressive adopter of this trend, which is still in its initial stages. For example, banks allow their retail banking customers to directly access their data from the bank’s data warehouse through Internet banking. By using this service, customers document all their cash transactions on the bank’s website. This helps banks to track individual customers and different categories where they spend.
In India, high-end customers would we be the first ones to ride the wave. And eventually, because of the proliferation of Internet access, consumer intelligence would become available to everybody. Other than BFSI, sectors such as retail, energy providers, and telecommunications will adopt this growing trend.