Post on SCN

Posted the second blog on SCN.
http://scn.sap.com/community/hana-in-memory/blog/2012/11/01/busting-two-myths-on-in-memory-management

There has been surge in the demand of system computing ability as more people have access to the applications with advent of smart phones and tablets. Data needs to be both real time and fast. In Memory Data management is one approach where we achieve the aforesaid objectives. But, the general question which everyone asks about In Memory is whether it is just a technology fad and whether it has been any applicability to the modern day Enterprises.

1. In Memory Data Management is just a technology fad

Fact: It is the need of the hour

Data is growing exponentially. With more automation, more structured data is being captured in our organizations. Also, with the widespread adoption of social media, views ,expressions and opinions are getting captured in Face book status messages, twitter tweets, Instagram’s pictures etc. which are categorized as unstructured data. The explosion of data can be summarized with the fact that the humans have doubled the amount of data that they produced in the last century in the last decade.

Nimble Organization analyzes information at faster speed to stay ahead of their competitors. For Example, Smart organizations are capturing their success of their product releases by analyzing the tweets and messages of its products real time and even making necessay course corrections. To make smart and faster analysis of the dataset so large and complex (which have been named Big Data for their sheer volume), having a faster system performance has become a basic hygiene factor of any system.

Disk Scan speed has only become faster by 10 times over the last 30 odd years. Compared to the data growth, it is not adequate and will add to more scan time. There is a requirement to remove the redundancy involved in scanning and moving the data from disk to CPU for processing. In Memory Data Management removes two time consuming processes to make the processing faster – Scanning the data from Disk and moving it to the main memory (DRAM). As the CPU have direct access to DRAM, the processing power is enhanced exponentially resulting in desired output.

2. In Memory Data Management Concepts are not applicable in Enterprises

Fact: In Memory Data Management concepts have evolved over the years to serve enterprises

Typical Relational Database Management System (RDBMS) stores the data in convient way so that Data can be easily writted on it but scanning these records is time consuming. Also, RDBMS cannot be compressed easily, a basic requirement, to load more data in DRAM. Using Columnar Database makes scan faster and enables more compression but with additional drawback of write back to Database overhead. Due to extensive encoding used for compression, it is expensive to insert, update and delete records in columnar database.

Enterprises end up using both the databases - RDBMS system to write to the database (OLTP Systems) and Columnar DB to scan the DB for analytics (OLAP systems). Data movement across the database apart not only adds overhead to the system but also makes data dated.

In Memory Data Management has evolved to cater to the Enterprise Needs. For Example, HANA, the In Memory Data Management Platform, has the columnar feature but performs only insert operations and avoids the time consuming update and delete operations. It segregates the data into passive data (data which have infrequent use) and active data ( regular/ recent data) for storage and operation optimization. Similarly, small modifications are done in the In Memory Data base to make it optimal for enterprise usage.

In Memory Data Management is the answers to today’s enterprise growing data computing demands. If optimized well, as seen in HANA, it can be deliver amazing results which can analyze a F1 race to de-codifying the genome project. Also falling memory prices and more processing power make In Memory data the most suitable technology to invest for future. Would love to listen to your comments on this article. Please put in your comments.