So is it the flash memory that solves the issue?

Yes. Flash memory has much higher response speed: optimized (in a lab environment) hard disk processes a query on average within 6-7 milliseconds and flash memory does the same within 0,1 millisecond. At the same time flash memory can process tens and hundreds more transactions as compared with hard drive that is limited to 150-200 transactions in a second.

So far flash memory is more expensive and doesn’t suit all tasks for that reason. And, as they have been predicting death of magnetic tape already for several decades, and it is still alive, same way hard drives will live for quite a long time.

How to compare the cost for a company?

Here you have to have a crystal clear understanding of whether company has IT tasks that can bring more profit if they will be performed faster. For example, depositors change the bank because they are frustrated with slow work of the ATMs. If there are such tasks then they can be streamlined with the help of flash memory. One can think that the original cause is in incorrectly written application but quite often problem can be solved by accelerating the disk subsystem. Effectiveness can be improved simply with the help of technologies, without rewriting the application.

Significance of flash memory is not in volumes but in speed and one should apply it for relevant applications and calculate the cost not in dollars per gigabytes but in dollars per transactions (IOPS).

To be more precise, flash memory is good for large data bases that live on high-end disk arrays. If you have a huge application system that eats the lion’s share of your IT budget, it is the first signal to think of it. One data storage system based on flash memory can replace one or even several racks in data processing centre. For example, when SAP users complain that everything is slow, most probably shifting the storage to flash memory can improve things.

If there is a large workstation virtualization project, one might also like to think about resorting to flash memory. We observed that when our clients have hundreds of virtual machines the existing data storage system simply doesn’t cope up with it. Can’t say that data storage systems based on flash memory suit only large companies although they can benefit the most in absolute terms. Even if you are a middle-sized company maybe it would be better to buy 3-4 flash disks that will be able to handle an important application at least not less effectively than 20-40 hard drives will.

How can be flash memory integrated into existing storage infrastructure?

There are several main ways of doing it:

• First way is to install flash memory directly in the server. It is the cheapest option. There are flash hard drives familiar to many of you, there are PCI Express cards that contain flash memory chips. It is inexpensive way of streamlining one server. Although it has a number of shortcomings due to which many companies refused keeping their data on internal disks and moved towards data storage systems: reduced resilience, difficult maintenance, insufficient capacity, inability to use flash memory simultaneously on several servers and so forth. The capacity of flash memory in one server is limited to the number of PCI slots and performance of RAID controller. It is unlikely to get over 2 TB. More advanced and already most common way to keep data – centralized. It means that there is a unified storage system to which information consumers connect via network – servers that solve these or that tasks in a company. It is resilient and one can distribute the resources of this expensive flash memory between several tasks. In my experience, only a few servers of our large clients can overload such data storage system.
• In this case there are several options as well: the first — traditional storage system vendors: IBM, HP, EMC, Hitachi that have been making data storage devices on ordinary mechanical hard drives for years. They support SSD already for several years. Therefore, there is quite a simple way of using flash memory for those who already have such system: buy several flash memory hard drives and install them into data storage system racks. The advantage is in simplicity and that you buy the solution from the time-tested vendor. The disadvantage is that these systems came from the past and they don’t have powerful controllers that contain millions of code strings tailored for mechanics. Even traditional RAID5 is not that effective in the flash world and requires reframing.
• There is a number of new vendors which started the design from scrap in the XXI century. The advantage is that systems have been created specifically to suit the needs of flash memory. They manage flash memory pool as a whole and allow minimizing shortcomings – rewrite cycles limit, insufficient write speed as compared with read speed and so on. One of the brightest examples here is Violin Memory – market leader. Several respected companies have invested in Violin, one of the largest investors is Toshiba that purchased NAND memory. If there is a high load application then one can put it completely on such new storage system. If it is too big and too expensive – move the busiest volumes. Specialized data storage systems scale up to tens and hundreds of terabytes of flash memory.
• And the final approach — using not simply flash data storage system but an attempt to add yet another level of cache into the storage area network (SAN) between servers and existing data storage systems, i.e. when this flash-based caching data storage system contains most relevant data. The approach is very progressive but risky, it is offered by young companies or even start up level companies. But if there is a slightest failure, one can lose data and subsequently money and time. That is why this method is currently an interesting experiment and we cannot recommend it right away. Other options – industrial, time-tested solutions.

What do we get as a result?

Flash accelerates operation of servers, optimizes occupied space on data processing center and saves energy. Today, data storage system based on flash memory are serious competitors to high end arrays. Such arrays are often filled with tens and hundreds of hard drives to give the necessary speed to application, the capacity is often secondary. If the company pays for leasing commercial data processing center, then it is quite a serious argument. Since corporate software like Oracle, SAP and so forth is licensed according to kernels, one can save on license by optimizing processes and reducing number of utilized kernels. If processes spend less computing time waiting for data storage system, they would be able to solve more tasks per unit of time. As a result, you will need less kernels to solve the same task.

And another important nuance: lifespan of flash memory is much longer than of ordinary hard drives, therefore less expenses towards support and less risk of data loss in the event if two disks fail simultaneously (to which standard storage systems are subjected).

In terms of the cost per gigabyte, flash storage systems will lose several more years but in terms of the cost of information processing (cost of transaction) they already exceed traditional systems by several times. There is number of cases in the global practice when huge storage systems have been replaced with small data storage systems based on flash memory with the cost several times less and at the same time producer an incredible acceleration of applications. I would assume that in the future the place of today’s disks with 15K and 10K RPM would be occupied by SLC and MLC chips. Low-speed disks with large capacity (SATA, 7.2K) will be there for many years to come.

Can one combine flash-based data storage systems with traditional data storage systems?

There is a number of tasks which do not require super fast processing speed. Usually you need to identify applications that require increased speed of disk system and move them to flash-based data storage system. The remaining applications on an “ordinary” disk array will breathe freely and their speed will increase as well. Since data relentlessly increase it is never a problem to fill in free space on data storage system.