RAID Data Recovery

RAID Data Recovery (64)

We have had three installments on RAID 5 failure and how the XOR operation gives us insight into discovering the stripe size and drive order for an array. Understanding the next installment is critical to the entire method of reverse engineering a RAID 5 configuration.

We understand that when we use an XOR operation on the same bits we always return FALSE. So, XORing a TRUE and a TRUE give you a FALSE, and XORing a FALSE and a FALSE give you a FALSE. Understanding this, we can hopefully understand the following.

The Master File Table (MFT) Magic number is ‘FILE’. Four ASCII letters represented in HEX format as 46h, 49h, 4Ch, 45h. These HEX values correspond to the letters ‘F’, ‘I’, ‘L’, ‘E’ respectively. Now, when you XOR 46h with 46h you get 00h. This is illustrated in the following diagram.

RAID 5 MFT Recovery

Figure 1

In this installment we will explore the practical use of XORing within the context of a RAID 5 recovery. Although the use of this math function in and of itself does not constitute RAID recovery, there are attributes of the mathematics that lend itself to data signatures that can then be used to acquire the RAID 5 configuration. In order to apply the XOR operation we must first understand the format of the Master File Table (MFT), the very crux of the Microsoft NTFS file system.

 

File systems, like most database handlers, use a simple flat file, index method to do look ups and quick displays. The flat file being the MFT, and then INDX record used for fast lookup and displays. Each has its own use within the file system and to a large part are dependent upon each other.

Since the INDX record is not used in the context of RAID 5 recovery for this particular grouping of tutorials we will dispense with its use within the NTFS file system architecture.

Let us leave it to say that in a more advanced context the INDX record does have its uses when assessing a RAID 5. The MFT however offers the clearest path to drive order and stripe size when using the XOR operation. The figure below (Figure 1) is a generic RAID 5 with three drives. The MFT is exposed using the utility WinHex in order to better illustrate how we use XOR to find the parity block within a RAID 5.

mft-boot-record

Figure 1

 

The last installment of this particular blog offered the basics of RAID 5 recovery technology and used a very simple truth table to illustrate the four (4) states of XOR mathematics. This week we will dig a little deeper into the technology and hopefully offer a clearer understanding of how the normal end user can in fact do RAID 5 recovery.

Once again, XOR mathematics offers four actions that work on a bitwise truth table. This truth table when used properly can in fact help one to not only recover their lost RAID data but find out critical facts like, drive order, stripe size, offset calculations, and RAID 5 symmetry. Let’s first take a look at what XOR math will do on actual live data. In order to do that, we must understand how data is stored on the computer in its most primitive state and how viewing that data in a certain way offers us a method for recovery.

We are going to start with a simple ASCII table which defines certain characters in an eight bit environment. Eight bit of course meaning the size of the data type. So each character has its own eight bit value and can be illustrated in several ways. As an example, let’s take the letter ‘A’. In the ACSII table the letter ‘A’ is represented by the decimal number 65. In binary, displaying all bits, the letter ‘A’ is represented by 01000001. Let’s now take the letter ‘Z’ and its ASCII table representation which is signified by the decimal number 90.  The ‘Z’ in binary is expressed as 01011010. So, to explain this further, when the computer sees the number 65 it translates it into an ‘A’, and when it sees a 90, it translates that into a ‘Z’.

Now that we have the binary representation of the letter ‘A’, and the letter ‘Z’, let’s perform XOR operations on all eight bits and see what happens. Please refer to the previous blog on the XOR operation to see how the result was achieved.

The multitudes of RAID 5 setups are not isolated in the world of Enterprise Servers, home and small business users use them as well. This fact has brought an interesting quandary to the RAID 5 data recovery specialist.

RAID 5, is without a doubt one of, if not the single most difficult recovery procedures in the industry. Along with that difficulty comes a very hefty price tag ranging from as little as $2500.00 to as much as $7500.00 for a three drive RAID 5 data recovery. The pricing range is based on drive size, operating system type, extent of damage, virtualization such as VMWare, and finally time allocated to recover the RAID. The small business, and or home user normally does not have that kind of ready cash to recover the RAID, but, the data to them may be just as important to them as any high end five star enterprise business. There are times when a ‘Do It Yourself’ solution can defray a great deal of the cost and ultimately allow a recovery for pennies on the dollar.

Mass storage media has gone through an extraordinary evolution in the past ten years. From the migration of server side technology only being available to high end enterprise corporate entities to the average end user in their home trying to maintain their critical data the influx of simple storage devices has been astronomical. There are two basic reasons for the market supporting larger mass storage solutions.

First, the amount of data that the end user creates is 100 fold more than it was ten years ago. With the common use of mobile phone technology used to store contact, image, video, and audio file information it is important to make sure that there is enough storage space to maintain all of the data generated by mobile technology. In addition, there is data generated on the personal computer such as tax, email, etcetera that also will take up space as an archival necessity.

Secondly, and more importantly, is the overwhelming requirement to ensure that the data created is safe, secure, and easily accessible. Initially the amount of data created and its importance was negligible but over the years and especially in the past five years, software technology as well as internet speed has made the personal computer an integral part of the day to day chores and entertainment of the average home. Tax data, email data, contact data, and to an even larger degree home business data has become a fulcrum of the personal computer and as such made the data it houses virtually irreplaceable.

With all of this being said the technology to help defray these two needs was poor at best and dangerous in many instances. Before the NAS device the world offered external hard drives. These hard drives were mobile and convenient using a simple USB connection to allow access to the device. The problem with this technology is many fold. First, hard drives are not meant to be mobile, they are meant to be stationary so they do not get bumped of dropped. An external USB drive is susceptible to all of these pitfalls and as such is prone to damage. Second, the drive is not cooled. For the most part USB drives do not have any type of cooling to make sure the drive does not overheat and then ultimately cause a head crash and data loss. Third, the power to a large extent is not steady and or enough to maintain the proper RPM of the drive. This can also lead to data loss and mechanical failure.

When our client's server was physically damaged by an employee they approached us to forensically investigate!

Identifying electronic evidence from a damaged RAID server...

After our client’s mail server stored on a RAID-5 was maliciously damaged by one of their employees, we were instructed by their solicitor to conduct a full forensic examination of the evidence. It was suspected that the server’s hard drives were concealing inappropriate correspondence. The hard drives had been removed and the circuit boards, physically destroyed. To exasperate the situation, backups had been securely erased and could not be retrieved.

Our technicians removed the RAID server from site to ensure preservation of the evidence. Before acquisition and analysis could begin, it was necessary to render the hard drives serviceable in order to take an exact sector level duplicate. We are one of the few computer forensic experts that also specialise in data recovery. The hard drives were dismantled in our clean room and rebuilt using donor parts.

After our client’s mail server stored on a RAID-5 was maliciously damaged by one of their employees, we were instructed by their solicitor to conduct a full forensic examination of the evidence. It was suspected that the server’s hard drives were concealing inappropriate correspondence. The hard drives had been removed and the circuit boards, physically destroyed. To exasperate the situation, backups had been securely erased and could not be retrieved.

Our technicians removed the RAID server from site to ensure preservation of the evidence. Before acquisition and analysis could begin, it was necessary to render the hard drives serviceable in order to take an exact sector level duplicate. We are one of the few computer forensic experts that also specialise in data recovery. The hard drives were dismantled in our clean room and rebuilt using donor parts.

We were then asked to establish whether or not there was sufficient evidence to show that activity was performed during a seven day window and whether the evidence of this activity had been forensically wiped (in addition to physical destruction of the hard disk drives in the RAID). Our data recovery engineers then imaged all the repaired hard drives in the RAID array, taking care to maintain the integrity of the evidence and adhering to Association of Police Chief Officers (ACPO) guidelines at all times.

Thankfully there was no media damage to the platters and we were able to image the drives without error. The acquired images were verified using hash functions and the data parameters calculated in order to rebuild the data on the server.

RAID stands for Redundant Array of Inexpensive Disks. It’s a set of independent hard drives able to provide redundancy, designed to keep the system online, and are mainly used in businesses server environments.

With the continued fall in hard drive prices, it is also becoming more common to find RAID configurations at home. Though these systems are not comparable with enterprise-class RAID systems, they are often used in cheap Network Area Storage (NAS) systems and are connected to form a home network for storing and sharing videos, photos, music and other entertainment content between multiple devices.

On the other hand, enterprise RAID storage architectures perform tasks that are usually business-critical and their efficiency and uptime are paramount. These RAID systems support the operation of virtual environments (eg. VMware & Microsoft Hyper-V), database (eg. Microsoft SQL and Oracle), email systems like Microsoft Exchange Server and all the applications that need performance, reliability and scalability.

In these situations, the failure of a RAID system paired with a data loss is a real disaster to be avoided.

To trust is good, not to trust is better

RAID systems are more reliable than the individual spinning hard drives or individual SSDs. This is true because using multiple disks and providing redundancy they can tolerate the failure of one drive in the array whilst staying online. In the event where there is a RAID system failure, the reconstruction of the information can be done using the parity calculated and stored during normal write operations.

Since to trust is good but sometimes not to trust is better, below follows a collection of comments and suggestions from our experts to help you to work better and be more comfortable with your RAID system.

After RAID 6 comes RAID 10 which combines mirroring and data striping in a nested array. This is also known as RAID 1 + 0 and is very fast and reliable, but does require at least four hard drives to work. Unlike RAID 5 whereby the user will experience data loss if two hard disks fail, RAID 10 can cope with multiple failures with only a minimal impact on performance. Whilst you may think the chances of two or more disks failing is remote, it is certainly not! RAID arrays use identical disks to maintain disk geometry. However this does mean that any manufacturing defects are likely to be evident across all disks in the array. If a hard drive fails, the strain on the others can often cause multiple failures.

The advantages of RAID 10 are clear. Higher performance combined with greater stability is one of the most important. Indeed, all disks in a single mirror have to fail for the system to collapse. Similarly, striping larger mirror sets also allows users to create larger virtual drives. Data can be read back from more than one disk simultaneously, meaning that read speeds are drastically improved.

In contrast, it should be remembered that mirroring means the data must be written twice. RAID 10 quite simply eats disk space, cutting what’s available in half! Also an additional disk drive is required for the mirror set which does effect the write speeds. Yet hard disks are cheap in comparison to data recovery services and this is often a sacrifice many users are willing to take to protect their data.

Setting up RAID 10 couldn’t be easier. We suggest two methods – software RAID and FakeRAID. Software RAID is driven by the operating system which handles all the logic. Fake RAID on the other hand is implemented at BIOS level and is often denoted by motherboard manufacturers. Always use hard drives of identical make, model, interface and capacity, ensuring they are flashed with the latest firmware. If you are using FakeRAID, make sure your motherboard has the ‘on board RAID’ feature. Unlike other RAID levels, Windows has no obvious options for creating RAID 10. So the user has to combine RAID 1 created in Storage Spaces and RAID 0 in the Disk Management utility. It may take a while to format the disks, but it is a straight forward process.

We received a RAID-5 array which had failed following a fire. There were four hard drives in the array and two were showing signs of physical damage to the PCBs. Although administrators had remotely logged in and started to extrcat data from the RAID following evacuation, the server crashed shortly into the process.

The server had all the critical data relating to the business and more importantly - all their client files in PSD, PDF, AI, PNG and JPG formats. There was no reliable off site backup. Tape backups stored on site had also been damaged in the fire and it was decided it would be more economical and effective to recover data directly from the RAID.

Upon examination the PCBs on each hard drive in the RAID had literally melted and when the chassis were opened there was considerable smoke damage. Where the hard drive electronics were damaged, it is normally necessary to completely rebuild the circuitry. However the damage was so extensive that we had to find replacements and reprogram them. Each fire damaged drive was carefully cleaned to prevent contamination of the platters by soot and smoke particles. All the hard drives except one responded well to this process and we were able to gain sufficient 'images' of the raw data from the other three drives. Thankfully the redundancy in RAID5 meant we were still confident of a good data recovery

Page 1 of 5