ALL ABOUT FRAGMENTATION by Lance Jensen, Executive Software Tech Support Rep
I hadn't intended an article on fragmentation, since these articles go to many people who already use Diskeeper, but so many of you have asked for the article on fragmentation that I had to write one!
I'll first discuss fragmentation in general, then take up why it is that fragmentation is the most significant issue in system performance.
Fragmentation
One of Diskeeper's measurements of fragmentation is "average fragments per file." This is a good index of fragmentation on any disk. An average fragments per file value of 1.2 means there are 20% more pieces of files on the disk than there are files, indicating perhaps 20% extra computer work needed.
It should be pointed out, however, that these numbers are merely indicators. Some files are so small that they reside entirely within the MFT. Some files are zero-length. If only a few files are badly fragmented while the rest are contiguous, and those few fragmented files are never accessed, then fragmentation may have no performance impact at all. On the other hand, if your applications are accessing the fragmented files heavily, the performance impact could be much greater than 20%. You have to look further to be sure. For example, if there were 1,000 files and only one of those files is ever used, but that one is fragmented into 200 pieces (20% of the total fragments on the disk), you have a serious problem, much worse than the 20% figure would indicate. In other words, it is not the fact that a file is fragmented that causes performance problems, it is the computer's attempts to access the file that degrade performance.
To explain this properly, it is first necessary to examine how files are accessed and what is going on inside the computer when files are fragmented.
What's Happening to Your Disks?
Tracks on a disk are circular sections divided into sectors. Files are written to groups of sectors called "clusters". Usually files are larger than one cluster, so when the first cluster is filled, writing continues into the next cluster, and the next, and so on. If there are enough contiguous clusters, the file is written in one contiguous piece. It is not fragmented. The contents of the file can be scanned from the disk in one continuous sweep merely by positioning the head over the right track and then detecting the file data as the platter spins the track past the head.
Now suppose the file is fragmented into two parts on the same track. To access this file, the read/write head has to move into position as described above, scan the first part of the file, then suspend scanning briefly while waiting for the second part of the file to move under the head. Then the head is reactivated and the remainder of the file is scanned.
As you can see, the time needed to read the fragmented file is longer than the time needed to read the unfragmented (contiguous) file. The exact time needed is the time to rotate the entire file under the head, plus the time needed to rotate the gap under the head. A gap such as this might add a few milliseconds to the time needed to access a file. Multiple gaps would, of course, multiply the time added. The gap portion of the rotation is wasted time due solely to fragmentation. Then, on top of that, you have to add all the extra operating system overhead required to process the extra I/Os.
Now, what if these two fragments are on two different tracks? We have to add time for movement of the head from one track to another. This track-to-track motion is usually much more time-consuming than rotational delay, since you have to physically move the head. To make matters worse, the relatively long time it takes to move the head from the track containing the first fragment to the track containing the second fragment can cause the head to miss the beginning of the second fragment, necessitating a delay for nearly one complete rotation of the disk, waiting for the second fragment to come around again to be read. Further, this form of fragmentation is much more common than the gap form.
But the really grim news is this: files don't always fragment into just two pieces. You might have three or four, or ten or a hundred fragments in a single file. Imagine the gymnastic maneuvers your disk heads are going through trying to collect up all the pieces of a file fragmented into 100 pieces!
On really badly fragmented files, there is another factor: The Master File Table record can only hold a limited number of pointers to file fragments. When the file gets too fragmented, you have to have a second, maybe third or more, MFT record. For every such file accessed, add to each I/O the overhead of reading a second (or third, or fourth, etc.) file record segment from the MFT.
On top of all that, extra I/O requests due to fragmentation are added to the I/O request queue along with ordinary and needful I/O requests. The more I/O requests there are in the I/O request queue, the longer user applications have to wait for I/O to be processed. This means that fragmentation causes everyone on the system to wait longer for I/O, not just the user accessing the fragmented file.
Fragmentation overhead certainly mounts up. Imagine what it is like when there are 100 users on a network, all accessing the same server, all incurring similar amounts of excess overhead.
What's Happening to Your Computer?
Now let's take a look at what these excess motions and file access delays are doing to the computer.
Windows NT is a complicated operating system. This is a good thing because the complexity results from the large amount of functionality built in to the system, saving you and your programmers the trouble of building that functionality into your application programs, which is what makes Windows NT a truly great operating system. One of those functions is the service of providing an application with file data without the application having to locate every bit and byte of data physically on the disk. Windows NT will do that for you.
When a file is fragmented, Windows NT does not trouble your program with the fact, it just rounds up all the data requested and passes it along. This sounds fine, and it is a helpful feature, but there is a cost.
Windows NT, in directing the disk heads to all the right tracks and clusters within each track, consumes system time to do so. That's system time that would otherwise be available to your applications. Such time, not directly used for running your program, is called overhead.
What's happening to your applications while all this overhead is going on? Simple: Nothing. They wait.
The users wait, too, but they do not often wait without complaining, as computers do. They get upset, as you may have noticed.
The users wait for their applications to load, then wait for them to complete, while excess fragments of files are chased up around the disk. They wait for keyboard response while the computer is busy chasing up fragments for other programs that run between the user's keyboard commands. They wait for new files to be created, while the operating system searches for enough free space on the disk and, since the free space is also fragmented, allocates a fragment here, a fragment there, and so on. They even wait to log in, as the operating system wades through fragmented procedures and data needed by startup programs. Even backup takes longer - a lot longer - and the users suffer while backup is hogging the machine for more and more of "their" time.
Fragmentation Compared to Other Performance Issues
I've stated before that fragmentation is the most significant issue in system performance. The reason: It actually *undercuts* the other major issues, and should be addressed before them.
Fragmentation vs. CPU Speed
A system that does a lot of number crunching but little disk I/O will not be affected much by fragmentation. But on a system that does mainly disk I/O (say a mail server), severe fragmentation can easily slow a system by 90% or more. That's much more than the difference between a 486/66 CPU and a 250MHz Pentium II!
Of course, for the vast majority of computers, the impact of fragmentation will fall somewhere in the middle of this range. In our experience, many Windows NT systems that have run for more than two months without defragmenting have, after defragmentation, at least doubled their throughput. It takes quite a large CPU upgrade to double performance.
You can see that it's a good idea to address fragmentation BEFORE addressing CPU as a performance issue.
Fragmentation vs. Memory
The amount of memory in a computer is also important to system performance; just how important depends on where you are starting from. If you have 16MB of RAM, it's almost a certainty that adding more will tremendously boost performance, but if you have 256MB, most systems would get no benefit from more. I do a lot of very memory- intensive work on my machine at home, and raising the RAM from 32MB to 96MB almost tripled performance. I see this as the high end of possible benefit from adding memory. The typical site, in my experience, will see about a 25% boost from doubling the RAM.
As shown earlier, we generally see more performance improvement from eliminating fragmentation, so the issue of fragmentation should be addressed first.
Fragmentation vs. Disks
Another performance issue constantly addressed is that of disk speed, space, and ways of improving disk I/O.
These are all very important, obviously. But it can be clearly seen from our earlier discussion on fragmentation that all of these issues are undercut by fragmentation. Disk speed is a physical limitation which can't be made faster - yet, with defragmentation, you'll certainly get the most out of a disk's inherent speed. Likewise, disk space is far better utilized, and retrieval time is vastly improved by defragmentation's reduction of disk I/O.
Fragmentation can be seen to be the most important issue in system performance because it undercuts the others. So the moral is, get Diskeeper and handle the fragmentation first!
This article was based on Chapter 4 of the book "Fragmentation - the Condition, the Cause, the Cure" by Craig Jensen, CEO of Executive Software. It has been modified for application to Windows NT.
