I'm just in the process of doing some research into NTFS and Windows and I'm a little confused as to how I should handle NTFS sparse files. I'm currently looking at the $UsnJrnl, which is used for update transaction journalling.
It is my understanding that a sparse file is just like any other file within the file system, however the file will contain large sections of zeros, and rather than writing zeros to the disk and essentially wasting space, only a count of the number of clusters containing zero is stored.
As an example, the data runs for the $UsnJrnl on my test system are (obtained using Winhex)
Cluster start 0
Number of clusters 1408
(Sparse)
Cluster start 510119
Number of clusters 128
Cluster start 256
Number of clusters 2448
This means that the $UsnJrnl file is occupying a total of 3984 clusters on the disk, however 1408 of those are sparse, so they aren't actually present on the disk.
So does this mean that the 1408 zero filled clusters are immediately before the 128 clusters starting at 510119?
Essentially what I'm trying to do is to to be able to determine the exact start and end offset of the file on the disk, e.g it runs from cluster x to cluster 512822, however I'm not sure if the sparse clusters are actually allocated directly before the second cluster run, making it one contiguous block, of if they could actually be allocated anywhere.
I hope that makes sense, and any information or advice would be greatly appreciated!
Sparse data is data that doesn't exist on a disk - there are no clusters allocated to it.
So for a file that has the first cluster(s) sparse there is no start cluster on the disk - it doesn't exist.
Essentially what I'm trying to do is to to be able to determine the exact start and end offset of the file on the disk, e.g it runs from cluster x to cluster 512822, however I'm not sure if the sparse clusters are actually allocated directly before the second cluster run, making it one contiguous block, of if they could actually be allocated anywhere.
Sparse clusters don't exist, so they are not allocated anywhere. That's what sparse means only those parts of the file that are stated to contain data (or that have been written to, regardless of value) are associated with physical clusters.
File sparseness is set by a call to DeviceIoControl (FSCTL_SET_SPARSE), and two file offsets the byte offset in the file where the sparse part starts, followed by the byte length of it. You can do this multiple times for the same file.
You can retrieve these by a related call (FSCTL_QUERY_ALLOCATED_RANGES), so the information is fairly certain to be stored somewhere close to the file itself.
It's actually a bit more complex than that, but unless you intend to write your own code, that won't matter.