Volume slack vs file system slack; confusing definition

I think there is a mixup of the winapi SetFileValidData and the fsutil switch SetValidData.

Quite right – sorry about that. The API call is SetFileValidData(). Nothing else changes.

I think there is a mixup of the winapi SetFileValidData and the fsutil switch SetValidData.

Quite right – sorry about that. The API call is SetFileValidData(). Nothing else changes.

I see now ).
I corrected my previous posts, please everyone check theirs.
The Windows API function is

SetFileValidData()

the fsutil parameter is

setvaliddata

example
fsutil file setvaliddata <filename> <size>

jaclaz

Slack space is the space in the air conditioned lab room where the lazy staff sits surfing the net, instead of doing the imaging…

File slack is created by many other file systems, not just NTFS.

At present, the only known source of this particular type of slack is NTFS. Strictly speaking, it has only been verified for Windows; status on non-Microsoft implementations of NTFS is still unknown.

Hmmm… What do you call that extra space between the end of the file, and the end of the cluster (or named disk segmentation) in ext2, HFS Plus, ZFS, FAT, or even BDOS of CP/M?

What is "SetValidData()"?

A Microsoft Windows system call – though 'system call' should not necessary be interpreted to mean 'kernel call' or 'BIOS call'. It's implemented by 'kernel32.dll' – that's why I call it a system call. You find the details in the usual MS places – try googling for 'MSDN SetValidData' for one possibility.

I know that EnCase 6 supports this kind of file system structure, though it's not classed as file slack. I'm less sure what EnCase 7 does, though it probably does the same, and I have no idea how other forensic suites or toolkits deal with it.

My point - i think you overcomplicated it.

It may not be a type of file slack that is the most important thing to get right in a list of definitions of slack in general, true. But it does belong on a list of types of file slack.

The SetValidData was a rhetorical question - and after my post the confusion is further demonstrated.

I think it is important to clarify if we are discussing uniquely Microsoft world or the whole world of slack space. Maybe that is why I think sparse files muddle the definition.
mrgreen

Hmmm… What do you call that extra space between the end of the file, and the end of the cluster (or named disk segmentation) in ext2, HFS Plus, ZFS, FAT, or even BDOS of CP/M?

In general, file slack. I call any space 'file slack' that occurs within the extent of a file, and that is not part of the content of a file.

Maybe that is why I think sparse files muddle the definition.

Could you be so kind as to explain to me WHERE inside a sparse file there is "slack"?
Or how a sparse file can "produce" slack?

As I see it, and as said, creating a sparse file or writing data to it does not create any kind of "peculiar" file slack (though it may create, just like any file "sector file slack" or "cluster file slack").

Am I missing something?
Can you fill the gaps?
Can you post an example of "file slack" that is "typical" of sparse files ( and of sparse files only)?

Possibly the difference is that you consider TWO types of "sparse files"

I have seen two types of sparse files. One, where the OS/application interprets a "empty" section of a large data set and compressed it on the disk. Second, where a an application reserves a large amount of disk space by creating a large file with empty content, to be used in future.

In either case, from the drive perspective, file slack may exist. In the first scenario there is no virtual "slack", as the empty space only exists as it is referenced from the OS/application. In the second scenario, there is "internal slack space" that may or may not have been used, but currently not in use.

whilst I consider only the first as being actually "sparse", the second being a form of the "setvaliddata" type.
Can you give some examples of some apps that actually "reserve a large amount of disk space by creating a large file with empty content" BUT that does not use the "setvaliddata" approach AND tyhat does not actually writes 00's to it?

jaclaz

I cannot because I do not disagree with you.

I referred to "slack" in sparse files, but I do not consider it real slack space as we define it.

I cannot because I do not disagree with you.

I referred to "slack" in sparse files, but I do not consider it real slack space as we define it.

Good, then "sparse files" do not belong (in the sense that they don't "deserve a special mention") to the list.

Right now we are still at

1. "file slack" &lt;- unindexed space between file size and allocated space, three types
1. "sector file slack" &lt;- unindexed space between file size and next sector border (Maxsize=SectorSize-1, i.e. on a 512 byte/sector device at the most 511 bytes)
2. "cluster file slack" &lt;- unindexed space between file size and next cluster border

(Maxsize=ClusterSize-1, i.e. on a typical 4096 byte/cluster filesystem at the most 4095 bytes)
"cluster file slack" is comprehensive of "sector file slack" .

3. "valid data file slack" &lt;- unindexed space between actual file size and declared file size (only on NTFS filesystems where the SetFileValidData() function or "fsutil file setvaliddata" has been used to allocate to the file a size exceeding the size of it's contents)

[/listo]

4. "filesystem slack"&lt;- unindexed space within the filesystem
5. "volume or partition slack" &lt;- unindexed space outside the filesystem but inside the partition/volume
6. "disk slack" &lt;- unindexed space outside the partition volume but (obviously) inside the disk

[/listo]

Ideas, corrections, corollaries?

jaclaz

Maybe slack space found within $MFT (kind of like file slack, but for the records) is worth a note. Don't forget that roughly 40-50% of the $MFT is slack space, so if your Master File Table is 400 MB you will have around 200 MB of slack space within in.

Maybe slack space found within $MFT (kind of like file slack, but for the records) is worth a note. Don't forget that roughly 40-50% of the $MFT is slack space, so if your Master File Table is 400 MB you will have around 200 MB of slack space within in.

Yes ) , this may be an addition.

But what contents may it contain? ?
I mean, let's start from the beginning.
You get a brand new disk drive.
From factory it is - unless I am mistaken "fully wiped" or all 00's.
Then *somehow* *someone* partitions it.
Then *somehow* *someone* formats the partition/volume as NTFS and creates the original $MFT.
All the "slack" space of the $MFT is 00's.
During the life of the disk (nowadays seemingly a shorter one than what we were used to) the disk may be re-paritioned/re-formatted several times, possibly without wiping the disk (though with the newer FORMAT behaviour of Vista and later this might happen increasingly more seldom).
I have seen that the location of $MFT is *somehow* (possibly there is a resource that explains the algorithm, but right now I don't recall a source for it) "hardcoded" in the sense that a "normal", "same" OS will place it on the "same" place on a "same" size volume. (on XP it is "usually" 786,432 or 0x000C0000)
So, unless the partition size is not changed "greatly", when a re-format is done, the $MFT will "fall" exactly in the same space.

I have not tested if the previous data "survive" the re-format. 😯

If it does, and the previous data were all 00's, we have nothing to find in it.
If it does not, and the new, even "quick" format re-writes to it 00's, we have again nothing to find in it.

So, provided that it is possible to write data to this "slack" without affecting the normal NTFS operations ? , we are not actually inside the "slack", but rather in what I consider a form of "steganography" i.e. there is no actual "unindexed" space, the areas are indexed as $MFT BUT they contain different data "hidden" there, and there is no possibility that "by chance" some non 00's data that was previously written "normally" to those sectors remains untouched and can be recovered/analyzed.
The exception being maybe IF the partition/volume size was greatly changed and/or another OS (I seem to recall that 2K used a different address for the $MFT) or a non-standard tool was used to do the formatting.
At first sight it seems to me like these conditions may be met rarely.

So IMHO the $MFT can be (I take your word for it, given you are an expert on the specific matter) a very good "hiding place" for data, but not really "slack" in the sense we have been using till now.

OT, but not much, what about "very small" files (the ones that are directly written to $MFT being less than 512 bytes)
http//www.ntfs.com/ntfs-mft.htm
what happens to them when the file is deleted/the volume is re-formatted (quick) or simply when they "grow in size" beyond the limit that makes them "fit" into the $MFT? ?

jaclaz