Creating the file was just a matter of making a copy of a file I knew was resident, and named it "w", with no extension. I knew the original file was 707 bytes, so it was already close to limit. Next I ran my own mftrcrd tool on the file to see visually what it looked like in the MFT record. I ran it like this
mftrcrd_x64 F\w -a attriblist=off indxdump=off
I had the file open in a hex editor and added bytes, then re-ran the tool, and repeated until it turned non-resident.
Point about the $FILE_NAME attribute is that the filename length will impact on the attributes size. The $STANDARD_INFORMATION attributes is as far as I know fixed in size. The record header is also fixed, however was different in earlier implementation (like in 2k I think), in that it did not have file references as is used now (8 bytes). I am not able to reproduce that here, but if you can, then you should be able to increase the max size of resident data by at least 8 bytes (maybe more, not sure).
….
I had the file open in a hex editor and added bytes, then re-ran the tool, and repeated until it turned non-resident.Point about the $FILE_NAME attribute is that the filename length will impact on the attributes size. The $STANDARD_INFORMATION attributes is as far as I know fixed in size. The record header is also fixed, however was different in earlier implementation (like in 2k I think), in that it did not have file references as is used now (8 bytes). I am not able to reproduce that here, but if you can, then you should be able to increase the max size of resident data by at least 8 bytes (maybe more, not sure).
Very good, thanks. D
But this does not answer the original question, which I will try to re-word.
I used three (actually 4) different methods to create a file with the SAME name (which is/was 1.txt) under the SAME account and in the SAME directory, with increasing sizes, until it "exited" the $MFT and it occupied a cluster.
In a "perfect world" the size that should trigger the "migration" of the file from $MFT to "cluster" would be the SAME, then if you change the name, create it under a different folder (permissions) or under a different user (ownership) or on a different system and what not, this "fixed no matter the tool used" limit may change (as an example resulting a bit higher for a shorter filename and a little bit lower for a longer filename).
I will do more tests, in a fully reproducible way, adding a few other "writing approaches" and will check if examining the results with your mftcrd tool I can pinpoint - if not the reason why - at least the HOW/WHERE the diffference comes out.
jaclaz
can you post up the files when they are at the maximum data attribute length for each process?
(I dont know why that sentence was so difficult to phrase)
can you post up the files when they are at the maximum data attribute length for each process?
(I dont know why that sentence was so difficult to phrase)
OK, started again from scratch. )
Here
http//
is a small (around 3 Mb) "floppy-like" image (suitable for being mounted in IMDISK or similar) with a set of pre-made files AND the tools and batches used to make them.
The size was chosen as it is (roughly) the smallest size you can make a NTFS filesystem in windows with a 4096 bytes cluster (i.e. same size as most "common" NTFS filesystems on disk) with the built-in in XP format command.
There is (as seen) a connection between filename length and "available space", but still fsz and fsutil *somehow* manage to make smaller sized files.
744 is confirmed to be the most you can put in a file which filename is within these lengths/form
1
12
1.1
123
The algorithm (valid for "normal" file generation through dsfo or ECHO/ECHOO and similar) is simple enough (by design this applies only up to 8.3 filenames), in batch
SET Max_size=744
SET /A Sum=%Name_length%+%Ext_length%-1
IF %Sum% geq 3 Set Max_size=736
IF %Sum% geq 7 Set Max_size=728
fsz and fsutil "steal" 😯 8 bytes from the above.
jaclaz
Re-uploaded file here
http//
(just in case someone wants to try and reproduce)
jaclaz
This is a cool Friday afternoon problem. )
I read through your reboot.pro thread and checked the provided
NTFS FS and as far as I understand there are four processes you used for your test files
- echo (738 byte limit)
- fsutil (728)
- dsfo (736)
- fsz (720)
Two of these are 3rd party tools specifically designed to manage FS data. As "echo" and "fsutil" are the only commands created by Microsoft, I think it is best to rely on them for determining "default" NTFS behaviour. So let's focus on them.
The nature of the file "creation" is not the same in your tests. As far as I can tell, your batch files perform the following operations
- echo - APPEND an existing file 1 byte at a time
- fsutil - CREATE new files in increasing sizes
I think there might be some key differences here. For my own test, I created a 4 byte file using a hex editor and saved it, using a standard Windows Save File dialog, to my NTFS volume. At this point, the data is RESIDENT.
I then expanded the file to 704 bytes - well within our limit - and after saving it went NON-RESIDENT. While initally this was confusing (as in your example, an APPENDed file went up to 738 bytes) it turns out that my other attributes were 320 bytes in length, so I only have 696 bytes available for resident data.
However - if I used the "Save As" dialog and created a new file - using the same 704 bytes of data - the new file is RESIDENT. So I can see different behaviour for "CREATE" and "APPEND" actions.
It is possible, therefore, that in the inner workings of the NTFS driver there is a different upper limit for the size of resident files depending if they are created or appended.
@Chris_ed
Yes, though you cited some preliminary results of tests done before creating the (hopefully) reproducible file just re-posted. 😯
My "final" results are as follows (as stated in my previous post and hinted in the readme file inside the given image
test_MFT_file.cmd is the demonstration that there is a difference between creating a file with fsz.exe or fsutil (MS tool) and with plainer methods.
The result of using dsfo are the same (tested) as using ECHO or ECHOO.COM, i.e. anything that "adds" bytes to the file.
fsz 736
fsutil 736
dsfo 744
echo (or echoo.com) 744
So there definitely is a difference between "create" and "append" ) , and this difference seems to be a constant of 8 bytes and is "common" between "original MS" and "corresponding third party".
Whatever "mechanism" fsutil uses it seems the same as the one fsz uses (create) and whatever mechanism ECHO uses, it seems the same as the one dsfo or echoo.com use (append).
The final result remains that given the optimum conditions
#1 a very short filename and #2 using the "append" method, as stated here
http//www.forensicfocus.com/Forums/viewtopic/p=6565939/#6565939
you cannot "stuff" more than 744 bytes in a resident file.
The "new" frontier of experimenting would be to see how many bytes can be stuffed on a 4 Kb "native" disk/filesystem (where the $MFT record is - I believe - the minimum accessible size, i.e. 4096 bytes instead of 1024).
jaclaz
Only to keep things as together as possible, just tested 4096 bytes sector see here
https://www.forensicfocus.com/Forums/viewtopic/p=6587693/#6587693
…. I quickly made one and tested the effect on a file "size.dat" enlarged by fsz.exe.
The limit is 3776 bytes, 3777 gets the "dignity" of occupying a cluster
…
this size may vary of a few bytes depending on the actual method that is used to write the file and on the length of the filename, for file size0123.dat the limit is 3768.
jaclaz
Haven't read the thread - maybe someone already commented.
I ran a challenge for my workmates on MFT resident data; I was able to fit just under 4k in JPG data in an MFT record.
It's because I made a disk that had 4k sectors instead of 512 bytes per sector.
Haven't read the thread - maybe someone already commented.
I ran a challenge for my workmates on MFT resident data; I was able to fit just under 4k in JPG data in an MFT record.
It's because I made a disk that had 4k sectors instead of 512 bytes per sector.
Which is EXACTLY the topic of the LAST post before yours (which you may have actually read, even if you didn't read the entire thread).
"just under 4k" is "obvious" since the $MFT record is 4096 bytes in size on a 4096 sectored disk, a more accurate size number (like it was 720-736 for the 1024 record on "normal" 512 bytes) has been just reported as being between 3768 and 3776 bytes.
jaclaz
