Hi,
I need to make a little program that reads MFT from a NTFS drive and lists all files in it. So, reading here and studing about the MFT and NTFS, i figured out how to do it.
Basically, i first read the first clusters from the drive and obtains data like
The bytes per sector, sectors per cluster, and the cluster number of MFT.
Using this info I read data at the sector defined by the start cluster of mft and find the first
attribute that is $MFT.
It's ok, then i continue reading, for example in my disk each mtf record is 1024 bytes, so i continue reading and each portion of 1024 bytes is a file on disk, I take its name and display it.
Everything is ok, i read exactly 102928 sectors and i find the last FILE block in the MFT.
My problem started when i realized that there were some files that were not in that list.
So, i downloaded a program called NTFS Walker which lists all files in a drive read from MFT.
The files are listed in the same order I list them with my program until it reaches the last file
I list in mine. But the list of files from NTFS walker continues and mine, doesn't.
So, I used a HEX editor to read the disk directly and know why my code stopped at some place and did not find any files.
If I read my disk with a hex editor i go to the last sector my code finds a file.
I see that if I go two the next sectors, they have no data and then, other data that doesn't seem to be a MFT file record (i distinguish then because they start with FILE).
So, i made a raw search to find the next file that NTFS Walker showed after the last my code shows, and i found its entry (and all the other) at other sector near the end of the disk. So, i think my MFT is fragmented. What i need to know is, how I know where the next MFT blocks are (or all the others if my mft is fragmented in more that two parts). I thought i could do it by reading the $MFT attribute and analizing the Data runs in the $DATA attribute, but i cant get the value of the sectors where the next part of MFT starts, so, how can i do??
Please Help!!!
Thanks from Argentina
Antonio Briones
.NET Software Developer
escribimeantonio@hotmail.com
The first MFT entry contains a datarun field to show where all the fragments are stored
The first MFT entry contains a datarun field to show where all the fragments are stored
Thank you very much! I've been asking this question in other forums and no one could reply!!.
I thougt it could be this way so started to read about the $DATA attribute and its content to apply this to first sector, the one you refer to, i think the $MFT should be.
So I search for the $DATA attribute, I localize it, 80 00 00 00 (in red) then its lenght 50 00 00 00 (in blue) but I dont know if what Im doing is ok.
I read out there that the $DATA attribute should finish with 00 00 and the, when attributes list finishes, FF FF FF FF.
In my case I find the FF FF FF FF (in green) but not the 00 00.
On the other hand, i read that the lenght (50, 80 in dec) is the byte count from the start of the attribute, so the attribute should finish at the number marked in yellow.
So, where does the data run start? At offset xxxx140?? (32), in that case, why does it start here??
If it starts there, 32 means the lenght is 5 (as i read out there), so the first data run is 42 32 00 00 0C???? the next data run should also start in the next byte, 32, its 5 bytes, so it should be FE 1C C0 D0 FB??? Then 00 00 means there are no more data runs?? and so 4C and 8C what mean??
Anyway, if all this is correct, I found the second part of mft starts at sector 262945984 (each sector 512 bytes), but i can't get this value anyway using the data above!!
Thanks for your help, it's the first place where someone replies and I cant find any help for this anywhere!
Thanks!!
Hi,
it sounds like you are assuming the MFT is one contiguous block. It can be fragmented, which would explain the issue you are getting.
As mentioned you need to process the $MFT entry, so that you get the cluster runs for the whole MFT.
Hi,
it sounds like you are assuming the MFT is one contiguous block. It can be fragmented, which would explain the issue you are getting.
As mentioned you need to process the $MFT entry, so that you get the cluster runs for the whole MFT.
Ok, the point is that I dont know a lot of all this, the $MFT entry youre refering to is the one Im showing above???
Well I cant confirm you found the very start of the $MFT at the sector you mentioned. But the second data run is to be interpreted as follows
Starting cluster 16,502,976 for a length of 7422 clusters.
Well I cant confirm you found the very start of the $MFT at the sector you mentioned. But the second data run is to be interpreted as follows
Starting cluster 16,502,976 for a length of 7422 clusters.
Ok, I think this is the first MFT record, since the file attributes of the next blocks are the ones I read are for ntfs itself and then start the rest of the files blocks.
$MFT
$MFTMirr
$LogFile
$Volume
$AttrDef
.
$Bitmap
$Boot
$BadClus
$Secure
$UpCase
$Extend
$Quota
$ObjId
$Reparse
$RmMetadata
$Repair
$TxfLog
$Txf
$Tops
$TxfLog.blf
$TXFLO~1 $TxfLogContainer00000000000000000001
$TXFLO~2 $TxfLogContainer00000000000000000002
MAINQU~2.QUE MainQueueOnline1.que
CONTEN~2.DIR Contents1.dir
Setup.evtx SETUP~1.EVT
AITEVE~1.003 AitEventLog.etl.003
AGGLGL~1.DB AgGlGlobalHistory.db
PFSVPE~1.BIN PfSvPerfStats.bin
AgRobust.db
mxdwdui.BUD
AGGLFA~1.DB AgGlFaultHistory.db
Panther
So, as you say the second data run is cluster 16502976 with a lenght of 7422.
The first data run is cluster 786432 with a lenght of 12866.
Cluster 786432 * 8 (sectors per cluster) is sector 6291456, which is the sector where this $MFT starts as you see in the pick above.
But If I do the same with the second, 16502976 * 8 = sector 132023808, theres nothing there.
I think I have to add it the previous localization(786432 + 16502976) * 8, in this sector I have nothing.
Anyway, I cant get to the next sector where the next FILE block for the next file is, It should be 262945984
Also, how I know that the firs pick is the firs $MFT block?
I read the first sector, and go to offset 0x30 and the 8 bytes in little endian are the first cluster of MFT as in this pick.
In dec it is cluster 786432, sector 6291456, which is the sector where i find the entry $MFT, in the first pick i enclosed above.
In this entry is where i read the $data attribute and find this data runs ive commented in the previous post.
Ok…. I am doing this from memory and not referring to notes I made while parsing the MFT, I don't have the source code since parting ways with e-fense.
Ideally, you want to process the "attributes list" attribute. Although the MFT is unlikely to be so fragmented that it requires one, other files may… When parsing them, bare in mind it is possible for the attributes list to become "non resident". If it out grows the amount of space available within 1024 bytes, it will need to be stored somewhere else… The attribute list becomes a bit compressed cluster run and once parsed, the cluster run on disk contains the attribute list…
This will allow you to get all of the entries for a given file should it be very fragmented. I suggest you do this, as it's important regardless of file to be certain you always read everything.
My parser started by reading the first 5 * 1024 bytes of the MFT, giving me the entry containing the $MFT. From there, check for an attributes list.
The cluster runs are bit compressed within the Data attribute. The values are signed, so can be positive or negative. If for example you start off at sector 1000, it is possible for the next part of a segmented file to be located prior to sector 1000.
As the Data attribute is bit compressed, what you have said is actually correct. You need to add the next value within the data attribute to your current location.
Hope this helps.
Data runs are offsets frim the previous data run. They can be positive or negative, hence the for a positive value, often a 0 has to be added to the offset if it is inthe range 0x8??? to 0xf???
Thus in your example, the second run will be
Cluster 0xc0000 + 0xfbd0c0 =
or 0xc0000 - 0x42f40
= 0x7d0c0 (approx, not entirely sure about arithmetic)
As the second value is 0xfbd0c0 this will be treated as a negative number. If the lengtn was 4, and not 3, then it would be a positive number.
NB, it is MUCH simpler to think in hex, rather than decimal when working with sectors, offsets etc
