NTFS: More than one INDX record pointing to the same MFT rec

@JimC
I said the volume originates back to the ntfs 3.0 age, not that it currently is in that state.

As jaclaz pointed out, the hint about it is in the mft record, and specifically the first 3 that was not wiped. The reason for the difference is because NTFS 3.1 extended the header by 8 bytes to accomodate for the Mft record number (4 bytes) and some padding (2+2 bytes), which is not present in earlier versions. This is the reason why the offset to the update sequence array changed 6 bytes from 0x2A to 0x30. The '*' and '0' as in 'FILE' are strictly speaking not part of that signature, but simply just a separate offset field.

Could also mention how I extracted the INDX's. I open the chunk in a hex editor and located $MFT. Looked up the root dir with name . (dot). Copied the hex values for the datarun for the $INDEX_ALLOCATION attribute, found at MFT offset 0x15B8-0x15D8. Pasted those 0x20 bytes into this tool; https://github.com/jschicht/ExtractFromDataRun, set the size to 0xA000 and opened the 105 GB file. The output file could then be easily parsed with https://github.com/jschicht/Indx2Csv. Here's the output containing all entries, even slack; https://drive.google.com/file/d/0B1M9FFDeCvMpM2puYWVsR1VDRmc/view?usp=sharing

As jaclaz also points out, the filesystem may have been through (years of) torture by various tools. However if NTFS still regards the filesystem as healthy, there is certainly something strange going on. I am therefore still very interested in knowing what chkdsk reports on that volume.

From what I can tell the only good clue is the flag at 0x24 in the INDX header (set in vcn 5) and that the index flag is set for all entries in INDX at vcn 5, possibly meaning that this is the last to interpret as valid data for the current volume. See the output linked further up.

@JimC
You say "the index starts at VCNChild=0x5 so we can start interpreting it at offset 0x5000". How did you arrive at that conclusion? Offset 0x24 in INDX header?

Another check is that the $MFT record on cluster 2 LBA 2064 (first MFT record) is "FILE*" (hence NTFS 3.0) but a number of other $MFT records, just as an example the one for the Receipts folder LBA 30162334 is "FILE0" (hence NTFS 3.1).

So definitely the "torture" thesis is a valid one wink .

The original preamble by Cybergonzo

This is just normal parsing of a good working file system.

While still accurate ) could be bettered into something *like*

This is just normal parsing of a seemingly good working file system notwithstanding the years it suffered the slings and arrows of outrageous tools in the hands of an end customer.

D

jaclaz

Thanks all, for the splendid explanations and analysis.

It does however make me painfully aware of how rusty I am on the deeper mechanics. As said it's been more than 5 years since I implemented this, blessed with no major issues there was no need to revisit this much.

I also notice in the code that I don't check the VCNs of the INDX records.

As far as data goes I never really worry if I see / find more files / folders than the actual OS. But of course (true) duplicates have no use at all.

To answer @Joakims's question I started the interpretation at VCN 0x5 (offset 0x5000) because this was the first VCN pointed to by the $INDEX_ROOT attribute (at offset 0x1568 in the MFT).

You can see this in the annotated MFT dump here http//goo.gl/rwaLT3

The procedure I used was very similar to that proposed by @Joakims. I used BMTK to generate a dump of the MFT file and then found the root directory at offset 0x1400. Within this I found the the $INDEX_ROOT attribute (offset 0x1518) and $INDEX_ALLOCATION attribute (offset 0x1570).

The $INDEX_ALLOCATION attribute lists the mapping pairs or "data runs" for the non-resident index at offset 0x15B0. I used my friendly hex editor to extract these clusters and rebuild the INDX stream. I ran this through BMTK again to generate an annotated dump of the index which I posted here http//goo.gl/Zev1rn

Maybe it would be possible to update Indx2Csv to list the tree node positions and hence identify records which are not in the "live" index?

James

www.binarymarkup.com

New version of Indx2Csv available, now with 2 new fields in output (Vcn and IsNotLeaf); https://github.com/jschicht/Indx2Csv

A (very very) late update, but I ended up parsing the $BITMAP attribute that indeed shows that INDX records 0 - 5 are active, yet not the other ones that are also referenced from the $INDEX_ALLOCATION attribute.

I still look into the 'inactive' INDX records as well, but I'm far more critical of their content and for instance check for duplicates.