NTFS: More than one INDX record pointing to the same MFT rec

These are image files sent by people 'out there' from their properly working Windows systems. Where they see my software list a few duplicate folders, but Windows obviously doesn't see them.
I have never seen such a situation myself, during testing, but I am now aware of two cases, from the many tens of thousands of installs, but then most people would probably not bother to report it so that is not a good indication

Well with a little of torture applied we managed to get that at least is not an overly common issue.
(which may mean that it was never noticed before)

I would point out anyway (maybe I am too skeptic) that the fact you never managed yourself to find this situation may mean that this

These are image files sent by people 'out there' from their properly working Windows systems.

may be written as

These are image files sent by people 'out there' from Windows systems that they affirm are working properly.

Subtle difference, still ….

Which software of yours is that?

OT (but not much) and JFYI
http//reboot.pro/topic/21558-files-now-think-theyre-directories/
Never seen that before in quite few years, a directory that is a sparse file AND that CHKDSK does not fix.

jaclaz

> Which software of yours is that?

www.isobuster.com

> Which software of yours is that?

www.isobuster.com

Ahh, good ) this shows how old I am getting ( , I still connected it with just CD/DVD's.

jaclaz

@CyberGonzo
I have seen many strange things with regards to NTFS, but it is hard to just summarize everything in an understandable way here. If you have an example with real data sometime, I can probably assist in sorting it out if you need further help.

Unallocated space

The INDX records could be more correctly called "Non-resident" NTFS indexes. Small indexes are stored purely in the MFT using the $INDEX_ROOT (0x90) attribute. When the index gets a little larger (typically just a handful of filenames) a $INDEX_ALLOCATION (0xA0) attribute is created and the bulk of the index pushed outside the MFT to normal data clusters. The clusters used are recorded in the $INDEX_ALLOCATION attribute and marked as allocated in the $Bitmap file. Each INDX cluster starts with the INDX_CLUSTER_HEADER and, as noted already, uses the VCN field to record the logical position of the cluster in the complete index.

As the directory contents changes the index is frequently rebuilt. If a cluster is no longer required (for instance if the directory has shrunk or maybe just because NTFS felt like storing the data elsewhere) it is marked as unallocated in the $Bitmap file but the contents will likely remain. The INDX record becomes an "orphan" recording the directory contents (with links back to the MFT) at a particular point in time.

Therefore, if parsing an NTFS file system for the INDX header I would suggest considering

a. Is the cluster currently allocated in $Bitmap

Yes - The INDX record is part of a "live" index
No - The INDX record is no longer in a "live" index. It is an orphan from the past

b. Searching the MFT for the $INDEX_ALLOCATION attribute that contains the cluster.

If appropriate, there may also be further mileage in

c. Searching volume shadow copies of the MFT for previous instances

d. Searching the USN Change Journal for operations on the MFT record of interest

There is an excellent explanation of this in Brian Carrier's book. As @joakims suggested, if you can post some example data it may be possible to help more.

Jim

www.binarymarkup.com

I just got back from some R&R and am now mentally preparing for the next mini trip starting Thurday -) Lucky me.

Anyway, I haven't dug in the code yet as I said I would, I figured I'd make an image first to share. The image I work from and which was sent to me is only 5 MB but I converted it to something generic (105 GB) and then compressed it (zip) to roughly 105 MB (still pretty big but doable). Otherwise you'd need to use IsoBuster to do some conversions yourself.

Download the image here
https://www.dropbox.com/s/1wdv4f3kyv3s6ab/Image%20File.ibq.zip?dl=0
So be warned, it will take up 105 GB space after expansion

As said there is only 5 MB data in there, the rest is all zeroes. Which means a lot of the data you may be after may not be in there (e.g. the bitmap). I don't use the bitmap for mere parsing of NTFS (finding files and folders), I understood that that is not needed.

Just look at the root folder.
You'll see (or not ) dupe folders such as "Photos_special" or "Program Files" and more …

Thx for looking with me.

@jaclaz

Indeed old man, indeed )
Try IsoBuster, it's light weight compared to many solutions and still very powerful.
No sales pitch … just proud

Try IsoBuster, it's light weight compared to many solutions and still very powerful.
No sales pitch … just proud

Hi Peter,

On a side note… a couple of the screenshots I took to demonstrate the ClearType and compressed XML issues in the forged report thread (the documents in question were found on CDs) came from IsoBuster. Nice work!

Mark

Hi Peter,

On a side note… a couple of the screenshots I took to demonstrate the ClearType and compressed XML issues in the forged report thread (the documents in question were found on CDs) came from IsoBuster. Nice work!

Mark

Oh, cool !
Thanks Mark

Thanks for the puzzle!

You surely put a little effort into preparing this image )

So I reassembled the $I30 INDX of the root directory. Although some of it (last indx) had been wiped by your tool, and little else of the FS was intact, we may say something. I noticed hardlink count in mft record header is 2 for those 5 affected folders. Also noticed the volume originates back to ntfs 3.0 age. I am not sure what to make out of this scarce information, but I am wondering what chkdsk reports on the original volume? What does Windows itself report back/present in explorer?