Join Us!

Notifications
Clear all

NTFS ins and outs  

Page 1 / 5
  RSS
CyberGonzo
(@cybergonzo)
Active Member

Hi,

I have implemented NTFS parsing the last 9 days and I have a nice working version now despite the horror of finding information on topic.

However, on my system, I still run into a few issues that seem to do with marginal behaviour. I found and tried other tools such as spiff NTFS explorer to check on the issues but they don't do better (or worse) than my app.

I can formulate several questions backed up with some data where possible but before I do I was wondering if this is the right place for it ?

Are there people reading this forum that know NTFS well (enough) to go deep with me and try to figure things out ?

In short;
- trouble finding second (/third, …) MFT entries for files that span multiple MFT entries. I think I have a few of those on my system
- File MFT entries without a DATA attribute (probably related to previous, ie spanned MFT entries)
- Issues with a few data runs. This works well for most files but there are some issues with some files. E.g. 10 bytes for an offset yet the 64 bit offset is (obviously) only 8 bytes long etc. So does the Header byte need to be filtered first with a value ? Header &= 0x77 ?

- And a few more questions, but above info to illustrate the issues I'm dealing with.

Cheers,
Peter

Quote
Posted : 04/02/2012 5:41 pm
harryparsonage
(@harryparsonage)
Active Member

Peter

Have you searched the forum? What about this topic, is it any help?

http//www.forensicfocus.com/index.php?name=Forums&file=viewtopic&t=8010

H

ReplyQuote
Posted : 05/02/2012 1:24 am
Patrick4n6
(@patrick4n6)
Senior Member

Try Carrier's book.

ReplyQuote
Posted : 05/02/2012 9:59 am
CyberGonzo
(@cybergonzo)
Active Member

Thanks guys,

@harry

Interesting. Especially the fixup explanation at the end. Still confusing however ? My system should be OK, or do Windows NTFS drivers fixup too ? Weird ?

@patrick

Care to elaborate ? I'm not familiar with that book.

Peter

ReplyQuote
Posted : 05/02/2012 12:39 pm
harryparsonage
(@harryparsonage)
Active Member

Carrier - http//www.amazon.com/exec/obidos/ASIN/0321268172/

H

ReplyQuote
Posted : 05/02/2012 2:20 pm
philh
(@philh)
Junior Member

Hi Peter,

I spent some time, a while back now, implementing some NTFs parsing in C#. I can't recall how far I got with it, but I can thoroughly recommend Brian Carrier's "File System Forensic Analysis" (as already recommended by others in this thread) - I used it almost exclusively whilst writing my software )

Phil H

ReplyQuote
Posted : 05/02/2012 3:44 pm
joakims
(@joakims)
Active Member

What is your application suposed to do exactly? Yeah, the fixups may turn out to be something different than you initially thought (as for me).. The ntfs driver handles this for you if target volume is ntfs, but probably the point is to not rely on that.

Regarding runs (which you probably have issues with), don't forget there may be negative relative moves too, and also that $MFT itself may be fragmented as well.

Btw, what language are you coding in?

ReplyQuote
Posted : 05/02/2012 10:52 pm
CyberGonzo
(@cybergonzo)
Active Member

Hi,

I code in C++ (Builder)
The app is data recovery software and I'm quickly implementing NTFS (basic for now) since I promissed it to my users (and the hired help let me down with nothing to show for it after 10 months). It's right up my alley however, I have written all code so far myself (UDF, FAT, ISO9660, HFS(+) etc.)

NTFS

I have code in place for negative runs (not tested properly yet) and a fragmented MFT is no problem either.

Finding the runs is not a problem for 99% of the files yet there are a few that create issues. For instance an offset nibble that contains 0xA … which is funky because I can only fit 8 offset bytes in a 64bit variable (duh).

This sort of weird behaviour is keeping me from releasing a beta version. I want it to at least work properly on my system. Windows itself has no issues with these files so I'm missing something … but what ?
What does NTFS do with a 0xA nibble ? AND with 0x7 ?
Unless .. unless unless …. Windows treats >8 nibbles as 0 nibbles ? A zero offset means a sparse run … something to check tomorrow.

This runs thing is one thing but I also have no clue how to find MFT records that belong to a base MFT for files spanning more MFT records.
Anybody an idea about that ?

ReplyQuote
Posted : 05/02/2012 11:41 pm
joakims
(@joakims)
Active Member

Yes the runs must be interpreted differently when file is compressed/sparse. Post one of those tricky records so we can have a look.

ReplyQuote
Posted : 05/02/2012 11:59 pm
athulin
(@athulin)
Community Legend

Finding the runs is not a problem for 99% of the files yet there are a few that create issues. For instance an offset nibble that contains 0xA … which is funky because I can only fit 8 offset bytes in a 64bit variable (duh).

Try mounting the volume on Linux, and see if it's NTFS implementation gets it right. If it does, enter the code and see what it does. Or ask on the apporpriate developer's list.

Or try the NTFS.com forum which seems to have a number of NTFS experts around.

ReplyQuote
Posted : 06/02/2012 2:09 am
CyberGonzo
(@cybergonzo)
Active Member

@athulin

I don't have Linux. Never used it in my life, so the learning curve would be too steep right now to get something done soon. Furthermore I think you're suggesting to look into open source code ? Never done that either. I'm not good with reading other people's code. I like to implement from a spec (which we don't have for NTFS, and I asked my contacts at MS, they couldn't give it).

@joakim

Ok I'll post the bytes (runs) for one instance (next post)

ReplyQuote
Posted : 06/02/2012 2:30 pm
CyberGonzo
(@cybergonzo)
Active Member

I implemented runs per this webpage
http//www.reddragonfly.org/ntfs/concepts/data_runs.html

Here's the byte sequence for a huge (3.5 GB file)
A fresh perspective on it may do wonders, maybe I'm looking over the obvious ?

42 20 01 2D 33 AF 00 33 00 00 01 E1 17 CC A3 02 70 01 79 10 03 33 30 A4 02 D5 71 01 42 10 7D CF 81 B7 00 43 ….

What I get from this is two valid runs with sizes not enough to cover the full file. The second run is negative (can you confirm this) and the third one fails for me as the offset is too big (I expect a value of 0 (=sparse) or a value <= 8 to fit in the 64bit offset variable)

ReplyQuote
Posted : 06/02/2012 2:46 pm
CyberGonzo
(@cybergonzo)
Active Member

Good news. I implemented the 'fixup' as explained here
http//www.forensicfocus.com/index.php?name=Forums&file=viewtopic&t=8010

And the runs issue went away !!
I still have to do an actual compare of the 3.5 GB file … but it's certainly looking very good !

ReplyQuote
Posted : 06/02/2012 3:41 pm
joakims
(@joakims)
Active Member

Unless the fixup solved it, I would have the whole record (1024 bytes) next time since it's easier to interpret then. Don't forget to cut off the slack data in the extraction of file (allocated-actual size).

ReplyQuote
Posted : 06/02/2012 9:06 pm
CyberGonzo
(@cybergonzo)
Active Member

It was the fixup !!

> Don't forget to cut off the slack data in the extraction of file

I don't understand what you mean by that ?

On a different but related topic, do you know how to find MFT records that belong to a base MFT record for files that span MFT records ?

Last, Alternate Data Streams (ADS). I understand they are simply DATA attributes in the file. How do I know, when I search for the DATA attributes, that I have the real file stream or an alternate one ? And where is the filename stored for ADSs ?

Cheers.
Made real good progress today after te fixup fix. Several "weird" issues went away.

ReplyQuote
Posted : 06/02/2012 9:15 pm
Page 1 / 5
Share: