±Your Account
Membership:
New Today: 0
New Yesterday: 4
Overall: 24209
Visitors: 57±Latest Webinar
±Latest Articles
· Android Forensics
· Geo-tagging & Photo Tracking On iOS
· KS – an open source bash script for indexing data
· Mobile Device Geotags & Armed Forces
· Categorization of embedded system forensic collection methodologies
· Interpretation of NTFS Timestamps
· What are ‘gdocs’? Google Drive Data – part 2
· What are ‘gdocs’? Google Drive Data
· Bad Sector Recovery
· Forensic Artifact: Malware Analysis in Windows 8
· Geo-tagging & Photo Tracking On iOS
· KS – an open source bash script for indexing data
· Mobile Device Geotags & Armed Forces
· Categorization of embedded system forensic collection methodologies
· Interpretation of NTFS Timestamps
· What are ‘gdocs’? Google Drive Data – part 2
· What are ‘gdocs’? Google Drive Data
· Bad Sector Recovery
· Forensic Artifact: Malware Analysis in Windows 8
±Follow Us
±Latest Jobs
Back to top
Skip to content
Skip to menu
Back to top
Back to main
Skip to menu
Go to page 1, 2 Next
analyzeMFT - a Python tool to deconstruct the Windows NTFS
analyzeMFT - a Python tool to deconstruct the Windows NTFS
Posted: Fri Jan 22, 2010 11:43 pm
I recently wrote analyzeMFT - a Python tool to deconstruct the Windows NTFS $MFT file
The project page is here: www.integriography.com
From the page:
Overview:
analyzeMFT.py is designed to fully parse the MFT file from an NTFS filesystem and present the results as accurately as possible in a format that allows further analysis with other tools. At present, it parses the attributes from a $MFT file to produce the following output:
* Record Number
* Good - if the entry is valid
* Active - if the entry is active
* Record type - the type of record
* Parent Folder - The file's parent folder
* Record Sequence - the sequence number for the record
* For the standard information attribute:
o Creation date
o Modification date
o Access date
o Entry date
* For up to four file name records:
o File name
o Creation date
o Modification date
o Access date
o Entry date
* Object ID
* Birth Volume ID
* Birth Object ID
* Birth Domain ID
* And flags to show if each of the following attributes is present:
o Standard Information, Attribute List, Filename, Object ID, Volume Name, Volume Info, Data, Index Root, Index Allocation, Bitmap, Reparse Point, EA Information, EA, Property Set, Logged Utility Strea
For each entry in the MFT a record is written to an output file in CSV format.
analyzeMFT will run on any system with Python installed. A standalone Windows executable will be released when possible.
Contributions and suggestions for improvement are quite welcome.
Sample output:
Sample output:
"Record Number","Good","Active","Record type","Parent Folder","Record Sequence","Filename #1","Std Info Creation date","Std Info Modification date","Std Info Access date","Std Info Entry date","FN Info Creation date","FN Info Modification date","FN Info Access date","FN Info Entry date","Object ID","Birth Volume ID","Birth Object ID","Birth Domain ID","Filename #2","FN Info Creation date","FN Info Modify date","FN Info Access date","FN Info Entry date","Filename #3","FN Info Creation date","FN Info Modify date","FN Info Access date","FN Info Entry date","Filename #4","FN Info Creation date","FN Info Modify date","FN Info Access date","FN Info Entry date","Standard Information","Attribute List","Filename","Object ID","Volume Name","Volume Info","Data","Index Root","Index Allocation","Bitmap","Reparse Point","EA Information","EA","Property Set","Logged Utility Stream"
"0","Good","Active","File","5 - 5","1","$MFT","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","","","","","","","","","","","","","","","","","","","","True","False","False","False","False","False","True","False","False","True","False","False","False","False","False"
-------
"110575","Good","Inactive","0","5422 - 5426","3","TRANSFERMGR.EXE-24D2A23F.pf","2009/12/27 18:35:57.625000","2009/12/28 05:32:01.390625","2009/12/27 18:35:57.625000","2009/12/28 05:32:01.390625","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","","","","","TRANSFERMGR.EXE-24D2A23F.pf","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","","","","","","","","","","","True","False","False","False","False","False","True","False","False","False","False","False","False","False","False"
Source:
analyzeMFT is Copyright (c) 2010 David Kovar. All rights reserved. This software is distributed under the Common Public License 1.0.
The source code can be downloaded here.
Background:
My original inspiration was a combination of MFT Ripper (thus the current output format) and the SANS 508.1 study guide. I couldn't bear to read about NTFS structures again, particularly since the information didn't "stick". I also wanted to learn Python so I figured that using it to tear apart the MFT file was a reasonably sized project.
Many of the variable names are taken directly from Brian Carrier's The Sleuth Kit. His code, plus his book "File System Forensic Analysis", was very helpful in my efforts to write this code.
The output format is almost identical to Mark Menz's MFT Ripper. His tool really inspired me to learn more about the structure of the MFT and to learn what additional information I could glean from the data.
I also am getting much more interested in timeline analysis and figured that really understanding the the MFT and having a tool that could parse it might serve as a good foundation for further research in that area.
Output from --help:
Usage: analyzeMFT.py [options]
Options:
-h, --help show this help message and exit
-f FILE, --file=FILE read MFT from FILE
-o FILE, --output=FILE
write results to FILE
-g, --gui Use GUI for file selection
-d, --debug turn on debugging output
Purpose:
analyzeMFT.py is designed to fully parse the MFT file from an NTFS filesystem
and present the results as accurately as possible in a format that allows
further analysis with other tools. At present, it will read an entire MFT
through to the end without error, but it skips over parsing some of the
attributes. These will be filled in as time permits.
Caution:
This code is very much under development. You should not depend on its results without double checking
them against at least one other tool.
Output:
The output is currently written in CSV format. Due to the fact that Excel
automatically determines the type of data in a column, it is recommended that
you write the output to a file without the .csv extension, open it in Excel, and
set all the columns to "Text" rather than "General" when the import wizard
starts. Failure to do so will result in Excel formatting the columns in a way
that misrepresents the data.
I could pad the data in such a way that forces Excel to set the column type correctly
but this might break other tools.
Future work:
1) Figure out how to write the CSV file in a manner that forces Excel to interpret the date/time
fields as text. If you add the .csv extension Excel will open the file without invoking the import
wizard and the date fields are treated as "General" and the date is chopped leaving just the time.
2) Add version switch
3) Add "mftr" switch - produce MFT Ripper compatible output
4) Add "extract" switch - extract or work on live MFT file
5) Finish parsing all possible attributes
6) Look into doing more timeline analysis with the information
7) Improve the documentation so I can use the structures as a reference and reuse the code more effectively
Clean up the code and, in particular, follow standard naming conventions
9) There are two MFT entry flags that appear that I can't determine the significance of. These appear in
the output as Unknown1 and Unknown2
10) Parse filename based on 'nspace' value in FN structure
11) Test it and ensure that it works on all major Windows OS versions
12) Output HTML as well as CSV
See other ToDos in the code
_________________
CISSP, CCE, EnCE, Licensed Private Investigator (CA)
The project page is here: www.integriography.com
From the page:
Overview:
analyzeMFT.py is designed to fully parse the MFT file from an NTFS filesystem and present the results as accurately as possible in a format that allows further analysis with other tools. At present, it parses the attributes from a $MFT file to produce the following output:
* Record Number
* Good - if the entry is valid
* Active - if the entry is active
* Record type - the type of record
* Parent Folder - The file's parent folder
* Record Sequence - the sequence number for the record
* For the standard information attribute:
o Creation date
o Modification date
o Access date
o Entry date
* For up to four file name records:
o File name
o Creation date
o Modification date
o Access date
o Entry date
* Object ID
* Birth Volume ID
* Birth Object ID
* Birth Domain ID
* And flags to show if each of the following attributes is present:
o Standard Information, Attribute List, Filename, Object ID, Volume Name, Volume Info, Data, Index Root, Index Allocation, Bitmap, Reparse Point, EA Information, EA, Property Set, Logged Utility Strea
For each entry in the MFT a record is written to an output file in CSV format.
analyzeMFT will run on any system with Python installed. A standalone Windows executable will be released when possible.
Contributions and suggestions for improvement are quite welcome.
Sample output:
Sample output:
"Record Number","Good","Active","Record type","Parent Folder","Record Sequence","Filename #1","Std Info Creation date","Std Info Modification date","Std Info Access date","Std Info Entry date","FN Info Creation date","FN Info Modification date","FN Info Access date","FN Info Entry date","Object ID","Birth Volume ID","Birth Object ID","Birth Domain ID","Filename #2","FN Info Creation date","FN Info Modify date","FN Info Access date","FN Info Entry date","Filename #3","FN Info Creation date","FN Info Modify date","FN Info Access date","FN Info Entry date","Filename #4","FN Info Creation date","FN Info Modify date","FN Info Access date","FN Info Entry date","Standard Information","Attribute List","Filename","Object ID","Volume Name","Volume Info","Data","Index Root","Index Allocation","Bitmap","Reparse Point","EA Information","EA","Property Set","Logged Utility Stream"
"0","Good","Active","File","5 - 5","1","$MFT","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","2007/07/31 19:16:13.734373","","","","","","","","","","","","","","","","","","","","True","False","False","False","False","False","True","False","False","True","False","False","False","False","False"
-------
"110575","Good","Inactive","0","5422 - 5426","3","TRANSFERMGR.EXE-24D2A23F.pf","2009/12/27 18:35:57.625000","2009/12/28 05:32:01.390625","2009/12/27 18:35:57.625000","2009/12/28 05:32:01.390625","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","","","","","TRANSFERMGR.EXE-24D2A23F.pf","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","2009/12/27 18:35:57.625000","","","","","","","","","","","True","False","False","False","False","False","True","False","False","False","False","False","False","False","False"
Source:
analyzeMFT is Copyright (c) 2010 David Kovar. All rights reserved. This software is distributed under the Common Public License 1.0.
The source code can be downloaded here.
Background:
My original inspiration was a combination of MFT Ripper (thus the current output format) and the SANS 508.1 study guide. I couldn't bear to read about NTFS structures again, particularly since the information didn't "stick". I also wanted to learn Python so I figured that using it to tear apart the MFT file was a reasonably sized project.
Many of the variable names are taken directly from Brian Carrier's The Sleuth Kit. His code, plus his book "File System Forensic Analysis", was very helpful in my efforts to write this code.
The output format is almost identical to Mark Menz's MFT Ripper. His tool really inspired me to learn more about the structure of the MFT and to learn what additional information I could glean from the data.
I also am getting much more interested in timeline analysis and figured that really understanding the the MFT and having a tool that could parse it might serve as a good foundation for further research in that area.
Output from --help:
Usage: analyzeMFT.py [options]
Options:
-h, --help show this help message and exit
-f FILE, --file=FILE read MFT from FILE
-o FILE, --output=FILE
write results to FILE
-g, --gui Use GUI for file selection
-d, --debug turn on debugging output
Purpose:
analyzeMFT.py is designed to fully parse the MFT file from an NTFS filesystem
and present the results as accurately as possible in a format that allows
further analysis with other tools. At present, it will read an entire MFT
through to the end without error, but it skips over parsing some of the
attributes. These will be filled in as time permits.
Caution:
This code is very much under development. You should not depend on its results without double checking
them against at least one other tool.
Output:
The output is currently written in CSV format. Due to the fact that Excel
automatically determines the type of data in a column, it is recommended that
you write the output to a file without the .csv extension, open it in Excel, and
set all the columns to "Text" rather than "General" when the import wizard
starts. Failure to do so will result in Excel formatting the columns in a way
that misrepresents the data.
I could pad the data in such a way that forces Excel to set the column type correctly
but this might break other tools.
Future work:
1) Figure out how to write the CSV file in a manner that forces Excel to interpret the date/time
fields as text. If you add the .csv extension Excel will open the file without invoking the import
wizard and the date fields are treated as "General" and the date is chopped leaving just the time.
2) Add version switch
3) Add "mftr" switch - produce MFT Ripper compatible output
4) Add "extract" switch - extract or work on live MFT file
5) Finish parsing all possible attributes
6) Look into doing more timeline analysis with the information
7) Improve the documentation so I can use the structures as a reference and reuse the code more effectively
9) There are two MFT entry flags that appear that I can't determine the significance of. These appear in
the output as Unknown1 and Unknown2
10) Parse filename based on 'nspace' value in FN structure
11) Test it and ensure that it works on all major Windows OS versions
12) Output HTML as well as CSV
See other ToDos in the code
_________________
CISSP, CCE, EnCE, Licensed Private Investigator (CA)
-

kovar - Senior Member
Re: analyzeMFT - a Python tool to deconstruct the Windows NTFS
Posted: Fri Jan 22, 2010 11:53 pm
Sounds interesting. I analyse MFT records using WinHex and block colouring. I'm always in favour of using CSV as an output format since it allows the user to present the data as the see fit.
_________________
Tony Patrick, B. Inf Tech, CFCE
www.patrickcomputerfor...s.com/blog
www.twitter.com/Patrick4n6
_________________
Tony Patrick, B. Inf Tech, CFCE
www.patrickcomputerfor...s.com/blog
www.twitter.com/Patrick4n6
-

Patrick4n6 - Senior Member
Re: analyzeMFT - a Python tool to deconstruct the Windows NTFS
Posted: Fri Jan 22, 2010 11:59 pm
Greetings,
One of the things I'd like to add to it is the combination of the CSV output and HTML output containing colored hex output. That sort of module could come in handy for other forensic analysis as well.
-David
_________________
CISSP, CCE, EnCE, Licensed Private Investigator (CA)
One of the things I'd like to add to it is the combination of the CSV output and HTML output containing colored hex output. That sort of module could come in handy for other forensic analysis as well.
-David
_________________
CISSP, CCE, EnCE, Licensed Private Investigator (CA)
-

kovar - Senior Member
Re: analyzeMFT - a Python tool to deconstruct the Windows NTFS
Posted: Sat Jan 23, 2010 1:30 am
Have you considered XML output?
-

jhup - Senior Member
Re: analyzeMFT - a Python tool to deconstruct the Windows NTFS
Posted: Sat Jan 23, 2010 6:35 am
David,
Very cool! Can you provide this as an EXE?
Very cool! Can you provide this as an EXE?
-

keydet89 - Senior Member
Re: analyzeMFT - a Python tool to deconstruct the Windows NTFS
Posted: Sat Jan 23, 2010 7:56 am
Greetings,
I'd not considered XML output but am happy to do so. Question is - is there a particular template I should follow?
I will produce it as an .exe when I get home. I did so just before I left and ran into cross platform problems. I think I know how to fix those but this is kinda virgin territory to me.
-David
_________________
CISSP, CCE, EnCE, Licensed Private Investigator (CA)
I'd not considered XML output but am happy to do so. Question is - is there a particular template I should follow?
I will produce it as an .exe when I get home. I did so just before I left and ran into cross platform problems. I think I know how to fix those but this is kinda virgin territory to me.
-David
_________________
CISSP, CCE, EnCE, Licensed Private Investigator (CA)
-

kovar - Senior Member
Re: analyzeMFT - a Python tool to deconstruct the Windows NTFS
Posted: Sat Jan 23, 2010 8:30 am
David,
> Question is - is there a particular template I should follow?
I get the same question every now and again with respect to RegRipper...oddly enough, when I ask for input on a style sheet, I don't hear back!
I hear you about the compatibility issues. I just went through something the past couple of evenings, trying to get something together for Matt Shannon's demos at DoD Cybercrime next week...
> Question is - is there a particular template I should follow?
I get the same question every now and again with respect to RegRipper...oddly enough, when I ask for input on a style sheet, I don't hear back!
I hear you about the compatibility issues. I just went through something the past couple of evenings, trying to get something together for Matt Shannon's demos at DoD Cybercrime next week...
-

keydet89 - Senior Member
















