Log2timeline on Windows

As I'm rewriting mft2csv, I also thought of adding an option to output in log2timeline format. Now if so (I'm not sure if it even makes sense), but how would that translate?

Per row
1 of 4 timestamps for either Standard Information, Filename1, Filename2, FilenameN attribute.

So for SI and FN1 that would equal 8 rows for the same mft record. ?

Joakim

Joakim,

That's a great question…which is why I prefer the TLN format. All I have to do is identify the source…"MFT_SI" or "MFT_FN", and I not only incorporate the information into my timeline, but this inherently provides the needed context.

Smiley emoticon aside, Dave, I think you misread the quote.

Why do you think that? D

This is kind of what I was alluding to in my previous comment. Given your creation of 4n6time, I'll have to assume that when you "press a few buttons and pop out a timeline", you're referring to either log2timeline or plaso…neither of which parses the AppCompatCache value data.

Correct.

If you're interested specifically in user activity, I seem to remember that it parses UserAssist data, but not other artifacts, such as Jump Lists, etc.

That's right, log2timeline and plaso do not parse every artifact. I am not aware of any tool that does everything, including the expensive ones that require a dongle. However, given that log2timeline and plaso are frameworks, similar to Reg Ripper, if something is missing and you need it, the capability of adding it is always there. In fact I did this with 2 artifacts last week.

This issue comes from analysts who espouse getting "everything", without knowing what their tools actually do and do not get.

I always start a project by mapping out a work plan. If a tool, such as log2timeline, does not provide or meet a specific need (and I don't have the time to "add it"), I'll figure out how to compensate for the deficiency with other tools and/or methods. Jump Lists is a great example, I have used other tools including yours in the past for analyzing this artifact. For any given examination I use at least 2-3 different tools.

And if your methodology works for you…great. I'd bet on an analyst who has a considered, reasoned approach over one who is just pushing buttons any day.

How is running a script and manipulating the output any better (then pushing a button)? As far as buttons, I don't just push them, I build them 8)

I'm not clear on what "methods" you're referring to. Also, I'm a bit surprised that you'd opt to use anything I developed or discussed.

This thread is getting WAY off topic..

If you would like to discuss how I use your tools professionally and in trainings please feel free to contact me offline to keep this thread on topic. You can also reference my blog that has a few references to your tools such as (http//davnads.blogspot.com/2011/11/extending-reg-ripper-every-day.html).

As the person that started the topic I'm very happy with where its gone.
It's been a great discussion that I've enjoyed reading.

I'm a little late to the party. I can't anwser the OP's orginial question but in one full swoop I'll touch on multiple replies

got the latest version of log2timeline and Chris Pogues instructions on how to do it (http//log2timeline.net/INSTALL.txt) .

Just a heads up. If you grabbed L2T 0.65 from the Google code site then you will need to install one more module not reflected in those instructions. You need to also do ppm install JSONXS Before any asks this info has been forwarded along

could not find any MFT Parsers that output to the correct log2timeline CSV format .

Another format to look for is anything that outputs to TLN or bodyfile (Sleuthkit compatible) formats. One example is TZworks ntfs walk (it also suports L2t) http//tzworks.net/prototype_page.php?proto_id=12

If another tool outputs into this format then it will not only work with L2t but also Harlan and TZwork's tools.

great if someone wanted to take on modifying Kovar's AnayzMFT.py script or some other parser to output in the correct format as a pet project. .

Speaking about AnalyzeMFT, the tool already outputs to bodyfile format so it wouldn't take much to make it work with L2T. The only thing that needs to be done is to make the bodyfile format so it is compatible with Sleuthkit. I haven't had the time but the fix should be fairly quick; looking at the code myself or contacting David about this is on my to-do list.

Another option is to use TSK's fls.exe to grab the filesystem metadata info. Yes fls.exe doesn't output the $FNA timestamps but the last MFT update in the $SIA timestamp can catch some timestomping programs. If timestamping is an issue then I would go with something that grab both timestamps. When I'm not worried about timestamping fls.exe has been the fastest way to grab the filesystem metadata.

Better yet, a separate script that converts, TLN output to l2t CSV format would be more useful. .

This already exists; it's in L2t. I can't speak for the new version since I still haven't tested it out yet but pretty much every Perl version can do this. L2T has the mactime and TLN input modules. All you need to do is one of the following

log2timeline.pl -f mactime -w timeline.csv file-in-bodyfile-format.txt
log2timeline.pl -f TLN -w timeline.csv file-in-TLN-format.txt

The first command converts bodyfile format to L2T's csv format while the second converts TLN format to L2T's csv format. I have been leveraging this ability for some time and it allows my to leverage multiple tools for timeline generation. I can use RegRipper to get registry info, Sleuthkit to get fileysystem, TZworks to get NTFS artifacts, and L2T to get other formats. I can bring everything in and output to L2T's csv format. This is fairly easy to do especially since I put together a cheatsheet outlining what each tool is capable of and what commands to run.

you're referring to either log2timeline or plaso…neither of which parses the AppCompatCache value data .

I have to second Harlan on this one. Solely relying on one tool doesn't provide all information that may be relevant. A better option is to combine the functionality of multiple tools. RegRipper has some pretty sweet plugins for timeline creation such as showing program execution (i.e. appcache, userassist, direct). Also, not sure if people read my latest post but the $UsnJrnl file contains a wealth of information that the only tool to provide it in a timeline format is TZworks (based on a past few posts this statement won't be true for long). Lastly, if anyone read David Cowen's post about the $Logfile it also contains a ton of useful information and as of now his tool is the only one to parse the info into a timeline. I guess what it comes down to is what data do you need and what tool or tools can be used to get you that data

As I'm rewriting mft2csv, I also thought of adding an option to output in log2timeline format .

Another option is to go with the bodyfile format that is compatible with Sleuthkit. The reason why I say this is that the majority of the timeline tools support this format. Harlan's tools has a script to parse bodyfile format, L2T has the mactime module to has this format, and the TZworks tools support this format. If you want your tool to be versatile then the bodyfile format will make your tool work with what is already out there.

Corey Harrell
"Journey Into Incident Response"
http//journeyintoir.blogspot.com/

Just a heads up. If you grabbed L2T 0.65 from the Google code site then you will need to install one more module not reflected in those instructions. You need to also do ppm install JSONXS Before any asks this info has been forwarded along

That's right, and thanks Corey for pointing this out to me originally a few months ago. I have since created a batch script that automates the ppm install process a bit. If anyone wants a copy feel free to PM me with your email address.

However, per randomaccess's earlier comments, an EXE would still be ideal. Then again, now we got plaso which is distributed as a EXE amongst other formats -)

One example is TZworks ntfs walk (it also suports L2t) http//tzworks.net/prototype_page.php?proto_id=12

The last time I used TZwork's ntfswalk I observed that time stamps where outputted in inconsistent formats. For instance, a timeline contained time stamp values of "11097" and "110902". This irregularity made it difficult to sort events chronologically and personally for me unusable. I exchanged some emails Jon regarding this issue but unsure if it was ever fixed. Do you know Corey?

This already exists; it's in L2t. I can't speak for the new version since I still haven't tested it out yet but pretty much every Perl version can do this. L2T has the mactime and TLN input modules. All you need to do is one of the following

That's correct and thanks for providing awesome examples on how accomplish this!

Perphaps I was not clear but yes, I was referring to the new version, plaso. That aside, as a general thought, one of the concerns I have with migrating between formats (i.e. TLN format > L2T CSV) is the difference in data points logged. For example, I don't think the TLN format logs the inode value.

Separately, for anyone not familar with Kristinn's new plaso output format, it personally the most comprehensive timeline format I have ever used. Here are some details for anyone interested https://sites.google.com/a/kiddaland.net/plaso/developer/libraries/storage
This will be the default format I use for input and output in my tools moving forward.

Solely relying on one tool doesn't provide all information that may be relevant. A better option is to combine the functionality of multiple tools.

Could not agree more with this comment. Perphaps the community should agree on a standardized output for timeline data? I think this will better position folks for combining the functionality (and the best) of multiple tools. My questions to the community are

Why do we need 3-4 different formats?

What's preventing us from using one common format?

Why do we need 3-4 different formats?

I don't know…good question. What are the different formats?

What's preventing us from using one common format?

Good question. Probably not something that can be addressed in this thread. I took your previous advice and sent you an email directly…maybe something similar should be done with this question (i.e., move to a different thread, change venue, etc.).

What are the different formats?

Below are some I know of

Body file - http//wiki.sleuthkit.org/index.php?title=Body_file

log2timeline CSV - http//code.google.com/p/log2timeline/wiki/l2t_csv

plaso protobuf - https://sites.google.com/a/kiddaland.net/plaso/developer/libraries/storage

TLN - http//windowsir.blogspot.com/2009/02/timeline-analysis-pt-iii.html

Mandiant does some stuff with XML - https://github.com/mandiant/AuditParser

Not a great attitude for an examiner to have, although clearly we're all usually operating under some kind of resource constraint (e.g. of available time, cost etc).

Agreed, but in my experience, it's self-imposed…we may have analysis goals and a time frame, but we end up spending too much time on the wrong thing.

I think the original constraints are often external, but I'd agree that we may constrain ourselves further by spending too much time on the wrong things. That was the reason for my comment that focusing resources on procedures relevant to a case is characteristic of a good examiner.

One of the issues I've been considering lately is the sufficiency of various tools. I recently encountered someone using a popular tool to create a timeline, and when they asked about the issue they were facing, I asked if they'd included specific data…their response was, "yes, it includes system and user activity." However, the tool that they used specifically does NOT incorporate the data in question…not by design, it simply doesn't parse it (yet).

So my thoughts are, when you're creating a timeline, are you incorporating _all_ data, the _right_ data, and/or sufficient data? How do you know?

Tricky one, for sure.

Part of the issue comes down to testing if an examiner wants to be sure that a tool is incorporating a particular type of data then they have to check it. Evidently there was a disconnect between what your correspondent thought they were checking and what their tool was actually checking. This could have been addressed by testing on the part of the examiner.

Regarding 'all' data I'm not sure that an examiner could ever be confident that they'd retrieved all timeline-relevant data from a system in anything but the simplest cases. There could always be relevant data structures present that an examiner has not parsed whether that's because the examiner doesn't know that they exist, knows that they exist but for whatever reason is not capable of interpreting them, or because they're stored in a proprietary or encrypted format that practically can't be interpreted. The same is true for a tool, even though its developers may have drawn on a broader/deeper/more accurate body of knowledge than an individual examiner.

Regarding 'right' or 'sufficient' data at the moment a lot comes down to judgment and experience. The relevance and correct interpretation of some artifacts may depend on the circumstances in which they occur (e.g. combinations and versions of software present now or historically on the examined device). Different examiners and software developers will make different judgments regarding what's 'right' or 'sufficient' based on their breadth and depth of knowledge and on what they've found to be useful in the past. I'm sceptical that we'll ever be able to agree 100% on an objective truth, although undoubtedly orthodoxies will emerge over time.

Part of the issue comes down to testing if an examiner wants to be sure that a tool is incorporating a particular type of data then they have to check it. Evidently there was a disconnect between what your correspondent thought they were checking and what their tool was actually checking. This could have been addressed by testing on the part of the examiner.

I'm not sure that's where the 'disconnect' occurs.

I found out recently (within the past year) that there are a good many analysts who do no tool testing or validation of their own, and tend to use tools 'blindly' (my term), often based on recommendations from others.

Regarding 'all' data I'm not sure that an examiner could ever be confident that they'd retrieved all timeline-relevant data from a system in anything but the simplest cases. There could always be relevant data structures present that an examiner has not parsed whether that's because the examiner doesn't know that they exist, knows that they exist but for whatever reason is not capable of interpreting them, or because they're stored in a proprietary or encrypted format that practically can't be interpreted. The same is true for a tool, even though its developers may have drawn on a broader/deeper/more accurate body of knowledge than an individual examiner.

I agree whole-heartedly, and that's why I cringe whenever I hear someone say, "I want to see everything…". Very often, if I can engage them further, I find that they're using one tool, and that's it. And given any particular image that they're examining, I can often point out half a dozen things that fall in the "everything" bucket that the tool doesn't even parse.

This not an indictment of a tool, most in particular, not the one mentioned in the subject line of this thread. Rather, I'm addressing the fallacy of the statement of wanting "everything".

Regarding 'right' or 'sufficient' data at the moment a lot comes down to judgment and experience. The relevance and correct interpretation of some artifacts may depend on the circumstances in which they occur (e.g. combinations and versions of software present now or historically on the examined device). Different examiners and software developers will make different judgments regarding what's 'right' or 'sufficient' based on their breadth and depth of knowledge and on what they've found to be useful in the past. I'm sceptical that we'll ever be able to agree 100% on an objective truth, although undoubtedly orthodoxies will emerge over time.

Agreed regarding judgement and experience. I strive to document that sort of thing in my case notes; "…given the goals of this engagement, the ____________ information was not included in the timeline for the following reasons…"

Also, I think that there's one statement that you made in particular that really bears repeating

Different examiners and software developers will make different judgments regarding what's 'right' or 'sufficient' based on their breadth and depth of knowledge and on what they've found to be useful in the past.

In many cases, tool developers are not, and have never been, analysts. I don't think that a lot of analysts realize this…I'm sure that many analysts learn about tools by attending training (hey, we only retain 10% of what we hear, right??) or by reviewing social media. As many analysts don't have programming skills (not a requirement) and don't tend to ask questions of others, they don't really know what's available within various data structures, and therefore cannot determine the sufficiency of the data that they're retrieving.

I'm not saying that I know everything…the Good Lord knows that's not the case at all. Rather, as I've dug into various data structures (URL entries in index.dat, $UsnJrnl change log, etc.), I've see that information available *and* that it's not being incorporated into analysis.