Thoughts on Tools/Processes

keydet89 · 2013-05-09T22:17:39Z

I am responding to a post to another thread…What is the practical suggestion?Do not use any third part software and write your own tool.Use multiple softwares and delve deeper if there are differences between results.Test as deeply as you can all available software, then choose one and only that one, because you already know which "quirks" it may produce.Other (please specify).[/listo] If #1, then all software houses could close down, and each forensic examiner will have to write, validate and eventually "defend" in court his/her tool against the findings of the expert witness of the other party. who also wrote his/her own tool and has exactly the same issue. If #2, then all forensic investigators should test the data/SIM/whatever against *all* available tools, freeware or Commercial and "hope" that no inconsistencies are found. (as one of the tools may be "right" in one specific point but "wrong" on another) If #3, the risk of a peculiar artifact not having being tested or tested properly seems to me rather BIG. I would say that all three approaches above are either very problematic or very non-productive. What is option #4? ?I don't think that any of the options is necessarily "practical". #1 is simply untenable, as is #2. #3 is simply not practical, either. I also don't think that a solution of any kind can necessarily be lumped into a single, absolute option.A while ago, I was asked for my assistance…the request came in this manner "The tool you gave me doesn't work." It took a bit of time and exchange of emails to find out that this analyst had been given Windows XP Event Logs by another analyst, and ran a tool I had written against them, and not gotten any output. I was assured that the files were indeed Windows XP Event Logs. I asked for a sample, and within seconds of it arriving in my inbox, I opened it in a hex editor and found that what was called "secevent.evt" was not, at all, a Windows Event Log.I don't expect every analyst to write their own tools, nor to test every possible tool against each data set. However, there are solutionsFirst, understand the data structure you're looking for/at, as well as what the tool you're using actually does with respect to the data. If you ask me for my MBR parsing script, and then run it against a memory dump, please don't then contact me with, "your tool doesn't work." Does this mean that every analyst has to memorize all of the various possible data structures? No, not at all…it's hard enough as it is just to keep track of Windows Event Logs…now throw formats of SMS messages on the different platforms, and that would simply be too much. Instead, share information. Document what you find. Make use of sharing sites such as forensicswiki.org. Crowd-sourcing your question by simply posting it to a forum can prove beneficial to some, but in the end, how many of us are actually sharing what we learned back with the community?Engage with other analysts. Not just in the general sense, but directly, as well. I know that not everyone has the time nor the interest to do this sort of thing…but I've seen quite a few analysts spend way too much time on something, all for the sake of figuring out for themselves. If you spend 8 hrs on something that you should have been able to complete in 15 minutes, then it should be pretty clear why you don't have time for other things. I once talked to an analyst who spent over three months working on something, trying to figure it out themselves, before asking for help. After three months, they finally decided to ask a question, and in less than 20 minutes were provided an education and a tool to help them with their task. Overall, I'm concerned about the level of engagement within the community, in that a great deal of information is being lost. Let's say someone has a need or question…they go to ForensicsWiki.org, do a search, and don't find what they were looking for. If they simply stop there, so much is lost. Did they search for the right thing? Did they ask for assistance from someone else? Did they let anyone know that what they were looking for wasn't there, so that someone can then research it and provide the information, i.e., fill in the gap?I didn't read Yunus' paper, in short because I don't deal with SMS messages and mobile devices at the moment. However, I have seen comparisons of tools before, and one of the things that has concerned me is the evaluation itself. I've seen tools be evaluated for being "scalable to the enterprise" and fail, when they were never designed to be scaled to the enterprise. There needs to be some peer review of the information in general, whether it's the description of a data structure, or the evaluation of a particular tool to parse and display that data structure, and a wiki is a great way to go about that.HTH

Page 2 / 2 Prev

General (Technical, Procedural, Software, Hardware etc.)

Last Post by jhup 12 years ago

14 Posts

8 Users

0 Reactions

991 Views

RSS

jaclaz

(@jaclaz)

Illustrious Member

Joined: 18 years ago

Posts: 5133

10/05/2013 6:55 pm

As for "everyone jumped at Yunus' throat" maybe a wee bit stretching it, eh? wink

Sure ) , that was intentionally overstressed.

Still, you start by assuming that Yunus testing procedure was failed somehow.

I start assuming that running three different tools on the same set of data should provide same results (or compatible results).

I also assume that (difficult to say since it's in Turkish) the "whole" message makes sense while the same message without first line appears like a truncated message and as well that the single line by itself makes no sense.

I simply refuse to believe that one of the three tools "invented" out of thin air a line (in Turkish) that makes sense as first line of another message and by "sheer luck" merged it to the "right" message.

So, the tool that provides a "monolithic" message seems "right",
The one that misses a line "wrong".
The one that finds the line but as a separate message also "wrong".
The above is the most probable situation (even before and besides checking the Raw data).

But still, the point I was trying to make is not about which tool (if any) is "right" and which is (are) "wrong" (which is something that should be investigated and ascertained, but that is "specific"), only that the found discrepancy seems related to a "rather common" situation (a multi-line SMS) and that seemingly noone encountered it before (or didn't make it "public" or didn't notify about it the Authors of the various tools).

This latter aspect is IMHO what - as keydet89 understood - prompts - maybe - to re-analyze processes and procedures, while you might be personally testing each and every phone dump with two (or more) tools, and additionally check the RAW data I believe that this is not the "most used" or "standard" procedure. ?
The specific report is about a very small set of messages, less than 30, how many messages do you find on average on an examined phone?
Tens, hundreds or maybe thousands?
Wouldn't this make possible that *something* slips by and goes unnoticed?

jaclaz

ReplyQuote

jhup

(@jhup)

Noble Member

Joined: 16 years ago

Posts: 1442

10/05/2013 8:09 pm

This latter aspect is IMHO what - as keydet89 understood - prompts - maybe - to re-analyze processes and procedures, while you might be personally testing each and every phone dump with two (or more) tools, and additionally check the RAW data I believe that this is not the "most used" or "standard" procedure. Question
The specific report is about a very small set of messages, less than 30, how many messages do you find on average on an examined phone?
Tens, hundreds or maybe thousands?
Wouldn't this make possible that *something* slips by and goes unnoticed?

Agreed, and it dovetails well into why Mr. Carvey started this thread.

(edited to properly reference this thread.)

ReplyQuote

jaclaz

(@jaclaz)

Illustrious Member

Joined: 18 years ago

Posts: 5133