> With respect to your advanced knowledge I think thats a complete cop out.
That's cool…so what do you think are the reasons?
> …I fail to see how this is the cause.
Okay. Do you have anything to add with regards to what the cause(s) may be?
Harlan
Harlan, sorry for the late reply read and fire at will!
“a complete copout†sounded a lot less rude in my head, so no offense meant. Your points about techies, newcomers and unwilling documenters are relevant, but I don't think they apply to the majority within the field and therefore can't entirely account for the situation.
I think the reasons are many and varied, but I do have an opinion. Without doubt the field of computer forensics is an embryonic one, and fundamentally one that's at an important divergence with regards to its identity. Firstly, I think it needs to be established just were exactly CF fits in relation to law enforcement. Is it a scientific discipline more akin to traditional crime scene forensics or is it solely a branch of the IT industry. I personally think these two 'locations', for want of a better expression, are mutually exclusive, it has to be one or the other. This in turn dramatically affects where the discipline is pushed, how hard and more crucially by whom. At the moment I see software developers and IT companies piloting at the helm whose motivations are anything but scientific. And from their seemingly dominant position they can ostensibly produce wide spread tools and proprietary techniques that become the de facto standard.
Hogfly asked earlier “Is there a solid method of testing live response tools that's been developed?â€. In addition to that are there any definitive industry wide standards for developing or testing static tools? Software manufacturers are turning out these essentially scientific tools without having to abide or adhere to any standards apart from software engineering best practice.
How did Encase for example, become accepted in court? How many people reviewed and tested it, with what data sets, for how long and what were their results and qualifications? I think the legal profession did, and does, play a major part also. The completely antiquated and in some cases none existed state of computer crime laws (DOS attacks are still not illegal under UK law) and the legal professions inexperience with digital crime put them at a severe disadvantage when setting legal precedents.
Why the absence of methodology and standards? I think these are driven by the scientific community not the IT industry. Also because guidelines that validate and essentially quality assure tools should be produced and hammered out before the tools themselves, in the case of some of the more popular static tools this isn't the case. There is a lack of incentive, and some would argue logic, to produce guidelines and standards for tools that have already been distributed and accepted over an extended period of time.
In relation to live tools, although there are people using them in anger, this area of the field is relatively new and quite possibly they are still on the drawing board. Another important factor is time. The overwhelming perception is that of a rising tide of cyber crime. There's too much pressure to provide results which in turn doesn't allow for a six month plus scientific validation study, and an unrealistic time to market.
Soapbox well and truly put away!
Do you not fancy carrying on the debate Harlan? cry
By the science, I mean science in terms of the definition used for computer forensics.
The science –through proven testing methodologies should be able to provide a set of methodologies that will give us expected results.
For instance. NTFS is a known evil. The science is there, it's what allows us to identify LCN's, VCN's and decipher the MFT and ultimately the file system structure to recover data. There are proven, tested methodologies.
Live response seems to be the wild west. Everyone says "use static binaries on a cd, use nc to get the data off" etc.. However, what's truly been tested? As we are beginning to see in the Helix discussions modifications are infact made to the system by live response tools, and in a seemingly random pattern. So in my estimation, Live response is still an art, not a science. It's relatively uncharted.
We (my group) are beginning to test and are discussing trying to get something together for DFRWS –but we'll see what happens and if we can agree on a topic.
are there any definitive industry wide standards for developing or testing static tools?
Yes there are, and software engineering moved from the best practices and encompasses a lot of rigid methodlogy and tools nowadays.
whether the Vendors producing the Forensic tools strictly adhere to these practices and techniques is a different matter, but this what qualified developers meant to do days in and days out.
In fact peole who follow strict Agile practices (or often called XP programming) wouldnt write a piece of code before writing the tests for it. this produces solid code that is fully tested, measurable and open to scrutiny (using the test harness and not necessarily the source code).
The golden question is what would software Vendors do when they add a new feature to their prodcuts, how they can guarantee it doesnt affect or alter exciting functionality, …etc. all these questions related to writing quality software are answered by practices for people who implement them.
Unfortunatly security breaches and vulnerabilties coming from top notch firm (you know who they are) tells us differently.
Sometimes big firms need to be challenged of what they say or produce and luckily digital forensic practitioners are able to do that and they should use it to their advantage.
youcefb9, by "definitive industry wide standards for developing or testing static tools", I didn't mean software applications in general I meant forensic software apps or tools specifically.
Do the major forensic software developers have a standard framework they follow for fully developing and testing their tools, not just testing that the source code does what's expected, but that they operate in a forensically sound way, don't taint the evidence etc. Or do they each just do their own thing and rely on their position in the market to sell their software. I'm not categorically saying they don't, but if they do I'd be interested to know who produced and validated the process.
> Do you not fancy carrying on the debate Harlan?
Sorry, I wasn't aware that there was a time limit.
> I think the reasons are many and varied…
Well, I'm not sure that things are really that varied. Also, I don't see how the field being in the embryonic stage really contributes to the lack of documentation.
> I see software developers and IT companies piloting at the helm whose motivations are anything but scientific.
I'm not sure which companies you're talking about, but for the most part, this is not something that's so much being driven by the companies, but that the practitioners are allowing it. For myself, I know that I and others are working closely with Chris Brown to improve ProDiscover. Also, Brian Carrier has TSK. Both of these efforts are more scientific, as are projects such as the programs release by Red Cliff.
Re software and standards. What standards? I haven't really seen any listed. But that doesn't obviate the need for testing.
> Why the absence of methodology and standards? I think these are driven by the scientific community not the IT industry.
So are you saying that the scientific community should produce these standards?
> There is a lack of incentive, and some would argue logic, to produce guidelines and standards for tools
Sounds like an excuse to me…
Harlan
>Sorry, I wasn't aware that there was a time limit.
There isn't, that was supposed to be lighthearted. Never mind.
>Idon't see how the field being in the embryonic stage really contributes to the lack of documentation.
So a new discipline evolves and the documentation suddenly appears without any time in development?
>Re software and standards. What standards? I haven't really seen any listed. But that doesn't obviate the need for testing.
I didn't say there were any standards, rather the opposite in fact, that was the point I was trying to make.
>So are you saying that the scientific community should produce these standards?
Personally yes, because I think computer forensics is fundamentally a scientific discipline and science has a history of research, precision and more importantly objectivity. Commercial companies and the IT industry have to look after the profit margin and that makes for a potential conflict of interest.
>Sounds like an excuse to me…
I wasn't using it as an excuse I was merely observing a possible trend.
fatrabbit,
First, of all, I'd like to avoid having this sound like it's coming out combative or adversarial. I'm trying to keep the discussion light…I realize looking over my last response to you that things really didn't come out that way, and I apologize for that.
> So a new discipline evolves and the documentation suddenly appears without any time in development?
Not at all, and I see your point.
When things in this field were centered around 25-40MB drives and MS-DOS (even including Win3.x), things seem to have been documented quite a bit. It's not hard to go back and find some of this stuff in the historical archives of the Internet. However, when something new presented itself, it wasn't documented in any way, really. It wasn't like a medical issue that was found interesting, and written up in the New England Journal of Medicine for other doctors to learn from.
I really don't feel that this discipline is all that "embryonic". I do, however, feel that while some of the things we face will change rapidly, the core tenets of the discipline remain the same. As such, we have a foundation…if we choose to use it.
To those outside the discipline, this still appears to be a "sexy service", much like pen testing became in the mid-90s. Companies built out penetration testing (later referred to as "ethical hacking") teams and marketed them as these sexy services…remember Foundstone and how they were all dressed in black? What this has to do with the discussion is that two things happen
1. These "sexy services" need a competitive business advantage, so they keep their findings close-hold. If they run into something new, like a device, or come up with some new technique, they have to keep it quiet and not share. Also, these services are generally small, staffed with a few very skilled people, and with the amount of work they do, they don't have time to write things up and present them. For a good example of this, ask Keith Jones how long ago he started working on the RDF book.
2. New people coming into the field want to dd images, run keyword searches, etc…but don't have the discipline to further their research, and then document it. There are a lot of really smart people who do not write anything down, and are very uncomfortable speaking in front of groups.
> I didn't say there were any standards, rather the opposite in fact, that was the point I was trying to make.
You've kind of made my point for me. What are you waiting for?
> Personally yes…
I see your point. However, I also see that there is lots of potential. Why sit back and wait for someone else to develop a standard? Develop it yourself and put it out there…get it peer reviewed. Get people using it.
Harlan
Harlan, I appreciate your reply. Everyone's entitled to their own opinion so it's good to reaffirm the forums ethos, that each post contributes to an objective debate rather than a personal argument.
>These "sexy services" need a competitive business advantage, so they keep their findings close-hold
Agreed. The competitive nature of business dictates the need to fill a market niche and keep hold, you cant blame any company for that, excluding a monopoly of course. I do think proprietary property can be an obstacle to open standards, not necessarily traditional open standards like HTML for example, but just general codes/guidelines of practice.
>New people coming into the field want to dd images, run keyword searches, etc…but don't have the discipline to further their research, and then document it
This is a point that I've already acknowledged. However, I don't feel that these people represent the majority and even if they do we cant let them hold us back. No matter what the field, for every one innovator there are ten people willing to sit back and just capitalize on the process or idea.
>What are you waiting for?
You're absolutely right. As practitioners within this field I believe we each have a responsibility to move it forwards. I have got areas in mind where I would like to conduct more research but I have to acknowledge my lack of expertise in certain areas and learn to walk before I can hurdle.