Open Source tools value and law enforcement agencies

There is an interesting article on the state of digital forensics in Italy in the May 2007 issue of the ISSA Journal.

Many Italian investigators use Open Source forensics tools because they are reliable and free. And trustworthy entire communities of developers view and work on the publicly available source code, and this public scrutiny enables forensic acquisition tools to be free of the objection of reasonable doubt that the acquisition process has compromised the evidence due to unknown and perhaps untrustworthy source code, thus invalidating the investigation.

There is an interesting article on the state of digital forensics in Italy in the May 2007 issue of the ISSA Journal.

Many Italian investigators use Open Source forensics tools….

Interesting, indeed. In the US most law-enforcement organizations prefer commercial tools. The reasoning seems to be that COTS (commercial off-the-shelf) tools have a proven track record, are generally accepted in court and offer certification programs allowing the practitioner an opportunity to demonstrate their competence with a particular tool.

Practitioners with a strong IT background may prefer open-source tools which, by definition, provide their source code for peer-review, meeting one of the tests of the US Daubert standard.

So here's the quandary in the US and, in all likelihood, many other countries The well-known commercial tools are accepted, but are expensive; the open-source tools are inexpensive, but are open to question in court.

A nice solution would be if Helix went through the NIST's Computer Forensics Tool Testing (CFTT) Project and was given a thumbs-up *and* if there was a Helix certification.

In the US most law-enforcement organizations prefer commercial tools. The reasoning seems to be that COTS (commercial off-the-shelf) tools have a proven track record, are generally accepted in court and offer certification programs allowing the practitioner an opportunity to demonstrate their competence with a particular tool.

I seriously doubt that's the reasoning.

Not to be too cynical, but the primary reason is the the learning curve for Windows based COTS tools is waaay more gentle than that of your average Open Source toolset. You can send a moderately bright cop to a couple of weeks worth of Encase training or an AccessDATA bootcamp and they're ready to find deleted files and nab the kiddie porn perps.

I'm NOT stating that this is a bad thing. On the contrary, such software provides a massive boost to productivity for the average PD (if they can afford the training and dongles). Once the cop gains more experience and learns the ropes of his chosen COTS tool, he/she gets more comfortable and we end up with alot of very competent examiners that testify using Encase or FTK (or Prodiscover, etc. - I don't mean to show favoritism).

the open-source tools are inexpensive, but are open to question in court.

Can you cite an example where an Open Source tool was called into question (and it's use denied) in court? Not a challenge…I'm genuinely interested if you've had any experience or knowledge of this.

A nice solution would be if Helix went through the NIST's Computer Forensics Tool Testing (CFTT) Project and was given a thumbs-up *and* if there was a Helix certification.

Helix is not a forensic tool, so to speak. It's a Linux Distribution that happens to provide a collection of individual tools. I don't recall Windows ever going through the CFTT process.

This is a classic paper on the subject of Open Source and legal arguments.

My $.02

Barry

Not to be too cynical, but the primary reason is the the learning curve for Windows based COTS tools is waaay more gentle than that of your average Open Source toolset. You can send a moderately bright cop to a couple of weeks worth of Encase training or an AccessDATA bootcamp and they're ready to find deleted files and nab the kiddie porn perps.

I'm NOT stating that this is a bad thing. On the contrary, such software provides a massive boost to productivity for the average PD (if they can afford the training and dongles). Once the cop gains more experience and learns the ropes of his chosen COTS tool, he/she gets more comfortable and we end up with alot of very competent examiners that testify using Encase or FTK (or Prodiscover, etc. - I don't mean to show favoritism).

That's a legitimate observation and it perfectly describes the computer fraud examiner with our county's sheriff department. I find it troubling, though, that an officer who can't set the time on their VCR can take two weeks of EnCase training and be "qualified" to testify as to what s/he found on a subject's PC. Perhaps they get a "free pass" because of the reasoning "Well, they *are* trained in evidence-collection and chain-of-custody procedures"? I have been part of discussions with attorneys, both defense and prosecution, where they've said they prefer that their examiners use the Big Name Commercial application suite so that they won't have to fight about the tool's legitimacy.

Can you cite an example where an Open Source tool was called into question (and it's use denied) in court? Not a challenge…I'm genuinely interested if you've had any experience or knowledge of this.

No, but I'm willing to do some digging in Lexis or FindLaw. You'll note in my earlier post I wrote that open-source "are open to question in court."[emphasis added] An attentive attorney will challenge any tool used by the opposing side that is not considered "generally accepted" by law-enforcement or the judiciary. That's all part of our "adversarial" legal system in the US.

Helix is not a forensic tool, so to speak. It's a Linux Distribution that happens to provide a collection of individual tools. I don't recall Windows ever going through the CFTT process.

…And aren't EnCase and FTK collections of individual forensic tools accessible from a cozy GUI? From the Helix home page "Helix is a customized distribution of the Knoppix Live Linux CD. Helix is more than just a bootable live CD. You can still boot into a customized Linux environment that includes customized linux kernels, excellent hardware detection and many applications dedicated to Incident Response and Forensics….

Helix also has a special Windows autorun side for Incident Response and Forensics." [emphasis added]

Your point about Windows not having '[gone] through the CFTT process' is irrelevant. Both Windows and *nix tools were evaluated. The OS was never under review. Hmmm… now THAT opens up a whole new question Is there an OS that is inherently a better platform for forensics tools?

This is a classic paper on the subject of Open Source and legal arguments.

Quoting from the conclusion of that document "…open source tools may more clearly and comprehensively meet the guideline requirements than would closed source tools. To further the acceptance of analysis tools in a legal setting, the following steps must be taken in the future…" The author is clear open-source tools are up to their task; they just aren't generally accepted and I've already commented on how attorneys will pounce on that.

The original posting to this thread was asking about open-source versus commercial software acceptance by law enforcement. My position is that it shouldn't be about the tool(s), but about the examiner.

Thanks for your comments, Barry. This discussion will certainly give our student friend some good material for class! 😉

-AWT

I find it troubling, though, that an officer who can't set the time on their VCR can take two weeks of EnCase training and be "qualified" to testify as to what s/he found on a subject's PC. Perhaps they get a "free pass" because of the reasoning "Well, they *are* trained in evidence-collection and chain-of-custody procedures"?

But that's the point entirely, isn't it…that they have a procedure or process that they follow?

Besides, one thing you need to keep in mind is that LEOs have to do LEO things to get promoted within their profession. That does not include programming VCRs.

…And aren't EnCase and FTK collections of individual forensic tools accessible from a cozy GUI?

No, I don't believe that they are. Even ProDiscover cannot be characterized in this manner. However, perhaps TSK can.

The original posting to this thread was asking about open-source versus commercial software acceptance by law enforcement. My position is that it shouldn't be about the tool(s), but about the examiner.

I agree wholeheartedly. But I am also aware that in the real world, particularly where it applies to LEOs, this isn't always the case. LEs deal with volumes of cases, and COTS tools provide a valuable force multiplier, in addition to training to certify the examiner in certain procedures.

H

I have been part of discussions with attorneys, both defense and prosecution, where they've said they prefer that their examiners use the Big Name Commercial application suite so that they won't have to fight about the tool's legitimacy.

Which, as you have already stated, is more a problem with the examiner than the tool. I still don't think that's a primary reason why LE chooses COTS tools… It may be a byproduct of the decision, but not a primary reason.

Can you cite an example where an Open Source tool was called into question (and it's use denied) in court? Not a challenge…I'm genuinely interested if you've had any experience or knowledge of this.

No, but I'm willing to do some digging in Lexis or FindLaw.

I'm interested to hear what you find. I still maintain that open source tools are no more subject to question in court than COTS tools. As long as I, as an examiner, take the time to test my tools in my environment, I am happy to testify in court (again, I think this is in agreement with your "examiner not the tool" approach).

..And aren't EnCase and FTK collections of individual forensic tools accessible from a cozy GUI?

Um…no, they're not. Can you point me to where I can download and test the "individual forensic tool" that does carving in EnCase or FTK? Hashing? Image catalog? And Helix is not simply a "cozy GUI". It's an OS…like Windows (or Debian or Slackware, etc.)

What exactly would you test to validate Helix as a "forensic tool"? It's carving capability? That's foremost or scalpel. It's imaging capability? That's dd or GRAB or AIR. It's "GUI" forensic tool apps are PyFLAG or Autopsy. Both are individual tools that are front ends to other tools (TSK).

Your point about Windows not having '[gone] through the CFTT process' is irrelevant. Both Windows and *nix tools were evaluated. The OS was never under review.

Which is exactly my point. Helix is an OS. Like windows. The fact that it has been optimized to provide a forensic environment and provides a *collection* of tools is what makes it so useful and different from many other Linux Distros. Another example is SMART Linux, a forensic distribution (OS) that also provides SMART for Linux (the forensic tool). So why would either OS go under review if Windows is not reviewed? That's precisely *why* the point is relevant.

My position is that it shouldn't be about the tool(s), but about the examiner.

And on that note, I can end with "we are in full agreement". 😉

For those who celebrate…Have a Happy and Safe Thanksgiving!

Barry

I find it troubling, though, that an officer who can't set the time on their VCR can take two weeks of EnCase training and be "qualified" to testify as to what s/he found on a subject's PC. Perhaps they get a "free pass" because of the reasoning "Well, they *are* trained in evidence-collection and chain-of-custody procedures"?

But that's the point entirely, isn't it…that they have a procedure or process that they follow?

Besides, one thing you need to keep in mind is that LEOs have to do LEO things to get promoted within their profession. That does not include programming VCRs.

That *does* bring up the problem that technically-illiterate LEOs performing digital forensics are basically "tool tykes" just waiting to get promote out of forensics. If I as an attorney am defending a client accused of downloading CP to his computer I'm going to push the officer to admit that he only knows how to "point-n-click" in EnCase. How does he know that critical system time-stamps haven't been altered? How does he know that the offending images weren't placed on the defendant's PC by malware? Or a walk-up user? What about hex editors?

If you answer "Well, the officer is experienced in…" I'm going to counter with "How many innocents were convicted by Officer Doofus' testimony as he was gaining his CF "experience" and is my client part of this officer's "continuing education?"

… I am also aware that in the real world, particularly where it applies to LEOs, this isn't always the case. LEs deal with volumes of cases, and COTS tools provide a valuable force multiplier, in addition to training to certify the examiner in certain procedures.

H

Yes, and EnCase and AccessData, et al. *are* commercial products for a reason their developers want to make money. This is not a bad thing, necessarily, but there is always the danger of the "market leader" fostering a mono-culture where only one or two products in their class are used to the exclusion of viable alternatives.

What happens when researchers come along and discover a vulnerability and release a press statement along the lines of "A critical flaw has been discovered in BIG NAME FORENSIC TOOL that is present in all versions since x.y and calls into question all case-work done using this product." Sound preposterous? It's happened with a certain company's "generally accepted" OSs.

Can you point me to where I can download and test the "individual forensic tool" that does carving in EnCase or FTK? Hashing? Image catalog?

I'd love to decompile EnCase, but I'd probably be in violation of the DMCA. If "someone" were to open up EnCase or AccessData's FTK (Hmmm… "FTK"… Forensic Tool Kit… does that not imply a "collection of tools"…?) I suspect they would find that these commercial products are comprised of subroutines that are based directly on open-source precursors, such as "dd."

Helix is an OS. Like windows. The fact that it has been optimized to provide a forensic environment and provides a *collection* of tools is what makes it so useful and different from many other Linux Distros. Another example is SMART Linux, a forensic distribution (OS) that also provides SMART for Linux (the forensic tool).

Re-read the Helix home page. It's a "customized distribution" of Knoppix with "many applications dedicated to Incident Response and Forensics…." Also note, Andy Rosen does not call his SMART Linux project an "OS," but says that it is based on Slackware and optimized for Digital Forensics and Incident Response. At this point I could suggest a new thread topic on just what exactly is an "operating system?" Just because Micro$oft includes Solitaire in its "distro," does that make the card game part of the OS? Nevermind, we probably shouldn't go there.

EnCase is not an OS, but requires an underlying Windows installation. Maybe one could make the argument that EnCase is only as reliable as the OS upon which it rides…? Takers?

What exactly would you test to validate Helix as a "forensic tool"? It's carving capability? That's foremost or scalpel. It's imaging capability? That's dd or GRAB or AIR. It's "GUI" forensic tool apps are PyFLAG or Autopsy. Both are individual tools that are front ends to other tools (TSK).

So… are you willing to stipulate that if the key forensics tool components of Helix have been subject to peer-review and are vetted by the NIST or some other competent testing body, then the Helix suite should be generally accepted by the forensics and legal communities? 😉

By the way, I *do* enjoy this interchange (Harlan Love your book; still reading it - Barry Good responses!). This topic is both relevant and timely.

-AWT

Well!

I knew I would have roused an interesting debate! wink

Your comments made me realize how controversial that issue is.

Thanks for your comments, Barry. This discussion will certainly give our student friend some good material for class!

You're right! Infact this thread gave me a good starting point to develop my thesis. D

bgrundy wrote
Can you cite an example where an Open Source tool was called into question (and it's use denied) in court? Not a challenge…I'm genuinely interested if you've had any experience or knowledge of this.

No, but I'm willing to do some digging in Lexis or FindLaw.

If you were be able to supply any useful information, It would be a great thing for me.

Thank you everybody!

Greetings!

A very interesting exchange of thoughts and ideas. Thanks for some good informative reading.
KP