Good morning,

I recently wrote a blog posting about the value of push button forensics and the expansion of the computer forensics domain to include people with less experience. The posting can be found here:

integriography.wordpre...forensics/

So as to not steal traffic from FF, the full text is below as well. If you're so inclined, a click on the blog site link would be appreciated if you find the article at all interesting.

-David


Access Data recently entered into a partnership with e-fense. In the announcement, they wrote: “Digital investigations are no longer the exclusive domain of highly trained experts.” I don’t think Access Data is wrong, and I think the forensics community needs to accept that “push button forensics” is here to stay. Further, I think it can be an important part of our future.

(Two notes: 1) For the purpose of this article, forensics and e-discovery are essentially interchangeable. 2) I’m using “technician” to describe someone with basic to moderate technical skills but lacking in deep forensics and/or e-discovery experience.)

“Push button forensics” (PBF) is often derided by computer forensics professionals. We rail against it, occasionally joke about it, and have even made “Find Evidence” buttons to stick on our keyboards. Certain facts suggest that we should embrace it, though perhaps while wearing PPE.

1. Tool vendors have a vested interest in selling forensics and e-discovery tools that can be used by people without forensics experience and certifications. If you can make a tool that any technician, lawyer, or IT person can use in a legally defensible manner, you will expand your potential market considerably. We are no match for the combined weight of the marketing departments of the vendors whose tools we are using.
2. Corporations, LE agencies, law firms, and other consumers of computer forensics services have a financial interest in acquiring tools that will perform complex forensics and e-discovery tasks and that can be used by technicians rather than by experts. The cost per hour of computer forensics services in the San Francisco Bay Area is around $250. There is a lot of appeal in buying a tool and using a $50 per hour in house technician if you can get the same results.
3. The volume and complexity of digital evidence is growing, and growing faster than we can cope with it. LE agencies at all levels have significant computer forensics backlogs, made worse by current budget issues. Corporate legal departments and law firms are under pressure to sift through enormous volumes of data more quickly, and more efficiently, than ever before. The number of people available who can manually sort through the complex evidence isn’t keeping pace, and the explosion in new computer forensics certification and degree programs will not solve the problem any time soon.

In addition to the facts that suggest we need to accept PBF into our environments, I’d like to suggest that, properly integrated, it can be very good for us personally and for our businesses. Here’s one example:

I’ve quite enjoyed following the development of Harlan Carvey’s timeline analysis tools and procedures. I’ve learned a lot from working through his examples, and I’d strongly encourage others to do so. But, the process is currently far too time consuming to use on any project with any significant pressure. We will need more automation, more “push buttoness”, to effectively employ it. And once it is “push button” AND validated, why can’t I farm that part of the process out to a technician? In doing so, I will:

* Acquire useful information in a more timely manner, speeding the investigation and saving the client money.
* Distribute the workload among more junior staff, enhancing their ability to contribute and decreasing the bottleneck on senior resources.
* Free up senior staff for tasks that truly require more experience and knowledge.

Put another way, from a consulting perspective, I can save my clients money, free up experienced people to work on more difficult problems, and safely incorporate people with less experience. The clients will be happy – better results for less money; the senior people will be happy – real challenges, less grunt work; and the junior people will be happy – more opportunity to gain experience.

Our forums are full of discussions about how to use an enormous number of tools, many of which automate and greatly simplify our processes.

* Anyone proficient with EnCase, FTK, X-Ways, or Sleuthkit could replicate Drive Prophet’s results but it would take hours longer, and the chance of missing something is greater.
* Similar point for web browser analysis – if there wasn’t a need to automate this, why do we have Mandiant Web Historian, Gaijin Historian, Cache Back, Pasco, Fox Analysis, NirSoft Mozilla History View, and Passcape History Viewer to name a few?
* With Mount Image Pro, I can provide a forensically sound image to a reviewer to examine with tools they’re comfortable with – Outlook, Explorer, dtSearch – without any risk that they’ll modify the evidence. This can save me a lot of back and forth to produce directory listings, copies of the My Documents folder, and .pst files.

If we look back through the archives of out discussion forums we’ll see that we’ve been automating and simplifying computer forensics processes since the dawn of the profession. In doing so we’ve made the profession more accessible to new practitioners, more valuable to our clients, and more interesting to ourselves. This mimics developments in the rest of the computer industry, and in every aspect of our lives. We’ve got push button cooking, push button flying (auto-land capability), push button navigation, push button photography, …. Push button forensics is here to stay. Accepting the fact and incorporating it into our processes and companies seems wise.

Mind you, I say this with several important assumptions in mind:

* The tools work as advertised, their behavior and results are well understood, and the process and results can be verified.
* The tools are verified internally.
* The use of the tools is supervised by experienced staff.

“Push Button Forensics” has a place in our business toolkits. Digital investigations are no longer the exclusive domain of highly trained experts. Validated PBF tools in the hands of properly trained and supervised technicians can be a very powerful combination for law enforcement agencies, law firms, corporations, and consulting firms.

I’d like to leave you with perhaps the most important point, one that is frequently overlooked or assumed – Finding the evidence is only a small part of the process. Tools can find keywords, put together a timeline, or show you the CP images. They cannot put any of that information in context. Interpreting the information, whether found manually or by PBF tools, still falls squarely in the pervue of a trained and experienced computer forensics investigator.

kovar wrote:

Interpreting the information, whether found manually or by PBF tools, still falls squarely in the pervue of a trained and experienced computer forensics investigator.

Knowing the strengths and limitation of your own science.

I like the sagacity of your conclusion.

We have to avoid mistaking conduct that is relevant to the enthusiastic amateur or hobbyist (which isn't accepted at court) and that of professional forensic examiner who is supposed to know the science, irrespective whether s/he can build it or not. No one wants to be found guilty because of evidence obtained by a hobbyist who only knows how to push buttons.

kovar wrote:

Interpreting the information, whether found manually or by PBF tools, still falls squarely in the [purview] of a trained and experienced computer forensics investigator.

Good commentary, David... you got *my* click!

I would like to expand briefly on your thoughts.

1. How many folks know how to build a radio, television, telephone, MP3 player, DVD player, et al? Yet, in spite of their ignorance of the underlying technology, billions of people use these tools/toys every day! The notion that 'Well, you can't POSSIBLY be a REAL forensic examiner if you don't know how to write a command-line hex-editor/file-carver/Registry-parser and compile it to run under Glubnix, version OICUR12' is absurd.

How many of you, dear readers, know how to build an automobile (a potentially life-endangering piece of equipment, btw), yet still have a LICENSE to drive?

PBF is no different. We're after RESULTS. Many times these results will be presented to clients, attorneys/barristers, judges, juries... non of whom have our exalted secret wisdom. If we wander into the 'technutia,' their eyes glaze over. The best forensic examiners are those that can KISS: Keep It Simple {Silly}.

2. Oh, and then there's the profit motive. Especially in the USA, developers of forensic software are trying to cash in on all the great stimulus-money that's being poured on Law Enforcement agencies. These are the folks who can afford a $4000 "forensics suite." Often these folks have NO technical expertise. For example: I know one LEO who got the job of "forensic examiner" because he knew how to "do things" on a PC. His department thought this qualified him to be sent for 2 weeks of EnCase training and certification.

Face the fact: PBF is reality. Let's embrace it, rather than fight it.

AWTLPI wrote:

1. How many folks know how to build a radio, television, telephone, MP3 player, DVD player, et al? Yet, in spite of their ignorance of the underlying technology, billions of people use these tools/toys every day! The notion that 'Well, you can't POSSIBLY be a REAL forensic examiner if you don't know how to write a command-line hex-editor/file-carver/Registry-parser and compile it to run under Glubnix, version OICUR12' is absurd.

I'm not sure that the analogy is correct or appropriate. Listening to a radio or watching a TV does not require you to assess, on the basis of your experience, whether the signal is accurate or complete. It is sufficient for you to enjoy the output whether the output is a valid representation of the input or not.

Moreover, when was the last time that you watched TV where you had to document every step of the process as you do when you handle evidence. You can push a button, but that doesn't prevent you from mishandling evidence.

AWTLPI wrote:

How many of you, dear readers, know how to build an automobile (a potentially life-endangering piece of equipment, btw), yet still have a LICENSE to drive?

And how many aeronautical engineers are also pilots? I'd venture to say not all. Qualification to operate the machinery still requires knowledge of principles but other types of training as well.

AWTLPI wrote:

PBF is no different. We're after RESULTS.

How many data carving tools do you use in order to be certain you have found every file that can be found? More than one? All that are available?

Ask anyone who has ever had to recover from, or determine whether PCI data exists on, a particular storage device whether any push button solution is sufficient. What "push button" solution allows you to determine whether there was deliberate spoliation of electronic evidence.

Results are the outcome of your work, not the input, which may or may not be evidence.

I am not debating whether there will be low cost service providers using triaging solutions such as David mentioned. I am questioning whether such practitioners will ever be qualified as experts for the purposes of litigating anything more than equitable distribution of property (if that).

AWTLPI wrote:

Many times these results will be presented to clients, attorneys/barristers, judges, juries... non of whom have our exalted secret wisdom. If we wander into the 'technutia,' their eyes glaze over. The best forensic examiners are those that can KISS: Keep It Simple {Silly}.

Sure. But distilling complicated technical processes into something that a judge or jury can understand without oversimplifying to the point of being innacurate is a skill, not something that you find on a USB stick.

I'm always looking for ways to do things less expensively (e.g., F-Response) and I'm not opposed to using tools such as Drive Prophet to simplify the gathering of background information, though I would want to verify anything of significance with another tool.

But I've also been in courtrooms with people whose only expertise was that they knew how to remove the shrink wrap, open the box, load the CD and run the program and watched them being torn to shreds in cross-examination.

While I think that there will be a role for lower-cost technicians to assist in digital forensics just as there is, now, for forensic pathology, I don't see these being serious alternatives to qualified digital forensic practitioners. It may even be the case that, for awhile, these technicians will harvest some of the low hanging fruit from more experienced professionals, at least until someone loses a high profile case due to the fault of the examiner.

But at some point someone will still have to render an expert opinion and "because COFEE found it" is rarely going to be sufficient.

seanmcl wrote:

AWTLPI wrote: 1. How many folks know how to build a radio, television, telephone, MP3 player, DVD player, et al? Yet, in spite of their ignorance of the underlying technology, billions of people use these tools/toys every day! The notion that 'Well, you can't POSSIBLY be a REAL forensic examiner if you don't know how to write a command-line hex-editor/file-carver/Registry-parser and compile it to run under Glubnix, version OICUR12' is absurd. I'm not sure that the analogy is correct or appropriate. Listening to a radio or watching a TV does not require you to assess, on the basis of your experience, whether the signal is accurate or complete. It is sufficient for you to enjoy the output whether the output is a valid representation of the input or not.

I have to agree - being able use a device does not make you an expert in examining the data present on it to understand how it has been used, e.g. being able to use a browser to surf the web does not make you an expert in understanding the likely provenance of internet history artifacts located in unallocated space, etc.

Regarding automated parsing, I would accept that an examiner doesn't have to be able to develop programs, but I would expect that they could (if required) perform the analysis of data present in defined locations according to defined structures, rather than just push a button with no fundamental understanding of the analysis being performed.


Phil.

pwakely wrote:

seanmcl wrote: AWTLPI wrote: 1. How many folks know how to build a radio, television, telephone, MP3 player, DVD player, et al? Yet, in spite of their ignorance of the underlying technology, billions of people use these tools/toys every day! The notion that 'Well, you can't POSSIBLY be a REAL forensic examiner if you don't know how to write a command-line hex-editor/file-carver/Registry-parser and compile it to run under Glubnix, version OICUR12' is absurd. I'm not sure that the analogy is correct or appropriate. Listening to a radio or watching a TV does not require you to assess, on the basis of your experience, whether the signal is accurate or complete. It is sufficient for you to enjoy the output whether the output is a valid representation of the input or not. I have to agree - being able use a device does not make you an expert in examining the data present on it to understand how it has been used, e.g. being able to use a browser to surf the web does not make you an expert in understanding the likely provenance of internet history artifacts located in unallocated space, etc. Regarding automated parsing, I would accept that an examiner doesn't have to be able to develop programs, but I would expect that they could (if required) perform the analysis of data present in defined locations according to defined structures, rather than just push a button with no fundamental understanding of the analysis being performed. Phil.

There is also an issue with what isn't there that should be? Or, what does this information from the Registry (for example) actually mean? Or, what artifacts in this file system put the evidence into context? This is where experience and training come into play. Just finding the evidence is usually not enough.

Always good to see my comments shredded by the Good Doc in Pittsburgh.

Let's put it another way. Forensics products are being made ever simpler for their primary audience: non-technical law enforcement officers/examiners. Their departments have the funds to shell out and these agencies want products "so simple a cop can use them." That's a direct quote from a now former officer who performs digital forensics.

When he gets stuck on a problem, he calls me and my usual answer is, "OK, let's get to a command prompt...." He recoils in horror, protesting that he "don't know nothin' 'bout no 'command prompt'." Great. But *he* is called to testify in Court as to his "methodology."

I see the same thing in the Information Technology classes I teach. Only 1 in 20 of my first-year students have ever seen a version of Windows older than Win 9x. Windows 3.1? Nope. MS-DOS? No way! I demonstrated EDLIN on the third week of classes and 4 students dropped out the next day. Coincidence? Maybe.

I agree that an understanding of the inner-workings of digital devices is valuable. Hey, I'm proud of my decades of experience in the digital realm. I am also a realist. I see more and more practitioners entering the field whose computer "skills" are limited to "point-n-click." We can debate all we care to on these boards about the Evils of Push-Button Forensics, but the Big Vendors are making products (and their "certification" programs) for a niche market that wants simplicity, not for those of us that aren't afraid to open a command prompt. Or use a hex editor.

In light of this topic, a well-timed post on The Register highlights the shortcomings that the ICSA finds in Info Security products, such as anti-malware, firewalls et al. Although not referring to forensics products, per se, their conclusion is relevant to those products we use:

Quote::

ICSA Labs advises end users to choose simplicity over complexity, and suggests a bias towards more established products over newer products that whose kinks are yet to be worked out. The advice runs contrary to conventional industry marketing, which would have users believe innovation is making products better-performing and more secure.

Despite their validity, I suspect those recommendations will be ignored.

Beetle wrote:

There is also an issue with what isn't there that should be? Or, what does this information from the Registry (for example) actually mean? Or, what artifacts in this file system put the evidence into context? This is where experience and training come into play. Just finding the evidence is usually not enough.

Indeed! Can the examiner build a time-line of events? (Oh, wait! Someone's building a push-button for that! ) Is there corroborating evidence? Or is one going to build a case against Jane Suspect based only on showing that User "JaneSuspect" was logged in when the nefarious computer deeds were performed? (Think: "Non-repudiation")

It all comes out in the end. Any person collecting and presenting data without a sound basis in forensics or electronic discovery will not fare well against an expert and clients will typically end up with what they paid for. Collecting data in-house by semi-trained persons is cheaper, yes. But in the end, it is way more expensive should data be thrown out, discredited, or otherwise be proven inaccurate by an expert.

Software companies will always try to sell to the biggest market they can, even promising that untrained persons can 'do forensics' with their product. I feel for the IT member that is crucified on the stand because he or she believed the printed brochures that anyone can 'do forensics' with a simple push of a button.

Maybe its just me, but I don't think anyone can start to feel comfortable in this field without having a few years of experience under their belt, spattered with lots of varied training.

AWT:

Let me put it this way. I've been using EnCase since V3 and other software before and since then and the only time that I have ever used the report writer in EnCase was when I was taking the EnCE practical and it was all but required.

Same with every other tool that I've used.

In fact, I was involved in a case where an investigator submitted, as evidence, the audit log from X-Ways Forensic which we, then, used to impeach his testimony.

Sure, there are programs to make it easier to address specific parts of an analysis. But the more than these try to automate the organization and presentation of data, the less reliable and useful the output will be.

What will happen will be that someone, most likely a prosecutor, will go into court armed with the output from one of these push button programs and be shredded to pieces precisely because they didn't look any further.

Law enforcement in many areas is already overstressed with respect to the handling of digital evidence. How much more stressed will they be for failing to discover or present exculpatory evidence as part of building their case. This has a "Law & Order" episode written all over it.

So my point really is that no matter how much more efficient these programs may be at finding things foryou, they can't think for you and they can't, ultimately, decide when the investigation is complete. And those who expect otherwise are going to find themselves serious embarrassed in court, I imagine, based upon my experiences with investigators who rely more on the tool than their own powers of deduction.

I had a case awhile back where CP was found on a public computer of an institution of higher education and so we were brought in to determine if any other machines had been used/affected on their network. We used one of the most advanced skin tone detection systems available which was promoted as a tool to be used by LE to scan for contraband images.

The school was a health sciences school.

Can you imagine what was the false positive rate?

Moreover, the suspect images were not even near the 90% confidence interval. Due to their lower quality and the fact that they had been digitally altered, the actual suspect images (of which there were many false positives), were around a confidence level of 39-51%. There were a significant number of false negatives, as well, for reasons that I won't go into because they affect the accuracy of the algorithm.

IMHO, these tools may help to steer an investigation. They may also help in issues such as probable cause. But they aren't going to settle many cases until they learn to think, which is a long way off.