Thank you to everyone that is taking part in this discussion. I personally don't have the answers to this complex discussion so I value everyone's input so that we can all better understand the pro's and con's of possible future standards.
From the 17 votes so far, it is clear that 17025 has its issues and few are a fan of this current standard.
For those that have voted that "Standards are NOT required", do you mind posting comments as to why you believe this? By understanding everyone's argument, hopefully, we can move our community in the correct direction.
Isn't lab certification for computer work overkill and unnecessary?
IMHO yes and no.
Yes, it makes no sense whatsoever because it is inherently a "moving target", and - provided that actual ISO17025 is applied (which I doubt) the consequence is that either the lab will "remain behind" or a lot of resources (which doesn't seem to be that much abundant) will be (should be) diverted to verification and compliance of tools and methods.
A "better" or "more suitable" standard (or however guideline or procedure) is - as I see it - however needed and should cover the whole process of discovery and reporting not only the mere activities in the laboratory.
Jaclaz Correct me if I'm wrong, but you feel that a new standard covering everything from the initial collection to the final testimony in court would be most appropriate. This is assuming the new standard is not too rigid allowing examiners to do their work in our ever-changing environment without being required to verify and validate tools individually on an ongoing basis.
In your experience, have you come across any current standards that could be adapted to fit within Digital Forensics?
My personal opinion is that a standard would work better if it was less rigid in regards to validations of tools and instead placed emphasis on the validation of the data that is extracted and included within the final Digital Forensic Report as we are taught to do. Each tool can and will extract and display data differently and its the validation steps we as examiners take afterward that is essential to confirm that the extracted data is correctly interpreted and presented within our reports.
I believe the "lab" should not be any part of regulation.
The path to make DFIR more like other forensic fields should be directed at the person, not the lab, as electronic evidence is different than any other type of evidence.
Brett Do you see any sort of standard for labs?
What's to stop a lab from following improper protocols by having inexperienced examiners taking on work beyond their skill level?
If an examiner is highly regarded and then does contract work in a lab that is later found to cut corners and not follow basic common sense procedures, will that examiner's reputation be tarnished unfairly?
How do you see individuals meeting standards? Would this not be similar to certificates?
I believe that perhaps a combination of standards for both the lab and examiner is needed. My fear is that if we eliminate a standard for the lab, then they have less of an incentive to ensure the work is done appropriately and instead worry more about the financial costs cutting corners where they can to gain an advantage over their competition.
Hi,
Before I came into digital forensics (in 2004) I worked in traditional scientific test laboratories, most notably a pharmaceutical lab (AH Robins) and a microbiology lab (Malthus Instruments).
I got to know these environments and working practices pretty well and when I read 17025, it makes complete sense to me for those environments.
One of the things I believe even some UKAS assessors don't fully appreciate is the level of translation already required to go from what 17025 was intended for, into a wet forensics environment.
To then port it across into digital forensics, requires an even greater level of translation to the point where some points have become meaningless when you consider what the original authors intended to achieve.
There's so much more I could and want to say but it's all pretty much been said many times over. I just wanted to make the point that we shouldn't forget this standard was not written for forensics at all and from my point of view, having worked in traditional scientific test laboratories and digital forensics, I can see how wide the gulf is between the two disciplines.
Steve
“Much of the digital forensic community desires to have their evidence seen in court as forensically sound and bulletproof, yet do not want to go through the rigors that other traditional forensic sciences have done to prevent evidence spoliation and other mishandling and misinterpretations.”
~ Josh Moulin
Deputy Chief Information Officer
US Federal Government, NationalISO 17025 is now the mandatory standard in the United Kingdom for all Digital Forensics Laboratories.
Will the Digital Forensics community in the United States, Canada, Australia and elsewhere adopt ISO 17025, another standard, or wait for one to be imposed on them by a government agency?
Merriora just an observation and at the same time throwing a spanner in the works. How would you envisage cross-border evidence being accepted. For instance, you send evidence to the UK acquired during a joint operation with the US and evidence is for submission to the UK Criminal Court. Should the court assess your evidence against your standards or ISO17025?
For the computer "labs" having to comply with medical lab standards, I would expect compliance failures to be a regular occurrence, which will impact forensic admissibility in the courtroom for the simple fact of failure to follow an impossible-to-follow policy regardless of how perfect of a forensic exam was conducted.
Medical labs typically answer very narrow questions ïs this human blood? does it match blood samples X, Y, Z? If it does, what is the probability for a random match? They aren't asked 'did X murder Y?' or 'How did the DNA of X end up under Y's nails? (And if they are, they should have the sense to refuse the job…)
Computer forensic labs are a odd mixture of investigation on one hand, and question-answering on the other. Much more investigation, much less scientific fact-finding.
In the general area of investigations, I don't think standardization is of any use. But the narrow questions, such as 'was this file modified on <date> <time>? by whom? With what result?' could and should be subject to standardization.
Just as that lab being asked 'does this blood speciment belong to anyone known?' should have a considered methodology for answering that question, with known source of error (including verifying that the specimen a) is blood, and b) is human blood before the question of identity using a DNA database is addressed), I believe that similar specific questions that a computer forensic lab is asked to answer should have a similar methodology, and similar appreciation of errors affecting a result.
That is, ISO 17025 has a place. But it seems it is being applied as a wet blanket in the hope that it may cover and cure everything, instead of being applied as hot poultice for a specifc purpose and limited area of application. (Please ignore my medieval notions of medical treatment …)
Merriora just an observation and at the same time throwing a spanner in the works. How would you envisage cross-border evidence being accepted. For instance, you send evidence to the UK acquired during a joint operation with the US and evidence is for submission to the UK Criminal Court. Should the court assess your evidence against your standards or ISO17025?
This is an excellent question and likely a valid argument for some sort of Lab accreditation compared to only having accredited examiners. But hopefully, if the DF community in the rest of the world moves towards a different standard, solutions would exist to allow for the sharing of information. Just because the UK is ISO 17025 should not force the rest of the world to follow along if the standard truly doesn't fit Digital Forensics.
Assuming a new standard is developed, I would hope that it would be sufficient to compete against 17025 in regards to reliability from the lab.
Perhaps if a new standard is developed, then the UK would look to follow (far future) if that made sense and issues still existed with the implementation of 17025.
Computer forensic labs are a odd mixture of investigation on one hand, and question-answering on the other. Much more investigation, much less scientific fact-finding.
In the general area of investigations, I don't think standardization is of any use. But the narrow questions, such as 'was this file modified on <date> <time>? by whom? With what result?' could and should be subject to standardization.
I think this is a very important distinction in the type of work we complete within our field. Often we are not giving expert opinions within our report on if XYZ occurred, but rather providing information on what we observed on the device which can then be used to compare to other information know about the incident. (Investigative)
Example
- Here is a list of all calls and SMS messages obtained from the device compared to the Call Detail Records (CDRs)
VS.
In cases where an expert opinion is required..
Examples
- Was this picture taken with this phone on <date>?
- Was Joe using this device on <date/time>?
In the latter examples, further standards should be developed both concentrating on the experience of the investigator and the tools/methods* used to come to that determination.
In this situation, further testing should be done and expected to be done by the courts to be able to provide that expert opinion for this particular question if its essential the case.
My apprehension in the accreditation debate of digital forensics "labs" is that most of the standards proposed do not apply to the DFIR field and therefore will negatively disrupt it.
A digital forensics "lab" is many times just a laptop connected to an external hard drive that contains a forensic image of electronic data that can be used for examination in virtually any location on the planet (or off the planet).
The forensic work is interpreting the data. Preserved data does not spoil or rot and is not affected by an analysis. For scientific analysis as intended by various ISOs, there is no element on the table of elements that can be compared to electronic data as the testing of any element will result in a change, or be altered or modified, or even be destroyed by a lab analysis. All elements, even without an analysis are affected by environmental conditions including the passage of time, some more so than others.
Electronic data is not an element. It can be preserved, perfectly duplicated, and tested (interpreted) forever without alteration. The environment does not affect data. The testing does not affect the data. The passage of time does not affect the data. Storage media may fail, but the data can be preserved onto new media forever. There is practically no difference between reading a book and examining a forensic image. Once preserved, the information/data is unchanging when reading/interpreting. This cannot be said of any element on the table of elements.
The focus should be on training and education standards for the examiner and processes for collection of electronic evidence, whether derived from modified ISO standards and/or commonly used methods used by the community.
Today, technology is a moving target. Tomorrow, it may be out of reach if we restrict our work by implying that the mere interpretation of data from a forensic image requires the same environmental standards as conducting an autopsy on a human body or on a single drop of blood.
The forensic work is interpreting the data. Preserved data does not spoil or rot and is not affected by an analysis.
The forensic work also includes such a significant element as data acquisition. It's easy to say "preserved data", it's not so easy to preserve the data during the acquisition. If data is preserved, then yes, data interpretation errors can be resolved by examining this data again, although such errors can remain invisible in a particular case (still, there are legal ways to reduce the risk of unnoticed data interpretation errors). When data is not preserved (during its acquisition or at a later time), a number of obvious issues may arise. Moreover, sometimes forensic examiners have to prove that data was actually preserved as expected (and there should be an easy way to do this).
Currently, forensic examiners are blindly attaching a magic box which makes the acquisition process forensically sound (this box is called a hardware write blocker) and courts are accepting this method. But this is so much wrong! We need better validation methods and standards for hardware write blockers and other tools. We need a disclosure standard for vendors of forensic software/hardware. The acquisition process is crucial, so critical issues with basic tools like write blockers should be publicly discussed, because the "examine the data again" approach won't always work if original data isn't intact.
I neglected the acquisition aspect since it is impossible to require all or even some acquisitions to occur in an ISO certified lab environment. Many acquisitions are conducted onsite by virtue of the systems or limited time allowed to acquire. If there is ever a requirement to have lab-only acquisitions, you can imagine the negative impact that will have on forensics.
Not being a professional in the field, I am allowed to say that - while of course data should not be changed at a whim - the current fixation on "total integrity" is mainly fluff that the industry of write blockers happily promotes and that risks to have forensic examiners - obsessed by this particular (largely) non-issue to focus on this aspect and leave unexplored or mis-explored other parts of the evidence.
Previous related discussion
https://www.forensicfocus.com/Forums/viewtopic/t=11739/postdays=0/postorder=asc/start=5/
Anyway there is not one reason in the world to have a hardware write blocker (let alone trusting it blindly) the fact that noone has put together a basic OS, open source and fully documented that runs (at a decent speed) on something inexpensive like a Pi or any given "standard" board (possibly with a processor that has NOT "speculative execution" wink ), and that is verified/certified by members the international forensics community should be proof enough that there is no actual consensus on this very basic aspect, there is simply no chance in any foreseeable future to have any senceful standard/procedure.
The good news about the forcing down the throat of the good UK forensicators the ISO 17025 norm could be the occasion to have them (and those from other countries, scared to death by the possibility that the same will happen to them before o later) to actually put their act together and propose (better) alternatives.
jaclaz
