Join Us!

ISO 17025 for Digit...
 
Notifications
Clear all

ISO 17025 for Digital Forensics – Yay or Nay?  

Page 1 / 9
  RSS
Merriora
(@merriora)
Junior Member

“Much of the digital forensic community desires to have their evidence seen in court as forensically sound and bulletproof, yet do not want to go through the rigors that other traditional forensic sciences have done to prevent evidence spoliation and other mishandling and misinterpretations.”
~ Josh Moulin
Deputy Chief Information Officer
US Federal Government, National

ISO 17025 is now the mandatory standard in the United Kingdom for all Digital Forensics Laboratories.

Will the Digital Forensics community in the United States, Canada, Australia and elsewhere adopt ISO 17025, another standard, or wait for one to be imposed on them by a government agency?
To find the best solution for those working outside the UK, the following article calls for an in-depth analysis of IS0 17025, and everything it represents.

Input from those working in ISO 17025 labs in the UK and other countries will help the rest of us better understand the reasons for and against this accreditation.

Please read the following article posted here on Forensic Focus and join the discussion by posting your comments here.
https://articles.forensicfocus.com/2018/01/24/iso-17025-for-digital-forensics-yay-or-nay/

Your input will help start the conversation on this important topic.

Quote
Posted : 24/01/2018 12:55 pm
minime2k9
(@minime2k9)
Active Member

So there is a great deal of problems that are plaguing the implementation of this standard
Consistency across audits - major problem
Massively used to areas which have standard methods - as in British Standard/International methods not common use of IEF
Areas where terms are made to fit digital standard - uncertainty of measurement
Competency - based around the idea that each method is very different and separate and that someone needs to demonstrate how to do it. I would argue that running IEF or Regripper or other tools doesn't require a specific competency, but investigations require competent staff. A work instruction detailing the former covers ISO but never addresses someones actual competency.
Specific work instructions - doesn't allow flexibility for taking into account unknowns. How do you write a work instruction for something you don't know exists.
Pace - ISO 17025 is designed for areas which rarely change. Digital Investigators are already behind and technology changes pace rapidly.
Calibration - requirement to 'calibrate' computers used in investigations, a requirement of ISO but stupidly applied.

There are more areas than I can show here right now, but the audits have shown me how ridiculous the whole process is. I know more than one or two labs have been asked how they protect hard drives from solar flare activity!

ReplyQuote
Posted : 24/01/2018 4:48 pm
jaclaz
(@jaclaz)
Community Legend

There are more areas than I can show here right now, but the audits have shown me how ridiculous the whole process is. I know more than one or two labs have been asked how they protect hard drives from solar flare activity!

Not only, the > 1 meter thick reinforced concrete walls of underground laboratories would probably need to be certified as "solar flare proof" or at the very least "solar flare resistant", according to a standard grading system developed at the time for "glasshouses" (which is obviously "closely related"). wink

jaclaz

ReplyQuote
Posted : 24/01/2018 4:59 pm
bshavers
(@bshavers)
Active Member

Isn't lab certification for computer work overkill and unnecessary?

Other lab types that work with chemical, biological, or nuclear substances surely need accreditation with substantial regulation for public safety measures. But certifying a computer lab?

I believe the "lab" should not be any part of regulation.

The path to make DFIR more like other forensic fields should be directed at the person, not the lab, as electronic evidence is different than any other type of evidence. Electronic evidence can be preserved in a static state (as if preserved in amber), reproduced, duplicated, transmitted, and copied without alteration. It can be examined countless times by countless persons using countless methods to obtain forensically sound results. The same cannot be said of other forensic fields. Once a physical substance has been tested (blood, drugs, a human body, etc..), it cannot be tested again as if it were never tested in the first place, nor can the substance be duplicated or preserved as electronic evidence can. Once preserved, electronic evidence does not spoil or rot.

For the computer "labs" having to comply with medical lab standards, I would expect compliance failures to be a regular occurrence, which will impact forensic admissibility in the courtroom for the simple fact of failure to follow an impossible-to-follow policy regardless of how perfect of a forensic exam was conducted.

ReplyQuote
Posted : 24/01/2018 5:57 pm
mcman
(@mcman)
Active Member

Isn't lab certification for computer work overkill and unnecessary?

Other lab types that work with chemical, biological, or nuclear substances surely need accreditation with substantial regulation for public safety measures. But certifying a computer lab?

I believe the "lab" should not be any part of regulation.

The path to make DFIR more like other forensic fields should be directed at the person, not the lab, as electronic evidence is different than any other type of evidence. Electronic evidence can be preserved in a static state (as if preserved in amber), reproduced, duplicated, transmitted, and copied without alteration. It can be examined countless times by countless persons using countless methods to obtain forensically sound results. The same cannot be said of other forensic fields. Once a physical substance has been tested (blood, drugs, a human body, etc..), it cannot be tested again as if it were never tested in the first place, nor can the substance be duplicated or preserved as electronic evidence can. Once preserved, electronic evidence does not spoil or rot.

For the computer "labs" having to comply with medical lab standards, I would expect compliance failures to be a regular occurrence, which will impact forensic admissibility in the courtroom for the simple fact of failure to follow an impossible-to-follow policy regardless of how perfect of a forensic exam was conducted.

As always Brett, you are a voice of reason. I agree 100% to this. While we have a lot of similarities to "traditional" forensic sciences, DFIR is unique and should be treated as such. Not forced into a bucket that doesn't fit.

ReplyQuote
Posted : 24/01/2018 7:12 pm
jaclaz
(@jaclaz)
Community Legend

Isn't lab certification for computer work overkill and unnecessary?

IMHO yes and no.

Yes, it makes no sense whatsoever because it is inherently a "moving target", and - provided that actual ISO17025 is applied (which I doubt) the consequence is that either the lab will "remain behind" or a lot of resources (which doesn't seem to be that much abundant) will be (should be) diverted to verification and compliance of tools and methods.

No, because - at least judging from the recent news - some (hopefully only isolated cases) laboratories (or analysts or both) make so sloppy (or plainly wrong or partial/incomplete) reports that *something* needs to be done about it.

Other lab types that work with chemical, biological, or nuclear substances surely need accreditation with substantial regulation for public safety measures.

Not only, it is much easier to create a standard for them, because the amount of variability in the "source" and the total number of possible tests are limited and - almost - always the "same" ones.

I believe the "lab" should not be any part of regulation.

The path to make DFIR more like other forensic fields should be directed at the person, not the lab, as electronic evidence is different than any other type of evidence.

Still that would amount to *needing* some "standard" certification of sorts linked to the person instead of the lab, a much better IMHO approach, but - without some good ideas on how exactly to have that - only moving the actual target without solving the issue.

Electronic evidence can be preserved in a static state (as if preserved in amber), reproduced, duplicated, transmitted, and copied without alteration. It can be examined countless times by countless persons using countless methods to obtain forensically sound results. The same cannot be said of other forensic fields. Once a physical substance has been tested (blood, drugs, a human body, etc..), it cannot be tested again as if it were never tested in the first place, nor can the substance be duplicated or preserved as electronic evidence can. Once preserved, electronic evidence does not spoil or rot.

Sure, but when possible and within limits even biological samples can be re-tested (and often are when there is an appeal or similar).
If the "scientific data" from the very first test has - and it increasingly seems like it is the case lately - the potentiality to put (and keep) innocent people in jail, or viceversa allow the culprit to get away free of charges, *something* in the quality assurance of these "scientific data" *needs* to be done.

For the computer "labs" having to comply with medical lab standards, I would expect compliance failures to be a regular occurrence, which will impact forensic admissibility in the courtroom for the simple fact of failure to follow an impossible-to-follow policy regardless of how perfect of a forensic exam was conducted.

Yes, and that is IMHO the worst aspect.

Since the "other part" (be it prosecution or defense) is essentially made by lawyers, i.e. people that by trade look for and find even minimal defects in whatever/whoever the other part brings in court (be that eyewitnesses, expert witnesses or reports), this would open a whole Pandora's Vase of "procedural exceptions" or similar.

With the consequence that digital investigators may shift their competence and focus from actually finding out what is contained in the evidence to making sure that the methods through which this content is extracted and interpreted is compliant as much as possible to the "impossible-to-follow" policy, i.e., in a nutshell, provide less or worse data.

jaclaz

ReplyQuote
Posted : 24/01/2018 7:46 pm
JaredDM
(@jareddm)
Active Member

This conversation makes me glad I moved into data recovery and stayed out of DF. I'm happy the government(s) neither understand or care enough about data recovery to even attempt regulation.

I find it hilarious when other data recovery labs try to post their "accreditation" of a certain ISO standard. Last time I checked DriveSavers was showing off their latest ISO certificate. But if you looked up the number it was just relating to cloud storage and has nothing to do with data recovery at all.

If ISO 17025 is like most ISO accreditation processes I've researched in past jobs, half the requirements are clearly written by bureaucrats who have no understanding of practical application at all.

Computer calibration. That's funny. Better make sure that your clock isn't running .23ms fast. That could get the whole case thrown out. lol

ReplyQuote
Posted : 24/01/2018 8:56 pm
minime2k9
(@minime2k9)
Active Member

No, because - at least judging from the recent news - some (hopefully only isolated cases) laboratories (or analysts or both) make so sloppy (or plainly wrong or partial/incomplete) reports that *something* needs to be done about it. [\quote]

Except the recent mistakes, assuming the rape trials, relate to non-disclosure of data. The phone extraction had been done and no evidence it wasn't done correctly. The accreditation wouldn't have stopped it. The other instance related to facebook messages extracted by a front line officer. Again outside the accreditation scope. So ironically wouldn't have affected them

ReplyQuote
Posted : 24/01/2018 9:46 pm
MDCR
 MDCR
(@mdcr)
Active Member

0710 2018-01-25

Once preserved, electronic evidence does not spoil or rot.

Given enough time (decades), digital media, even preserved on the best of whatever we have today will degrade and become unusable. I'm part of a retro community that deals with preservation of software, and many are reporting some disks are reported as being unable to read from.

Not to forget, hardware go away and is replaced by new technology, requiring hardware also to be stored for media to be recovered in the future, and hardware has a shorter shelf life (i.e. electrolytic capacitors go bad and leak, as well as batteries that barf acid all over the motherboard). I know of one company that has been storing tech since the 60s and their technology storage is significant, I'd call it a museum.

No, because - at least judging from the recent news - some (hopefully only isolated cases) laboratories (or analysts or both) make so sloppy (or plainly wrong or partial/incomplete) reports that *something* needs to be done about it.

So it's more of a people problem. Can also be taken care of by having 2-hand sign-off instead of forcing people to pay for a piece of paper.

Computer calibration. That's funny. Better make sure that your clock isn't running .23ms fast. That could get the whole case thrown out. lol

That kind of stuff. People applying standards because of knee j**k reaction on things that are outside the standard and someone trying to make money on certification/accreditation. It's the PE licensing idiocracy thing all over again.

ReplyQuote
Posted : 25/01/2018 6:24 am
jaclaz
(@jaclaz)
Community Legend

The accreditation wouldn't have stopped it.

Exactly. )

Maybe I expressed myself poorly ? , I referenced those recent case to point out how "digital data" have a great relevance in Police and Courts decisions and thus mistakes revolving around these "digital data" (both inside and outside the laboratory) may have serious consequences. thus it makes little sense to have very strict norm on the way the data is acquired in a laboratory if data can be acquired outside the laboratory or can be mishandled by other actors in the process.

The decision to apply ISO 17025 to digital forensics laboratories - besides and before discussing whether the norm is applicable/right or not (it is not IMHO) - only patches a small part of what is involved in a fair and correct trial.

A "better" or "more suitable" standard (or however guideline or procedure) is - as I see it - however needed and should cover the whole process of discovery and reporting not only the mere activities in the laboratory.

At the end of the day the people - righteously - expect that the judiciary system will punish (adequately) the guilty and leave the innocent free, i.e. they want justice to be - as much as possible - just.

jaclaz

ReplyQuote
Posted : 25/01/2018 1:26 pm
Merriora
(@merriora)
Junior Member

Thank you to everyone that is taking part in this discussion. I personally don't have the answers to this complex discussion so I value everyone's input so that we can all better understand the pro's and con's of possible future standards.

From the 17 votes so far, it is clear that 17025 has its issues and few are a fan of this current standard.

For those that have voted that "Standards are NOT required", do you mind posting comments as to why you believe this? By understanding everyone's argument, hopefully, we can move our community in the correct direction.

Isn't lab certification for computer work overkill and unnecessary?

IMHO yes and no.

Yes, it makes no sense whatsoever because it is inherently a "moving target", and - provided that actual ISO17025 is applied (which I doubt) the consequence is that either the lab will "remain behind" or a lot of resources (which doesn't seem to be that much abundant) will be (should be) diverted to verification and compliance of tools and methods.

A "better" or "more suitable" standard (or however guideline or procedure) is - as I see it - however needed and should cover the whole process of discovery and reporting not only the mere activities in the laboratory.

Jaclaz Correct me if I'm wrong, but you feel that a new standard covering everything from the initial collection to the final testimony in court would be most appropriate. This is assuming the new standard is not too rigid allowing examiners to do their work in our ever-changing environment without being required to verify and validate tools individually on an ongoing basis.

In your experience, have you come across any current standards that could be adapted to fit within Digital Forensics?

My personal opinion is that a standard would work better if it was less rigid in regards to validations of tools and instead placed emphasis on the validation of the data that is extracted and included within the final Digital Forensic Report as we are taught to do. Each tool can and will extract and display data differently and its the validation steps we as examiners take afterward that is essential to confirm that the extracted data is correctly interpreted and presented within our reports.

I believe the "lab" should not be any part of regulation.

The path to make DFIR more like other forensic fields should be directed at the person, not the lab, as electronic evidence is different than any other type of evidence.

Brett Do you see any sort of standard for labs?

What's to stop a lab from following improper protocols by having inexperienced examiners taking on work beyond their skill level?

If an examiner is highly regarded and then does contract work in a lab that is later found to cut corners and not follow basic common sense procedures, will that examiner's reputation be tarnished unfairly?

How do you see individuals meeting standards? Would this not be similar to certificates?

I believe that perhaps a combination of standards for both the lab and examiner is needed. My fear is that if we eliminate a standard for the lab, then they have less of an incentive to ensure the work is done appropriately and instead worry more about the financial costs cutting corners where they can to gain an advantage over their competition.

ReplyQuote
Posted : 25/01/2018 2:40 pm
steve862
(@steve862)
Active Member

Hi,

Before I came into digital forensics (in 2004) I worked in traditional scientific test laboratories, most notably a pharmaceutical lab (AH Robins) and a microbiology lab (Malthus Instruments).

I got to know these environments and working practices pretty well and when I read 17025, it makes complete sense to me for those environments.

One of the things I believe even some UKAS assessors don't fully appreciate is the level of translation already required to go from what 17025 was intended for, into a wet forensics environment.

To then port it across into digital forensics, requires an even greater level of translation to the point where some points have become meaningless when you consider what the original authors intended to achieve.

There's so much more I could and want to say but it's all pretty much been said many times over. I just wanted to make the point that we shouldn't forget this standard was not written for forensics at all and from my point of view, having worked in traditional scientific test laboratories and digital forensics, I can see how wide the gulf is between the two disciplines.

Steve

ReplyQuote
Posted : 25/01/2018 3:38 pm
trewmte
(@trewmte)
Community Legend

“Much of the digital forensic community desires to have their evidence seen in court as forensically sound and bulletproof, yet do not want to go through the rigors that other traditional forensic sciences have done to prevent evidence spoliation and other mishandling and misinterpretations.”
~ Josh Moulin
Deputy Chief Information Officer
US Federal Government, National

ISO 17025 is now the mandatory standard in the United Kingdom for all Digital Forensics Laboratories.

Will the Digital Forensics community in the United States, Canada, Australia and elsewhere adopt ISO 17025, another standard, or wait for one to be imposed on them by a government agency?

Merriora just an observation and at the same time throwing a spanner in the works. How would you envisage cross-border evidence being accepted. For instance, you send evidence to the UK acquired during a joint operation with the US and evidence is for submission to the UK Criminal Court. Should the court assess your evidence against your standards or ISO17025?

ReplyQuote
Posted : 25/01/2018 4:06 pm
athulin
(@athulin)
Community Legend

For the computer "labs" having to comply with medical lab standards, I would expect compliance failures to be a regular occurrence, which will impact forensic admissibility in the courtroom for the simple fact of failure to follow an impossible-to-follow policy regardless of how perfect of a forensic exam was conducted.

Medical labs typically answer very narrow questions ïs this human blood? does it match blood samples X, Y, Z? If it does, what is the probability for a random match? They aren't asked 'did X murder Y?' or 'How did the DNA of X end up under Y's nails? (And if they are, they should have the sense to refuse the job…)

Computer forensic labs are a odd mixture of investigation on one hand, and question-answering on the other. Much more investigation, much less scientific fact-finding.

In the general area of investigations, I don't think standardization is of any use. But the narrow questions, such as 'was this file modified on

Just as that lab being asked 'does this blood speciment belong to anyone known?' should have a considered methodology for answering that question, with known source of error (including verifying that the specimen a) is blood, and b) is human blood before the question of identity using a DNA database is addressed), I believe that similar specific questions that a computer forensic lab is asked to answer should have a similar methodology, and similar appreciation of errors affecting a result.

That is, ISO 17025 has a place. But it seems it is being applied as a wet blanket in the hope that it may cover and cure everything, instead of being applied as hot poultice for a specifc purpose and limited area of application. (Please ignore my medieval notions of medical treatment …)

ReplyQuote
Posted : 25/01/2018 4:29 pm
Merriora
(@merriora)
Junior Member

Merriora just an observation and at the same time throwing a spanner in the works. How would you envisage cross-border evidence being accepted. For instance, you send evidence to the UK acquired during a joint operation with the US and evidence is for submission to the UK Criminal Court. Should the court assess your evidence against your standards or ISO17025?

This is an excellent question and likely a valid argument for some sort of Lab accreditation compared to only having accredited examiners. But hopefully, if the DF community in the rest of the world moves towards a different standard, solutions would exist to allow for the sharing of information. Just because the UK is ISO 17025 should not force the rest of the world to follow along if the standard truly doesn't fit Digital Forensics.

Assuming a new standard is developed, I would hope that it would be sufficient to compete against 17025 in regards to reliability from the lab.

Perhaps if a new standard is developed, then the UK would look to follow (far future) if that made sense and issues still existed with the implementation of 17025.

ReplyQuote
Posted : 25/01/2018 4:54 pm
Page 1 / 9
Share: