ACPO Principles Revised

The badass cybercriminals are increasing the cost of their tools…so they will be able to afford iso17025 and absorb the costs within their commercial concerns…

Naah, only a small subset of honest Criminals[1] will do that without an intervention by the Regulator, not entirely unlike what UK forensic firms did at the time (or did I miss anyone actually volunteering for ISO 17025 [2]?)

jaclaz

[1] yes, this is an oximoron
[2] and yes, "volunteering for ISO 17025" more than an oximoron is a logical impossibility

Actually there is a reason for a place to want to do ISO17025 and it will have happened….but it's not to improve the quality of the forensic work…..it's for companies to use as a selling point to appear better than the competition and/or win work/contracts (despite it obviously not being worth the paper it's printed on).

We wrote to the regulator and explained that we were the only company in the UK specialising in the examination of compromised payment systems and equipment used to defraud banks over almost 20 years. We also pointed out that we had assisted almost all the Police forces in the UK as well as many abroad and that we had been able to obtain funding for most of this work from the UK banking community, so at no cost to the Police or the public.

She replied

"Your company sounds, from your letter, to be very successful in obtaining
work and funding. You also say that you are regarded by most as the leading
experts in the field. It is, however, difficult for many who commission work in
the criminal justice system (CJS) to know which companies have real
expertise and which simply operate low-cost data extraction tools and
interpret the outputs based on their own view of the world. Adoption of quality
standards cannot completely eliminate the possibility of error, but it does
provide assurance to the CJS that the organisation has the sustainable
competence to deliver the work, with the appropriate equipment and/or
software and methods that can be relied upon. Defence review cannot be the
sole quality control, as many cases do not involve such review. Indeed, many
of the more common failures in digital forensics would result in evidence not
being found, and justice being denied to victims."

So we closed the company and then one of the Police units we used to provide support to sent a computer and a skimmer to an "approved" ISO17025 company who then returned the skimmer un-examined with an explanation that they had not been able to examine it as they did not have the correct download cable.

If they had had the correct cable and the correct download program they would then have discovered that the device was password protected.

So UK Law Enforcement have now lost a valuable and free resource as a result of the Regulator's insistence on ISO17025

She replied

"It is, however, difficult for many who commission work in the criminal justice system (CJS) to know which companies have real expertise and which simply operate low-cost data extraction tools and interpret the outputs based on their own view of the world."

Wow.. last time I read something like that was Gerald Ratner…

"How can you sell this for such a low price?", I say, "because it's total crap."

[https://www.telegraph.co.uk/business/2017/08/03/mirren-nine-times-people-undermined-products-supposed-selling/]

Maybe CCFI you can confirm date and time of that statement by the FSR?

She replied

"It is, however, difficult for many who commission work in the criminal justice system (CJS) to know which companies have real expertise and which simply operate low-cost data extraction tools and interpret the outputs based on their own view of the world."

Wow.. last time I read something like that was Gerald Ratner…

"How can you sell this for such a low price?", I say, "because it's total crap."

[https://www.telegraph.co.uk/business/2017/08/03/mirren-nine-times-people-undermined-products-supposed-selling/]

Maybe CCFI you can confirm date and time of that statement by the FSR?

The letter was received by email as an attachment on the 17th January 2017 @ 1336

I keep a copy in my Dropbox

I then raised my concerns with my MP on 12 March 2018 who forwarded my email to Nick Hurd who was the Minister of State for Policing and the Fire Service at the time.

He sent a reply attached to an email to my MP dated 18th April 2018 but received by me on 23rd April 2018 @ 1157

It also makes for an interesting read.

I keep a copy of his letter in my Dropbox too.

"It is, however, difficult for many who commission work in
the criminal justice system (CJS) to know which companies have real
expertise and which simply operate low-cost data extraction tools and
interpret the outputs based on their own view of the world. Adoption of quality
standards cannot completely eliminate the possibility of error, but it does
provide assurance to the CJS that the organisation has the sustainable
competence to deliver the work, with the appropriate equipment and/or
software and methods that can be relied upon."

There is a single German word for this
Verschlimmbessern

https://en.wiktionary.org/wiki/verschlimmbessern

jaclaz

It’s a square wheel - it fits the box perfectly, but it’s not much use as a wheel

It’s a square wheel - it fits the box perfectly, but it’s not much use as a wheel

Well, the improved version, the triangular wheel, removes one bump wink .

jaclaz

However on a serious note it is just so sad that this has put my field of digital and electronic forensics back by about 20 years for Law Enforcement in the UK

The resources and technical support that we could access was absolutely cutting edge and was way beyond the capabilities and access of most in-house units and commercial companies.

However I still do some consultancy work for other Countries and the occasional case in the UK although I now live in a warmer Country (24c and sunny here today)

However on a serious note it is just so sad that this has put my field of digital and electronic forensics back by about 20 years for Law Enforcement in the UK

You're right, and on a serious note it is perhaps worth considering a number of aspects of [iso17025 plus FSR Rules] which is now de-facto as setting 'benchmarks'.

iso17025 holds authority because of its mandated position as the Standard with requirements to meet accreditation assessment.

There has been no success achieved moving digital forensics to a new umbrella standard. The give away reason is in the following titles

ACPO Guidelines
ISO/IEC 270372012, Guidelines for identification, collection, acquisition, and preservation of digital evidence
ISO/IEC 270422015 “Guidelines for the analysis and interpretation of digital evidence”

Guidelines are simply what they are, nothing more, nothing less.

The FSR was smart enough to recognise that fact and include these "guidelines" in FSR guidance/codes where candidates might wish to use them to support the foundation requirement of iso17025. By doing this they were shown to have meaningful, but lesser status than the accreditation standard itself.

As it currently stands, there is no expectation that validation requires the entire functionality of any given tool to be tested; it is not feasible. It is for each (main or satellite) facility to ensure an acceptable tool testing process is in place with 'justification' and 'rationale'. Equally, 'validation' should show robustness under scrutiny.

iso17025 accredited labs assessment will be widened again to be assessed for Cybersecurity and InfoSec.

In opposition to the above, digital forensics commentators, and when giving evidence to the Committees (House of Commons and House of Lords), have expressed concerns but have not been able to put forward another assessment Standard for the accreditation of digital forensic labs. For instance, a BSI Standard containing digital forensic lab and testing assessment requirements (and I don't mean solely iso9001 which can be used when transitioning to iso17025) would have been helpful!

It would appear if iso9001 + demonstration of competence standard + demonstration of test facility standard were shown to be an acceptable opposition to iso17025 then could that really raise the profile for any change digital forensics wants or needs?

However, spare a thought for those law enforcement facilities that have been successful in gaining accreditation under iso17025 - what does that say about their achievements? What have they been doing to pass assessment for their digital forensics facility? Why are they not speaking out about excesses and over-complication?

iso17025 & FSR Codes, as they currently stand, are not helpful to small business and single-person enterprises. If you haven't yet read, do read Forensic Science Regulator's Annual Report 17 November 2017 – 16 November 2018 [Dr Gillian Tully 15 March 2019] "2.8 Sole Traders and Small/Micro-Businesses"

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/786137/FSRAnnual_Report_2018_v1.0.pdf

======
Short background material about iso170252017

ISO/IEC 170252017, General Requirements for the Competence of Testing and Calibration Laboratories, released in 2017, contains all the requirements that testing and calibration laboratories must meet to demonstrate that they operate a quality management system, are technically competent and can generate technically valid results.

This ISO/IEC 170251999 initial release of the standard replaced ISO/IEC Guide 25 and the European Union’s EN 45001. ISO/IEC 17025 goes beyond both of these standards by adding new requirements, along with significant changes to previous requirements. The 2017 version of the standard was released in November of 2017 to better align the standard with the requirements of ISO 90012015. While many management system elements of ISO/IEC 170252017 mirror those of ISO 9001, the international quality management system standards, its additional technical competency requirements are unique for testing and calibration laboratories.

ISO/IEC 170252017 is the international basis for accrediting calibration and testing laboratories. It applies to both freestanding laboratories, as well as laboratories which are part of a larger facility. When a laboratory is part of a larger facility, ISO/IEC 170252017 accreditation can be achieved simultaneously with ISO 90012015