Controversy has been raging around ISO 17025 ever since the standard was adopted for digital forensics back in October 2017. Although many people who work in the industry agree that standardisation is advisable and probably necessary if we are to keep moving forward, there have been many criticisms of ISO 17025 and its effectiveness when it comes to digital forensics.
The baseline of the problem seems to be that ISO 17025 was not specifically designed for digital forensics; instead, it takes the standards of ‘wet’ or traditional forensics and applies them to computing devices. This has a number of issues, not least the fact that technological advances are constantly happening; in a field where most large apps are being updated a couple of times per month as a minimum, it becomes very difficult to properly standardise tools and methodologies.
Another concern for many people is the cost associated with accrediting a lab and keeping up with ISO 17025. Reports of accreditation costing in excess of £50,000 have made some practitioners nervous about applying.
If you want your opinion on ISO 17025 to be heard by the people who make the decisions, now’s your chance. The House of Lords’ Science and Technology Select Committee is conducting an enquiry into forensic science and inviting individuals and companies to submit evidence for consideration.
In total there are seventeen questions making up the enquiry, three of which are specifically focused on digital forensics:
- Are there gaps in the current evidence base for digital evidence detection, recovery, integrity, storage and interpretation?
- Is enough being done to prepare for the increasing role that digital forensics will have in the future?
- Does the Criminal Justice System have the capacity to deal with the increased evidence load that digital forensics generates?
The current enquiry springs from the 2015 Government Chief Science Advisor’s Report, Forensic Science And Beyond, which included a section on ‘the domain of cyberspace’. The report discussed questions such as the global nature of cybercrime; the proliferation of devices and data; and a shortage of skills in the field. These are all questions that need to be addressed, and in our own recent survey Forensic Focus’ readers on the whole agreed that standardisation would be a positive step for the industry.
When asked whether their organisations were planning to attain ISO 17025 accreditation, the majority of respondents to our survey said ‘No’.
However, when asked about the necessity of a standard for digital forensics in general, many people replied more positively.
Almost 62% of respondents agreed that some means of standardisation is necessary for the community. However, people were less likely to advocate for ISO 17025 specifically, with just 23% of people agreeing that it would be good for digital forensics. Just under 23% of respondents were on the fence about whether ISO 17025 could be used to cover all necessary aspects of digital forensics standardisation, but only 6.67% of people thought it could.
So if we agree that standardisation would be helpful, why isn’t the digital forensics community embracing ISO 17025 with open arms? Forensic Focus asked respondents to our recent survey for their thoughts on the standard; here are some of the replies.
“[ISO 17025 accreditation is] too expensive, the money could be spent on training.”
The financial shortcomings presented a popular reason for people’s reluctance to put themselves forward for accreditation. Running a forensics lab is an expensive endeavour at the best of times, whether you’re creating your own tools or using other people’s. There will always be a new development needed, or training on a new product, or a new operating system that suddenly plunges everything into the dark again.
Several people spoke about their frustration at the onus being on the labs themselves to demonstrate digital forensics tools’ effectiveness, rather than on vendors:
“[ISO 17025 is] inappropriate for the UK LE digital forensics community. Standardisation is a good thing, so why are different forces receiving different advice and assessments? Why does each force have to validate their tool use, why aren’t the tool vendors being assessed directly?”
Conversations both online and offline recently have veered towards this topic, often with people expressing confusion about what they are even meant to do in order to become accredited.
“It’s an incredibly time consuming & expensive process – there’s no central governance to go [to] for help, or to share best practice. Everyone seems to be going it alone.”
Respondents also expressed concern that the standard was focusing on the wrong things, and meant that people were spending less time on the jobs at hand and more time on the bureaucracy required to keep up the standard, thus arguably having the opposite effect from its goal.
“ISO 17025 should have been driven from the centre and should not force each organisation to spend considerable time and effort to get to a place where it is obvious people need to be employed simply to [be] administrators and checkers. At the moment valuable time is spent not processing case work but checking others’ work or following a tick box regime rather than empowering people to think for themselves, solving problems in a logical way appropriate to the investigation in hand.”
Some are worried that larger corporations will see it as a money-making exercise rather than a way to ensure consistent standards across the industry, and that this may have a negative effect on digital forensics as a whole. The amount of time it takes to adhere to ISO 17025 is another frequently cited challenge.
“It is being massively interpreted across the public sector. It is supposed to set standards; however, to reach those new standards, inconsistent procedures are being put into place. ISO is seen by many as purely a money-making exercise and is not respected by a lot of colleagues. Where law enforcement is concerned, it has massively increased the time taken to examine an exhibit, with little or no benefit in return.”
Other respondents were skeptical about the usefulness of standardisation on the whole:
“It is liable to create too much emphasis on having the accreditation, which organisations are spending an obsessive amount of time on, in turn neglecting the core role of doing digital forensics. As long as protocols are adhered to within the law of the land then that should be sufficient. The evidence test in a courtroom will NOT be whether you have the ISO standard! A DFI whose organisation has ISO will likely achieve the same/similar results to a DFI who does not have ISO.”
The general feeling among the community seems to be that standardisation on the whole is a good idea, but that ISO 17025 might not be the right way to go about it. If you’d like to have your say and make your views heard, you can find more details about how to do so in this PDF or at Parliament.uk. Responses need to be no longer than six sides of A4 paper, and they must be submitted to the Committee by the 14th of September 2018.
Given that the aim of ISO 17025 accreditation and Forensic Science Regulation is to reduce the number of times forensic science goes wrong it would be interesting to see how often this occurs for digital forensics and for what reasons. Would people like to post instances of where there have been miscarriages of justice (or similar)? Was this because of the use of bad tools and/or procedures which gave misleading results? Or because wrong inferences were drawn? Or that a reconstruction of events had been over-selective, suffered from confirmation bias, and had not considered possible alternative hypotheses? Or because experts gave evidence outside the scope of their actual expertise? Obviously the greater the detail, the more interesting the scenario. Or, heaven forbid, perhaps there aren’t all that many examples?
After working with this for 20+ yrs, Technology will always change the the Digital Forensic tools need to as well. BUT,As investigators we need to reveal the facts. When tested on the same data set, three tools gave separate results. UGH. The big BUT is Clients and Counsel have specific needs, Investigators want to go all the way. Remember who is your buyer, and never get in bed with one company
Always good when you notice a typo in your first sentence after clicking post (world is = world it).
Although I’m sure there’s many more in that giant diatribe of mine!
Hi Rich, I can see the full text of your comment via the email alerts but it’s not showing up in the approval queue – I think it may have been too long to go through. We might be able to publish it as a separate article / opinion piece; if this would be of interest, please drop me a line on scar@forensicfocus.com and we can look into it.