±Forensic Focus Partners

Become an advertising partner

±Your Account


Username
Password

Forgotten password/username?

Site Members:

New Today: 0 Overall: 36317
New Yesterday: 0 Visitors: 183

±Follow Forensic Focus

Forensic Focus Facebook PageForensic Focus on TwitterForensic Focus LinkedIn GroupForensic Focus YouTube Channel

RSS feeds: News Forums Articles

±Latest Articles

±Latest Videos

±Latest Jobs

ACPO Principles Revised

Computer forensics discussion. Please ensure that your post is not better suited to one of the forums below (if it is, please post it there instead!)
Reply to topicReply to topic Printer Friendly Page
Forum FAQSearchView unanswered posts
Page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9, 10  Next 
  

trewmte
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 15, 19 20:41

- tootypeg
Curiously, is it the testing of tools which people think is a bigger issue or the evaluation of whether people are interpreting the output correctly? Slightly outside of the scope of things but I bet the discovery of substantial tools errors would be far less than discovering misinterpreted (even partially) findings.


Depends if you deny the tools should be validated or the opinion/interpretation?
_________________
Institute for Digital Forensics (IDF) - www.linkedin.com/groups/2436720
Mobile Telephone Examination Board (MTEB) - www.linkedin.com/groups/141739
Universal Network Investigations - www.linkedin.com/groups/13536130
Mobile Telephone Evidence & Forensics trewmte.blogspot.com 
 
  

tootypeg
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 15, 19 21:03

I think tools should/need to be tested. I do also think this is a very difficult task. I think how a practitioner interprets findings needs to be tested.

I dont know which is the bigger task/bigger threat.

At the moment I think there is a lot of focus on tool-testing, on the assumption that they need to be validated. This suggests theres an expectation to find error. My question is, is it assumed that tested tools is all we need to do? I see very little narrative around the practitioner. A fully working tool does not guarantee quality results.

If we assume the vendors are doing some testing of their tools, I suspect there is little to none in regards to testing practitioner interpretation. So where are the resources better spent? It seems at the moment we are doubling up with vendors to test, and nothing on the other issue i raised.  
 
  

trewmte
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 16, 19 07:58

- tootypeg
I think tools should/need to be tested. I do also think this is a very difficult task. I think how a practitioner interprets findings needs to be tested.

I dont know which is the bigger task/bigger threat.

At the moment I think there is a lot of focus on tool-testing, on the assumption that they need to be validated. This suggests theres an expectation to find error. My question is, is it assumed that tested tools is all we need to do? I see very little narrative around the practitioner. A fully working tool does not guarantee quality results.

If we assume the vendors are doing some testing of their tools, I suspect there is little to none in regards to testing practitioner interpretation. So where are the resources better spent? It seems at the moment we are doubling up with vendors to test, and nothing on the other issue i raised.


Very good, and insightful tootypeg. iso17025 and FSR Codes both identify training. Using clauses from iso17025:2017

6.2.5 The laboratory shall have procedure(s) and retain records for:
a) determining the competence requirements;
b) selection of personnel;
c) training of personnel;
d) supervision of personnel;
e) authorization of personnel;
f) monitoring competence of personnel.

6.2.6 The laboratory shall authorize personnel to perform specific laboratory activities, including but
not limited to, the following:
a) development, modification, verification and validation of methods;
b) analysis of results, including statements of conformity or opinions and interpretations;
c) report, review and authorization of results

Do you see these as the relevant clauses to your observations "I suspect there is little to none in regards to testing practitioner interpretation." and "I see very little narrative around the practitioner."?

- How do you envisage educating deep-level skills and experiences?
- Does that include Metrology and so on?
- Is it solely focussing on data interpretation with relevance to the outside world and/or data found in the device?
- Do you foresee this only for Police labs or accredited service providers, also?

In the alternative, do you envisage the use of expert subcontractors to an accredited lab (Police/Private)?
_________________
Institute for Digital Forensics (IDF) - www.linkedin.com/groups/2436720
Mobile Telephone Examination Board (MTEB) - www.linkedin.com/groups/141739
Universal Network Investigations - www.linkedin.com/groups/13536130
Mobile Telephone Evidence & Forensics trewmte.blogspot.com 
 
  

trewmte
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 16, 19 08:22

- CCFI
- trewmte
- Rich2005
So what do they do....blame it all on "unregulated" digital forensics and impose a silly standard that makes things worse in a variety of ways. It's counter-productive but those at the top of the tree will bang on about things being better now despite it being clearly untrue (because doing otherwise would make them look bad and solving the real issues isn't the aim).


As always your feedback is very enlightening. Are you in a position to (even if it is anonymous) provide substantive evidence or demonstratively, traceable example where things are not better due to iso17025 or FSR Codes?


Hi - I can

In 1999 we developed some in-house software that was able to recognise bankcard magnetic strip data at very high speed from a computer image copy data file. It was also able to recognise the bank and the country that issued each recovered record.
The banks heard about it and called us up to a meeting in London. At the meeting we demonstrated the software to the police and the banks. In a couple of seconds we recovered 341 bankcard records from an image copy of a floppy disk.
The banks said “This is exactly what we're looking for. At the moment there is a nine month backlog to examine any computer and then it takes a year to come to court and then maybe somebody tells us the bank accounts that were found. This means that we are exposed to fraud for almost 2 years on each account which results in a loss to the banks.
What if we pay you to examine the computers and extract the account numbers and get them up to us within 24 hours so that we can block the accounts and investigate any fraud spend, and then we'll pay you to examine the computers and produce an evidential pack for the police in 6 to 8 weeks.”
The police said “So we get a free evidential pack in 6 to 8 weeks instead of nine months?”
So that's what we did - any police force seizing computers or computer equipment that were suspected of being used to defraud the banks could send it to us for rapid examination.
So the police could arrest a suspect, and deliver the computers to us and we could extract a comprehensive list of all the bank cards found within it.
The police have a 24-hour custody clock, and have to release suspects, unless they get a 12 hour extension, if they don't get enough evidence to charge.
Many of the OCGs involved in bank card fraud are operating across national borders, so when released “on bail” or “under investigation” the suspects simply disappear.
We could give the police a schedule of compromised card numbers and a witness statement in a matter of hours which could be put to the CPS for a charging decision within the custody clock time.
Now we are no longer operating, the police cannot obtain this free and fast service, and many of the suspects simply disappear.
There is no way that this can be described as an “improvement” because of the introduction of ISO17025.
And we did it many hundreds of times for 18 years and recovered hundreds of thousands of compromised account details which meant that they could not be used to raise funds for further OCG use.


Being as this is your business case, it doesn't benefit the wide ranging level of practitioner in the marketplace. The outline is good about your previous success but in itself it would be an uphill struggle for you to succeed in the current climate. What you could do is to an give up-to-date foundation to your claim to precisely define which iso17025 and FSR parts, clauses and codes are flawed when analysed against the business outline you have given above.

Equally, you may think it a good idea to be in-touch with the College of Policing and the NPCC to flat-plan (layout) the process of flaws or even failures. The representation might include subcontractor to an accredited lab and do not rule out tying in with a existing accredited service provider. I suggest the last point because I think there is something of an opportunistic opening for you that could be a take away from the Police Chief's comments to you about paying "once".

As a prospective subcontractor you would need to show support and adherence to the laboratory quality and compliance principles in addition to gauging how you fit into the FSR Code (Issue 4) provision:

Code of Practice for forensic units providing forensic science services

1. Introduction

1.1.1. This Code of Practice is aimed at all those providing forensic science services to the Criminal Justice System (CJS), whether individual practitioners, academics, public or private sector forensic science providers. Previous versions of the Codes referred to these as providers, however as this is interpreted by some as commercial providers. This version of the Codes refers to all as forensic units in line with the terminology used in ILAC G19:08/2014. These can be small teams in larger organisations, sole practitioners or large providers and can be instructed by the prosecution or the defence.
_________________
Institute for Digital Forensics (IDF) - www.linkedin.com/groups/2436720
Mobile Telephone Examination Board (MTEB) - www.linkedin.com/groups/141739
Universal Network Investigations - www.linkedin.com/groups/13536130
Mobile Telephone Evidence & Forensics trewmte.blogspot.com 
 
  

GumStickStorage
Newbie
 

Re: ACPO Principles Revised

Post Posted: Nov 16, 19 10:29

Well this is very interesting stuff.

This thread has essentially answered a bit of my dissertation. I have a list of guidelines to study and analyse and everything said here is once again, valuable information. The constructive criticism just reinforces the value of this thread.

Despite its popularity, the ACPO guidelines is part of that list, and everything being said here will help me tackle what could be changed. For example, I saw a comment (which I can't find now due to so many replies) that ACPO focuses too heavily on the theoretical side rather than practical. This has now become one of my sub-questions within this project.

The great thing about what I'm doing is that there is no right or wrong answer: I could say that the ACPO principles (and the rest of the guidelines) are OK the way they are, or I could say that they need amendments. It's the fact that I've conducted research methodologies as an attempt to get an answer, but everything that is said here is a goldmine.

Thanks for this post tootypeg, you've really helped me out here, along with those who have responded.  
 
  

Rich2005
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 16, 19 11:26

- trewmte
- Rich2005
So what do they do....blame it all on "unregulated" digital forensics and impose a silly standard that makes things worse in a variety of ways. It's counter-productive but those at the top of the tree will bang on about things being better now despite it being clearly untrue (because doing otherwise would make them look bad and solving the real issues isn't the aim).


As always your feedback is very enlightening. Are you in a position to (even if it is anonymous) provide substantive evidence or demonstratively, traceable example where things are not better due to iso17025 or FSR Codes?


That's practically impossible because nobody's going to want to admit the reality publicly and definitively (ie document the fact they just simply ignore them, or pass them then ignore them, or pass them and just do a token effort to maintain compliance, etc. Nobody is going to be following them to the letter, because it's simply impossible, for a lab conducting DF on a wide range of devices with a good range of tools).

You won't get evidence because nobody's going to document publicly that "I'm not following them because they're prohibitively expensive in time and money" or "We have to follow them but in practical terms it's prevented me taking the best course of action on case X/Y/Z, or wasting considerable time and effort, to the detriment of the quality/volume of work we can get through, and therefore ultimately the detriment of the public" or "we have to follow them, or follow them to win business, and now we're allocated even less time per case, and more corners are cut, because quite simply the time/money wasted on ISO17025, has to come from somewhere".

In many ways I think the CRFP was a more sensible starting point than something we know is going to be essentially impossible to achieve in DF (testing a decent sample of the tools/methods against the various potential data sets). I'm not particularly well versed in the practicalities of its demise but I suspect it suffered from good intentions but woeful lack of resources to achieve the kind of thing that it was setting out to do.

Everyone wants competent examiners and accurate tools...but ISO17025 doesn't even come remotely close to achieving either of those things. It certainly doesn't solve the problems of the scandals, that it got rammed through on the back of, which are essentially nothing to do with DF, or the standard of work produced. It won't solve problems with untrained officers rather than DF individuals interacting with evidence. It won't solve disclosure issues (if there are any). It won't make software reliable or anything like that. As you know, I don't think it actually provides any meaningful benefit to DF, at the cost of huge amounts of time and money.

In my view there's never been enough thought and planning right at the top, to specify what (if anything) is the problem needing to be solved, and THEN what can MEANINGFULLY solve or improve those issues. It's all been a combination of very political, commercial, and with a smattering of old FSS resentment (that's a guess and not specific to DF though - based on conversations with others outside of this field).

Many of the issues causing previous scandals could, and should, be addressed but they don't involve DF at their core (or at all).  

Last edited by Rich2005 on Nov 16, 19 12:18; edited 1 time in total
 
  

jaclaz
Senior Member
 

Re: ACPO Principles Revised

Post Posted: Nov 16, 19 11:49

- GumStickStorage
For example, I saw a comment (which I can't find now due to so many replies) that ACPO focuses too heavily on the theoretical side rather than practical. This has now become one of my sub-questions within this project.


Check my signature Wink .

And check (only tangentially related) the summing up of the tests made by NIST on image carving tools (which is a subset of a subset anyway):
www.forensicfocus.com/...c/t=18141/
www.forensicfocus.com/...2/#6601012


jaclaz
_________________
- In theory there is no difference between theory and practice, but in practice there is. - 
 

Page 9 of 10
Page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9, 10  Next