UK FSR Digital fore...
 
Notifications
Clear all

UK FSR Digital forensics method validation: draft guidance

21 Posts
10 Users
0 Likes
1,537 Views
(@trewmte)
Posts: 1877
Noble Member
 

Below are observations.

The document is only a draft so understandable the consultation document (metaphorically) speaks in a language at times undesirable to the technical specifics of the field of science or forensics it is referring. Always difficult to create a concept using language that if too widely ranged waters down the concept to mushy nothing-ness or too tight and it can narrow the scope making the concept unreasobale to produce any observed possible outcomes.

I must say I do like the fact the document language strives to make sure those producing evidence having recorded test results after an event has happened (e.g. cell site analysis) the respondent's report should avoid old evidential cliches e.g. 'the evidence is consistent with the defendant's mobile phone being at ….' and similar types of opinionated.

It would have been helpful to have identified those people involved in submitting the content as opposed to only knowing the collated content submitted in the draft was produced by the FSR. This would have been helpful to know to see whether the document has a swing in favour of public sector bias for their aims and ambitions or a free, undominated market where no one particular, or no handful of, orgainsation(s) or private company(ies) are influencing production line (bang it on, bang it out) evidence.

The document could have usefully stated that "accuracy of original evidence" is paramount and thus those producing the original evidence would themselves be subject to standards for compliance. The politics of the matters (but I could be wrong on this) suggests avoid historic references or inferring to e.g. repealed S69 PACE (the computer working properly at the material time) removing obligations on the network operators etc (a presumption of everything is ok) - thus no fornsic standard imposed - but requiring a similar approach to the old repealed s69 PACE for the examiner's equipment producing results using information from the original source to prove it is working properly at the material. I am not against the latter, but a forensic standard produce requires a quality in an unbroken (end-to-end) chain of evidence. It is hoped that the FSR defines the importance of the accuracy of information from original sources and underpins the important that those producing original evidence should meet that standard. I mentioned politics because s69 PACE was bemoaned as causing too high a standard on those producing evidence and too expensive for corporate or private companies.

There was at one time a principle operated in English law that e.g. an [SIC]operator could not profit from crime. The analogy of the millions of calls made by drug dealers and other crimes who paid the full profit price of those calls to the operator. The principle suggests the operator could only deducted that amount for the cost of running the calls and not keep the profit. The trade-off used to be provision of call records etc and a standard applied in their production as evidence. That was blown out of the water by repelaing at least one attempt at a safety net (the repealed S69 PACE). Things change in life, we all understand this. Until the Forensics arena gets to grips with quality in an unbroken chain of evidence there will a guarded approach to over-commit to new standards as the Forensic person or groups will not want to be savaged by unrecoverable running costs. To not take a stance can play into the hands of a Forensic oligarchy controlling the arena and that dreaded political utopia we are told about of 'living of the crumbs falling from the oligarchy's table'.

A well intentioned document and lots of positives, but it would benefit from the Forensics arena en masse setting out known pitfuls where the quality in the evidence is poor at the outset yet the examiner's report implies the original source material was accurate at the outset.

 
Posted : 30/10/2014 11:19 am
neddy
(@neddy)
Posts: 182
Estimable Member
 

It would have been helpful to have identified those people involved in submitting the content as opposed to only knowing the collated content submitted in the draft was produced by the FSR. This would have been helpful to know to see whether the document has a swing in favour of public sector bias for their aims and ambitions or a free, undominated market where no one particular, or no handful of, orgainsation(s) or private company(ies) are influencing production line (bang it on, bang it out) evidence.

Some info here https://www.gov.uk/government/organisations/forensic-science-regulator/about/our-governance

 
Posted : 30/10/2014 2:55 pm
(@dan0841)
Posts: 91
Trusted Member
 

It would have been helpful to have identified those people involved in submitting the content as opposed to only knowing the collated content submitted in the draft was produced by the FSR. This would have been helpful to know to see whether the document has a swing in favour of public sector bias for their aims and ambitions or a free, undominated market where no one particular, or no handful of, orgainsation(s) or private company(ies) are influencing production line (bang it on, bang it out) evidence.

Some info here https://www.gov.uk/government/organisations/forensic-science-regulator/about/our-governance

A number of very respected names on the digital group.

 
Posted : 30/10/2014 4:59 pm
Chris_Ed
(@chris_ed)
Posts: 314
Reputable Member
 

Then why are they making such a pig's ear of it?

Edit is it not an enormous, glaring conflict of interest to have a software vendor on the panel? I wonder if Magnet were asked to send a representative? Or Guidance?

Edit 2 Oh, I suppose this is why NetAnalysis v2 has words like "validating procedure" everywhere. And why the unworkable example in the documentation was for web browsing.

 
Posted : 30/10/2014 6:00 pm
(@trewmte)
Posts: 1877
Noble Member
 

Some info here https://www.gov.uk/government/organisations/forensic-science-regulator/about/our-governance

Thanks Neddy.

I didn't find the details of the names who make up the digital evidence group but only saw minutes of meetings. Do you have a weblink for the names and companies who make the digital evidence group. I want to trace who the likely candidates are who produced the content about cell site analysis and mobile phones.

 
Posted : 30/10/2014 11:14 pm
(@dan0841)
Posts: 91
Trusted Member
 

Do you have a weblink for the names and companies who make the digital evidence group. I want to trace who the likely candidates are who produced the content about cell site analysis and mobile phones.

I can't see a link to the list of names.

But if you mean in the original validation document there are a list of participants on page 102 of the document which was posted by the OP. They appear to relate to the appendix authors.

 
Posted : 31/10/2014 12:53 am
(@athulin)
Posts: 1156
Noble Member
 

FWIW I think UK practitioners are sleep walking into a bureaucratic disaster in the years ahead.

You migh need to explain why. 'Guidance and advice' it says … to my mind that means just that – if it was strict requirements or mandatory processes it would be another thing. But I suspect they try to start in right corner.

Overall, I'm positively surprised at last someone's trying to put the 'science' into 'digital forensic science'.

It probably means a minor hell for unaffiliated solo artists – if they want to follow the guidance, they have to do a lot of fairly basic work, which probably can't be justified economically.

On the other hand, that may mean that DF societies or similar interest groups may take up methods and validation as a kind of 'special interst group' work kind of thing, and build up a body of work that could be referenced by members. (LE would be such an organization to itself, probably.)

And that might lead to another kind of specialization in digital forensics becoming less special – one I thought would be restricted to LE (well, I can always hope …) and very major players for at least a decade or so – that of the validation expert.

Looks like this might be a reaction to that 2009 report on the state of forensics in the US. Are similar things happening elsewhere?

 
Posted : 31/10/2014 2:11 am
(@mark_adp)
Posts: 63
Trusted Member
 

As far as I can tell, this 'guidance and advice' follows very closely inline with ISO 17025 requirements on validation and testing of tools and procedures?

 
Posted : 31/10/2014 10:02 am
(@trewmte)
Posts: 1877
Noble Member
 

I can't see a link to the list of names.

But if you mean in the original validation document there are a list of participants on page 102 of the document which was posted by the OP. They appear to relate to the appendix authors.

Thanks dan0841. I did see those names but they aren't attributed to each specific content. It is not clear whether any of the panel used their own company details or used details from others.

Additionally, the content itself is unattributed to foundation stone principles. For instance

As far as I can tell, this 'guidance and advice' follows very closely inline with ISO 17025 requirements on validation and testing of tools and procedures?

"As far as I can tell" - guessing shouldn't be necessary in a government document but actually known that the scope of a particular statement is anchored to a particular principle/clause.

The above are observations only and in fairness, the dictionary term "draft document" is referred to 'as a work in progress'.

 
Posted : 31/10/2014 11:26 am
neddy
(@neddy)
Posts: 182
Estimable Member
 

I do not think that regulation of digital forensics is a bad thing, I welcome it. I do however think that it should become a reality as a result of expertise and innovation by the practitioners and not by forces engaged in the art of politik.

At the moment, it appears that politik is the driver behind the attempts to solidify such an accreditation entity and I am of the opinion that this will deliver a system that is pretty meaningless at great cost.

We work in a very fluid environment and I am sure that we can find a better fit solution to give reassurance to those that require it without reliance on a framework that is best suited to less diverse areas. I am reasonably familiar with ISO17025 and can agree with many of its principles; my problem with it is that it's advocates are easily undermined when asked to justify it's implementation in a laboratory with many diverse methods and the 'naysayer' just as easily undermined when they argue it's unsuitability in said environs because the principle is all we know. This dichotomy seems to indicate that it is a flawed framework for digital forensics and may well go a long way in explaining why very little advancement has been made in its implementation in the last five years.

This is all very well I suppose and we can wait for this life cycle to reach it's zenith or we can encourage all serious digital forensic practitioners to remind themselves that we are the individuals that will leave a legacy for those that follow and that we should do all we can to make it a good one.

So what does all that mean?

I think we need to accept that there is more we can do in digital forensics to raise the bar in terms of quality systems, validation of methods, competency and the application of the fundamentals of scientific endeavor. We should accept that this is our job right now and that it can only be achieved from the ground up.

So what can we do?

I think that every lab should engage by setting up a few simple computers that have diverse operating systems and encourage staff to use, install and abuse them in every possible (legal) way and at the same time log, in a scientific manner, every detail of their activity. Forensic images should be made at regular points and supplied along with the logs to staff to examine, validate and observe the consequences of the activity. Staff should be encouraged in this and I would hope that the information derived from these experiments could help us in getting to a meaningful level of validation that may make the ISO17025 pill easier to swallow.

As with all discussions I have had on ISO17025; I always feel like I have given the impression that I disagree with a part of it only to have then followed up by proposing an alternative that is entirely compatible with that part!

That my friends is how devilish it is!

 
Posted : 01/11/2014 1:31 am
Page 2 / 3
Share: