Computer Forensics ...
 
Notifications
Clear all

Computer Forensics project

15 Posts
8 Users
0 Likes
3,807 Views
(@gumstickstorage)
Posts: 14
Active Member
Topic starter
 

Hello everyone,

Just a quick introduction. I am a third year computer forensics student currently starting off the our dissertations for the year. I was recommended by my supervisor to get on this forum for better communication with digital forensic affiliates. Comically he also stated that I should ensure that my first post be a good post as to show a smart first impression!

The topic I've chose is, in a nutshell, for me to have a look at some guidelines used in digital forensics and determine if any amendments could be made, naturally undertaking practical experiments to see if methods stated in these guidelines are the most efficient, yet most comprehensive way to process evidence, or other tasks (This was heavily inspired by this nice little post here which I found a couple months ago)

So far, I've been looking at some guidelines. The first being everyone's favourite ACPO Good Practice Guide for Digital Evidence, to ones I ended up discovering myself such as ISO/IEC 27037, 27041, 27042 and 27043. I feel by standard, I should have ISO/IEC 17025, which I notice through research and looking in the forums here that it has mixed reviews. I'm also looking at academic literature such as digital forensic frameworks proposed by Beebe and Clark, and Reith et al. I've also been suggested by my supervisor to look at Forensic Science Regulator. There's also a handy sounding book called Digital Forensics Processing and Procedures meeting the requirements of ISO 17020, ISO 17025, ISO 27001 and Best Practice Requirements written by Watson and Jones in 2013. Keep in mind that my dissertation focuses on the UK so I don't think the likes of NIST would be helpful to me, although I would happily take in any knowledge you have of it as general learning.

This is just scratching the surface, as my dissertation progresses, I expect to look at more guidelines and do *something* with those too (compare, maybe analyse, it's all in thought right now).

What I essentially would like is expert opinion. It's all well and good proposing something like this but not get an opinion from those who actually conduct real-world work in the field. What do you think about the guidelines you may follow or have followed? Would you personally like to see changes? Do you think they're OK the way they are? They don't have to be limited to just those guidelines so if you have any in thought, please do mention it below.

Your words will be automatically classed as qualitative data so I'd really appreciate you guys taking the time to state your opinions. )

Thanks for reading.

 
Posted : 29/10/2019 10:06 pm
(@rich2005)
Posts: 535
Honorable Member
 

In a nutshell

ACPO guidelines sensible and practical.

ISO17025 - totally ridiculous for digital forensics and it's push to be mandated on DF in the UK overseen by someone with a total lack of understanding of the complexity of the field (who seems to think digital forensics is akin to plugging samples into a big DNA machine - which funnily enough is her background). A giant waste of time and money that will (in reality - not the ideal world) only lead to a more factory production line method of looking at cases with poorer quality work being produced as a result.

Would I like to see changes? Yes - scrap ISO17025 completely. Appoint someone who understands the complexity of the field and try to design measures to improve the actual quality of digital forensic science. Being ISO17025 accredited doesn't prevent "poor quality forensic science" that they often refer to. It just gives the appearance of quality via a rubber stamp saying the lab is producing work to a quality standard. That work can still be a load of garbage, whether because of the ISO17025 accredited tool not producing accurate results, or because of the poor knowledge/interpretation/decisions of an examiner. Any competent examiner, spending time checking their work, is probably finding far more problems with forensic software, on a regular basis, than the ISO17025 testing regime ever has, or will.

A sensible step would be a standards body for digital forensics, composed almost solely of DF practitioners spending all their time testing tools, on decent "real world" data sets, that is funded solely to test forensic software and hardware. Report any errors found publicly on a website (so that examiners are aware) and report back to the manufacturer for fixing urgently. If penny-pinching, they could probably even introduce a compliance system for major DF software, charge them a small amount of money for this testing, in order to say that tool is compliant with the UKDF testing regime, which aims to detect and fix software bugs quicker.

Of course the issue of a rogue or incompetent examiner isn't alleviated at all really by ISO17025, despite that seemingly being most of the reason behind the push for it. That's a tougher nut to crack and our adversarial court system is a good measure against that (with adequately funded defence work). However the race to the bottom in terms of resources in terms of cutting legal aid (or police budgets) again just makes things worse.

 
Posted : 30/10/2019 3:50 pm
(@gumstickstorage)
Posts: 14
Active Member
Topic starter
 

Thanks for responding Rich.

I get that 17025 is a pretty unpopular set of guidelines due to unrealistic expectations and costs. Unfortunately I can't just write that I want to scrap it but I'd love to take a crack at that.

I'm sure you're aware that ISO goes through a periodic review every five years for any major corrigendum. However, ACPO's guidelines have been published in 2012 and no matter how hard I look, I can't seem to find anything that tells me that their guidelines have gone through some sort of review. Would you say, from your recent experiences, that it still relates to today's real world digital forensic issues?

 
Posted : 30/10/2019 10:15 pm
jaclaz
(@jaclaz)
Posts: 5133
Illustrious Member
 

I get that 17025 is a pretty unpopular set of guidelines due to unrealistic expectations and costs. Unfortunately I can't just write that I want to scrap it but I'd love to take a crack at that.

I see it (and please note, from the outside, not being a professional in the field, and certainly not UK based) as a three different issues
1) costs <- these may only affect small and independent firms, not large laboratories/organization, while it is debatable whether in fact killing from the start small, new laboratories is fair or not is only essentially a political issue
2) expectations <- actually there are no real expectations from the regulator, it simply imposed this (absurd) obligation without any real world means to verify that it is applied properly (since it is impossible to apply it properly and as a matter of fact if you actually try your best to apply it as properly as possible you will end up affecting - negatively - the results of the investigation, please read as either less data obtained properly validated or enormous delays to have the same data validated ). Again this is essentially a political issue, the government or the regulator appointed decided something and the UK professionals cannot but try their best to be compliant, and whether this is fair, intelligent, etc., is out of the scope of a computer forensics dissertation.
3) intellectual honesty <- this is IMHO the worst part, any digital forensics investigator that actually knows where his towel is will know how most of the papers and procedures related to 17025 are either fake, falsified or not actually applied in practice.
The consequence being that the integrity of the investigator is undermined ex legibus.
This is where - maybe - there is some space for making a computer forensics dissertation, making a comparison between a procedure not conforming to ISO 17025 and the same procedure conforming to it, in terms of possibilities, time needed, results obtained. And about the moral compromises needed to comply.

jaclaz

 
Posted : 31/10/2019 9:54 am
(@dcs1094)
Posts: 146
Estimable Member
 

I don't want to make my response all about ISO 17025, because everyone knows its not fit for purpose and the FSR does not have a scooby-doo. I've also been through several assessments where we passed, but from a validation perspective it nothing I've seen is worth the paper its been written on. Funniest thing I found was we were penalised for not having a hoover - are you telling me that effects the quality of digital evidence? Or that they expected me to give a "car-wash" to every tower seized? Yeah thats where I lost respect for it, but anyhow I've gone off topic…

So, go for ACPO guidelines which everyones always adhered to, as they are written by respected practitioners who know what they are talking about. Consider todays challenges compared to 2012. I believe it still refers to floppy disks etc… The amount of occasions where we attended warrants and found servers running in bedrooms, open IRC chats not being logged to disk, crucial artefacts sat in RAM (cryptocurrency private keys, encryption keys, internet evidence currently cached in memory but not written to disk yet), which would be lost if not examined on-site etc, IOT devices, vehicles with on-board infotainment data etc.

I think it would be good to have a revamped ACPO for on-scene and internet of things (IoT). Yes, you cannot account for every eventuality, but a solid baseline to work from. Many of the principles will remain the same as original ACPO guidelines, however more apparent over the years is the fact that alot of the time you will have to make changes to data, in order to extract what is required, but if a certain set of criteria can be created to follow and practitioners are competent at explaining their actions, then happy days. For example, dump RAM from a computer, you will need to plug in a USB or pipe it out over the network, or extract from a phone well you'll likely have to turn the phone on etc. I do not envy those waiting for ISO 17020 to come into play (which is the 17025 for on-site forensics).

 
Posted : 31/10/2019 3:17 pm
(@rich2005)
Posts: 535
Honorable Member
 

Thanks for responding Rich.

I get that 17025 is a pretty unpopular set of guidelines due to unrealistic expectations and costs. Unfortunately I can't just write that I want to scrap it but I'd love to take a crack at that.

I'm sure you're aware that ISO goes through a periodic review every five years for any major corrigendum. However, ACPO's guidelines have been published in 2012 and no matter how hard I look, I can't seem to find anything that tells me that their guidelines have gone through some sort of review. Would you say, from your recent experiences, that it still relates to today's real world digital forensic issues?

As DCS says, the ACPO guidelines were written well enough, that they applied sensible principles/guidelines, and have stood the test of time (with minor tweaks over the years). They most certainly relate to today's issues because of that. They're not an all-encompassing guide (or set of rules) to everything and hence guidelines is a perfect name.

As jaclaz says, the big problem with ISO17025 is it's so unrealistic to do it properly in digital forensics, anyone doing it is basically fudging it (putting it nicely) to give the appearance of compliance. Consider the proper testing of a tool like Axiom, and the hundreds (or thousands) of artefacts it tests for, and all their thousands or tens of thousands of variants/versions for each app/program/file-system/OS/etc, against various real-world data sets. It's just ludicrous to even contemplate. The world political jaclaz used is 100% correct. Great for a DNA machine, with one process, that you update once a year, and can test/validate before going live. That makes sense and absolutely makes sense to have a quality system for. It is quite simply madness to try to apply that to something that changes probably hourly or more and with an endless number of tests/methods and changing shape of target data.

 
Posted : 31/10/2019 6:02 pm
steve862
(@steve862)
Posts: 194
Estimable Member
 

Hi,

I wanted to suggest another angle you might look at.

When we talk about best practice it is usualy on an exhibit by exhibit basis. 'What is the best and most efficient way of getting the data off exhibit A, processing it for review by investigators and how to provide reliable provenance for what is found'. How about looking at what digital forensic providers need to do to continue to meet the demands of the criminal justice system?

Digital forensic units in UK policing face a number of challenges including -

1. Retention of staff, (too many going into corporate work after a number of years)

2. Managing the quantity of devices being seized by officers, (how many do you accept? Will disclosure be a problem if you only examined 3 out of 14 devices?)

3. Managing the quantity of data being collected and stored by digital devices, (more devices, do more things, are being used for more hours each day)

4. Managing the greater speed of change in the technology, (keeping up to date with best practice for newer technologies has never been more challenging)

5. What to do with IoT in the future (how long before Smarthome devices get submitted? What happens when pretty much every thing we use is collecting data about us?)

6. Industrial control systems (cyber attacks are becoming more common and more disabling. Currently LE agecnies do very little of this type of work)

The list could go on.

It might be interesting to play the role of Head of Digital Forensics and ask, how do I keep us operating, meeting the needs of the CJS with at best a modest budget increase each year?

Another angle might be to look at the function of experts who provide services to the defence. Legal Aid funding and the unaffordable costs of accreditation has driven large numbers of experts out of the field. This ties back into stnadards again but it is a legitimate concern for me as a LE practitioner and someone could explore it.

The role of the defence is a very different one from the prosecution. Firstly, the defence will review the prosecution evidence and explain it to their client. They are unlikely to handle the exhibit themselves let alone do any extractions from it but instead work on the images/extractions provided to them. In these two respects what they do and how they work is very different.

One might make the argument that they don't need to be accredited when they might go no further than verifying the prosecution evidence and make judgements about whether the interpretations being presented by the prosecution are valid.

I think if you look at overall methodology you are going to be hard pressed to score well unless you come to the conclusion that ISO17025 is correct for this field. It gets a bit messy if you disagree with the authority that decides what we do in this industry. When you apply for jobs, if it is in LE, then you'll be working to that standard and you'll be expected to say good things about it in the interview.

If you consider specific methodology, the rate of change of technology becomes an issue. Good practice can become bad practice with one incremenet of OS version on the device you are examining. There would also be varieties of best practice depending on how the device was used, what the case type is and so on. Investigative and technical strategies are better when they are tailored towards the investigation.

I don't know if what I've said helps. I'm not saying you should do a report on these other areas but I wanted to provide you with information and options.

Steve

 
Posted : 01/11/2019 6:20 am
(@athulin)
Posts: 1156
Noble Member
 

I get that 17025 is a pretty unpopular set of guidelines due to unrealistic expectations and costs. Unfortunately I can't just write that I want to scrap it but I'd love to take a crack at that.

17025 do not specify 'guidelines'.

ISO 17025 specifies a *framework* for quality management for technical laboratories. It is not directly relevant for computer forensic work, in that it does not say 'what to do' during lab work. It does say that there must be methods used in lab work, and it may say something about how those methods must be developed, formulated, and maintained, but it does not go further than (as far as lab methods are concerned). One certified lab may have a method for a particular test, while another certified lab may not or may have a different one, without there being any kind of contradiction or problem involved, as far as the standard itself goes. For that reason, the standard itself may not be relevant for your project. The book you mentioned (Watson & Jones) may be slightly more appropriate the best would probably be an actual lab's own ISO 17025 implementation, or at least the lab methods specified by it.

 
Posted : 01/11/2019 7:03 am
(@trewmte)
Posts: 1877
Noble Member
 

This is just scratching the surface, as my dissertation progresses, I expect to look at more guidelines and do *something* with those too (compare, maybe analyse, it's all in thought right now).

What I essentially would like is expert opinion. It's all well and good proposing something like this but not get an opinion from those who actually conduct real-world work in the field. What do you think about the guidelines you may follow or have followed? Would you personally like to see changes? Do you think they're OK the way they are? They don't have to be limited to just those guidelines so if you have any in thought, please do mention it below.

It would be useful if your approach was challenging to fixed norms of thinking. For instance, reference to ACPO Guidelines could be seen as absurd given where we are today

a) ACPO doesn't exist, defunct as of 2015, and is now replaced by NPCC
b) ACPO Guidelines were last produced when (what year?). How are the Guidelines relevant to today's tech in 2019, which some tech are only several years old?
c) ACPO Guidelines refers to a principle to make "visible and legible", but there is a missing component which has been well established long before ACPO Guidelines were first produced - what is the missing component?
d) ACPO principles although redundant are still referenced as the backbone, of course, they have been preceded by the FSR codes and iso17025 as being the de facto standards for testing labs (i.e. Digital Forensic Units). Why wouldn't you agree with this? Who validated ACPO Guidelines as de facto Principles?

These are just a few points above, but there are numerous questions today that have been left unanswered, so do check as you said you would what other Guidelines have been produced and run a comparison.

Additionally, consider the positive challenge that Guidelines should be for all, not merely a specific public sector who graciously condone to allows others to follow them if they wish. Make the Guidelines truly global.

 
Posted : 01/11/2019 6:43 pm
(@tootypeg)
Posts: 173
Estimable Member
 

The ACPO points are interesting and whilst there might be 'guidance' provided, often its the 4 principles that stand out. I feel like these could be revisited.

 
Posted : 02/11/2019 12:23 pm
Page 1 / 2
Share: