Join Us!

Notifications
Clear all

Computer Forensics project  

  RSS
GumStickStorage
(@gumstickstorage)
New Member

Hello everyone,

Just a quick introduction. I am a third year computer forensics student currently starting off the our dissertations for the year. I was recommended by my supervisor to get on this forum for better communication with digital forensic affiliates. Comically he also stated that I should ensure that my first post be a good post as to show a smart first impression!

The topic I've chose is, in a nutshell, for me to have a look at some guidelines used in digital forensics and determine if any amendments could be made, naturally undertaking practical experiments to see if methods stated in these guidelines are the most efficient, yet most comprehensive way to process evidence, or other tasks (This was heavily inspired by this nice little post here which I found a couple months ago)

So far, I've been looking at some guidelines. The first being everyone's favourite ACPO Good Practice Guide for Digital Evidence, to ones I ended up discovering myself such as ISO/IEC 27037, 27041, 27042 and 27043. I feel by standard, I should have ISO/IEC 17025, which I notice through research and looking in the forums here that it has mixed reviews. I'm also looking at academic literature such as digital forensic frameworks proposed by Beebe and Clark, and Reith et al. I've also been suggested by my supervisor to look at Forensic Science Regulator. There's also a handy sounding book called Digital Forensics Processing and Procedures meeting the requirements of ISO 17020, ISO 17025, ISO 27001 and Best Practice Requirements written by Watson and Jones in 2013. Keep in mind that my dissertation focuses on the UK so I don't think the likes of NIST would be helpful to me, although I would happily take in any knowledge you have of it as general learning.

This is just scratching the surface, as my dissertation progresses, I expect to look at more guidelines and do *something* with those too (compare, maybe analyse, it's all in thought right now).

What I essentially would like is expert opinion. It's all well and good proposing something like this but not get an opinion from those who actually conduct real-world work in the field. What do you think about the guidelines you may follow or have followed? Would you personally like to see changes? Do you think they're OK the way they are? They don't have to be limited to just those guidelines so if you have any in thought, please do mention it below.

Your words will be automatically classed as qualitative data so I'd really appreciate you guys taking the time to state your opinions. )

Thanks for reading.

Quote
Posted : 29/10/2019 10:06 pm
Rich2005
(@rich2005)
Senior Member

In a nutshell

ACPO guidelines sensible and practical.

ISO17025 - totally ridiculous for digital forensics and it's push to be mandated on DF in the UK overseen by someone with a total lack of understanding of the complexity of the field (who seems to think digital forensics is akin to plugging samples into a big DNA machine - which funnily enough is her background). A giant waste of time and money that will (in reality - not the ideal world) only lead to a more factory production line method of looking at cases with poorer quality work being produced as a result.

Would I like to see changes? Yes - scrap ISO17025 completely. Appoint someone who understands the complexity of the field and try to design measures to improve the actual quality of digital forensic science. Being ISO17025 accredited doesn't prevent "poor quality forensic science" that they often refer to. It just gives the appearance of quality via a rubber stamp saying the lab is producing work to a quality standard. That work can still be a load of garbage, whether because of the ISO17025 accredited tool not producing accurate results, or because of the poor knowledge/interpretation/decisions of an examiner. Any competent examiner, spending time checking their work, is probably finding far more problems with forensic software, on a regular basis, than the ISO17025 testing regime ever has, or will.

A sensible step would be a standards body for digital forensics, composed almost solely of DF practitioners spending all their time testing tools, on decent "real world" data sets, that is funded solely to test forensic software and hardware. Report any errors found publicly on a website (so that examiners are aware) and report back to the manufacturer for fixing urgently. If penny-pinching, they could probably even introduce a compliance system for major DF software, charge them a small amount of money for this testing, in order to say that tool is compliant with the UKDF testing regime, which aims to detect and fix software bugs quicker.

Of course the issue of a rogue or incompetent examiner isn't alleviated at all really by ISO17025, despite that seemingly being most of the reason behind the push for it. That's a tougher nut to crack and our adversarial court system is a good measure against that (with adequately funded defence work). However the race to the bottom in terms of resources in terms of cutting legal aid (or police budgets) again just makes things worse.

ReplyQuote
Posted : 30/10/2019 3:50 pm
GumStickStorage
(@gumstickstorage)
New Member

Thanks for responding Rich.

I get that 17025 is a pretty unpopular set of guidelines due to unrealistic expectations and costs. Unfortunately I can't just write that I want to scrap it but I'd love to take a crack at that.

I'm sure you're aware that ISO goes through a periodic review every five years for any major corrigendum. However, ACPO's guidelines have been published in 2012 and no matter how hard I look, I can't seem to find anything that tells me that their guidelines have gone through some sort of review. Would you say, from your recent experiences, that it still relates to today's real world digital forensic issues?

ReplyQuote
Posted : 30/10/2019 10:15 pm
jaclaz
(@jaclaz)
Community Legend

I get that 17025 is a pretty unpopular set of guidelines due to unrealistic expectations and costs. Unfortunately I can't just write that I want to scrap it but I'd love to take a crack at that.

I see it (and please note, from the outside, not being a professional in the field, and certainly not UK based) as a three different issues
1) costs <- these may only affect small and independent firms, not large laboratories/organization, while it is debatable whether in fact killing from the start small, new laboratories is fair or not is only essentially a political issue
2) expectations <- actually there are no real expectations from the regulator, it simply imposed this (absurd) obligation without any real world means to verify that it is applied properly (since it is impossible to apply it properly and as a matter of fact if you actually try your best to apply it as properly as possible you will end up affecting - negatively - the results of the investigation, please read as either less data obtained properly validated or enormous delays to have the same data validated ). Again this is essentially a political issue, the government or the regulator appointed decided something and the UK professionals cannot but try their best to be compliant, and whether this is fair, intelligent, etc., is out of the scope of a computer forensics dissertation.
3) intellectual honesty <- this is IMHO the worst part, any digital forensics investigator that actually knows where his towel is will know how most of the papers and procedures related to 17025 are either fake, falsified or not actually applied in practice.
The consequence being that the integrity of the investigator is undermined ex legibus.
This is where - maybe - there is some space for making a computer forensics dissertation, making a comparison between a procedure not conforming to ISO 17025 and the same procedure conforming to it, in terms of possibilities, time needed, results obtained. And about the moral compromises needed to comply.

jaclaz

ReplyQuote
Posted : 31/10/2019 9:54 am
DCS1094
(@dcs1094)
Active Member

I don't want to make my response all about ISO 17025, because everyone knows its not fit for purpose and the FSR does not have a scooby-doo. I've also been through several assessments where we passed, but from a validation perspective it nothing I've seen is worth the paper its been written on. Funniest thing I found was we were penalised for not having a hoover - are you telling me that effects the quality of digital evidence? Or that they expected me to give a "car-wash" to every tower seized? Yeah thats where I lost respect for it, but anyhow I've gone off topic…

So, go for ACPO guidelines which everyones always adhered to, as they are written by respected practitioners who know what they are talking about. Consider todays challenges compared to 2012. I believe it still refers to floppy disks etc… The amount of occasions where we attended warrants and found servers running in bedrooms, open IRC chats not being logged to disk, crucial artefacts sat in RAM (cryptocurrency private keys, encryption keys, internet evidence currently cached in memory but not written to disk yet), which would be lost if not examined on-site etc, IOT devices, vehicles with on-board infotainment data etc.

I think it would be good to have a revamped ACPO for on-scene and internet of things (IoT). Yes, you cannot account for every eventuality, but a solid baseline to work from. Many of the principles will remain the same as original ACPO guidelines, however more apparent over the years is the fact that alot of the time you will have to make changes to data, in order to extract what is required, but if a certain set of criteria can be created to follow and practitioners are competent at explaining their actions, then happy days. For example, dump RAM from a computer, you will need to plug in a USB or pipe it out over the network, or extract from a phone well you'll likely have to turn the phone on etc. I do not envy those waiting for ISO 17020 to come into play (which is the 17025 for on-site forensics).

ReplyQuote
Posted : 31/10/2019 3:17 pm
Rich2005
(@rich2005)
Senior Member

Thanks for responding Rich.

I get that 17025 is a pretty unpopular set of guidelines due to unrealistic expectations and costs. Unfortunately I can't just write that I want to scrap it but I'd love to take a crack at that.

I'm sure you're aware that ISO goes through a periodic review every five years for any major corrigendum. However, ACPO's guidelines have been published in 2012 and no matter how hard I look, I can't seem to find anything that tells me that their guidelines have gone through some sort of review. Would you say, from your recent experiences, that it still relates to today's real world digital forensic issues?

As DCS says, the ACPO guidelines were written well enough, that they applied sensible principles/guidelines, and have stood the test of time (with minor tweaks over the years). They most certainly relate to today's issues because of that. They're not an all-encompassing guide (or set of rules) to everything and hence guidelines is a perfect name.

As jaclaz says, the big problem with ISO17025 is it's so unrealistic to do it properly in digital forensics, anyone doing it is basically fudging it (putting it nicely) to give the appearance of compliance. Consider the proper testing of a tool like Axiom, and the hundreds (or thousands) of artefacts it tests for, and all their thousands or tens of thousands of variants/versions for each app/program/file-system/OS/etc, against various real-world data sets. It's just ludicrous to even contemplate. The world political jaclaz used is 100% correct. Great for a DNA machine, with one process, that you update once a year, and can test/validate before going live. That makes sense and absolutely makes sense to have a quality system for. It is quite simply madness to try to apply that to something that changes probably hourly or more and with an endless number of tests/methods and changing shape of target data.

ReplyQuote
Posted : 31/10/2019 6:02 pm
steve862
(@steve862)
Active Member

Hi,

I wanted to suggest another angle you might look at.

When we talk about best practice it is usualy on an exhibit by exhibit basis. 'What is the best and most efficient way of getting the data off exhibit A, processing it for review by investigators and how to provide reliable provenance for what is found'. How about looking at what digital forensic providers need to do to continue to meet the demands of the criminal justice system?

Digital forensic units in UK policing face a number of challenges including -

1. Retention of staff, (too many going into corporate work after a number of years)

2. Managing the quantity of devices being seized by officers, (how many do you accept? Will disclosure be a problem if you only examined 3 out of 14 devices?)

3. Managing the quantity of data being collected and stored by digital devices, (more devices, do more things, are being used for more hours each day)

4. Managing the greater speed of change in the technology, (keeping up to date with best practice for newer technologies has never been more challenging)

5. What to do with IoT in the future (how long before Smarthome devices get submitted? What happens when pretty much every thing we use is collecting data about us?)

6. Industrial control systems (cyber attacks are becoming more common and more disabling. Currently LE agecnies do very little of this type of work)

The list could go on.

It might be interesting to play the role of Head of Digital Forensics and ask, how do I keep us operating, meeting the needs of the CJS with at best a modest budget increase each year?

Another angle might be to look at the function of experts who provide services to the defence. Legal Aid funding and the unaffordable costs of accreditation has driven large numbers of experts out of the field. This ties back into stnadards again but it is a legitimate concern for me as a LE practitioner and someone could explore it.

The role of the defence is a very different one from the prosecution. Firstly, the defence will review the prosecution evidence and explain it to their client. They are unlikely to handle the exhibit themselves let alone do any extractions from it but instead work on the images/extractions provided to them. In these two respects what they do and how they work is very different.

One might make the argument that they don't need to be accredited when they might go no further than verifying the prosecution evidence and make judgements about whether the interpretations being presented by the prosecution are valid.

I think if you look at overall methodology you are going to be hard pressed to score well unless you come to the conclusion that ISO17025 is correct for this field. It gets a bit messy if you disagree with the authority that decides what we do in this industry. When you apply for jobs, if it is in LE, then you'll be working to that standard and you'll be expected to say good things about it in the interview.

If you consider specific methodology, the rate of change of technology becomes an issue. Good practice can become bad practice with one incremenet of OS version on the device you are examining. There would also be varieties of best practice depending on how the device was used, what the case type is and so on. Investigative and technical strategies are better when they are tailored towards the investigation.

I don't know if what I've said helps. I'm not saying you should do a report on these other areas but I wanted to provide you with information and options.

Steve

ReplyQuote
Posted : 01/11/2019 6:20 am
athulin
(@athulin)
Community Legend

I get that 17025 is a pretty unpopular set of guidelines due to unrealistic expectations and costs. Unfortunately I can't just write that I want to scrap it but I'd love to take a crack at that.

17025 do not specify 'guidelines'.

ISO 17025 specifies a *framework* for quality management for technical laboratories. It is not directly relevant for computer forensic work, in that it does not say 'what to do' during lab work. It does say that there must be methods used in lab work, and it may say something about how those methods must be developed, formulated, and maintained, but it does not go further than (as far as lab methods are concerned). One certified lab may have a method for a particular test, while another certified lab may not or may have a different one, without there being any kind of contradiction or problem involved, as far as the standard itself goes. For that reason, the standard itself may not be relevant for your project. The book you mentioned (Watson & Jones) may be slightly more appropriate the best would probably be an actual lab's own ISO 17025 implementation, or at least the lab methods specified by it.

ReplyQuote
Posted : 01/11/2019 7:03 am
trewmte
(@trewmte)
Community Legend

This is just scratching the surface, as my dissertation progresses, I expect to look at more guidelines and do *something* with those too (compare, maybe analyse, it's all in thought right now).

What I essentially would like is expert opinion. It's all well and good proposing something like this but not get an opinion from those who actually conduct real-world work in the field. What do you think about the guidelines you may follow or have followed? Would you personally like to see changes? Do you think they're OK the way they are? They don't have to be limited to just those guidelines so if you have any in thought, please do mention it below.

It would be useful if your approach was challenging to fixed norms of thinking. For instance, reference to ACPO Guidelines could be seen as absurd given where we are today

a) ACPO doesn't exist, defunct as of 2015, and is now replaced by NPCC
b) ACPO Guidelines were last produced when (what year?). How are the Guidelines relevant to today's tech in 2019, which some tech are only several years old?
c) ACPO Guidelines refers to a principle to make "visible and legible", but there is a missing component which has been well established long before ACPO Guidelines were first produced - what is the missing component?
d) ACPO principles although redundant are still referenced as the backbone, of course, they have been preceded by the FSR codes and iso17025 as being the de facto standards for testing labs (i.e. Digital Forensic Units). Why wouldn't you agree with this? Who validated ACPO Guidelines as de facto Principles?

These are just a few points above, but there are numerous questions today that have been left unanswered, so do check as you said you would what other Guidelines have been produced and run a comparison.

Additionally, consider the positive challenge that Guidelines should be for all, not merely a specific public sector who graciously condone to allows others to follow them if they wish. Make the Guidelines truly global.

ReplyQuote
Posted : 01/11/2019 6:43 pm
tootypeg
(@tootypeg)
Active Member

The ACPO points are interesting and whilst there might be 'guidance' provided, often its the 4 principles that stand out. I feel like these could be revisited.

ReplyQuote
Posted : 02/11/2019 12:23 pm
GumStickStorage
(@gumstickstorage)
New Member

Thanks for all your responses, opinions, corrections, and suggestions. This is some pretty overwhelming, yet very valuable information I'm getting.

This forum post will (if allowed) definitely be part of my literature review and other pieces of literature to go with it. The suggestions and methodologies I've been reading will be stored and used if appropriate. It would be nice if I could attempt to even visit a digital forensics lab in action but I think I'd be pushing it by attempting to get that.

Either way, this thread is still being actively read (until January when I will most likely start the main bulk of the report) so any other comments will be greatly valued.

ReplyQuote
Posted : 03/11/2019 3:03 pm
GumStickStorage
(@gumstickstorage)
New Member

The ACPO points are interesting and whilst there might be 'guidance' provided, often its the 4 principles that stand out. I feel like these could be revisited.

Thanks for your response tootypeg.

Just by looking at this thread, the ACPO guidelines seem to be popular and are happy the way it is. Do you have any reasons why you think it should be revisited?

ReplyQuote
Posted : 04/11/2019 9:25 pm
jaclaz
(@jaclaz)
Community Legend

Just by looking at this thread, the ACPO guidelines seem to be popular and are happy the way it is. Do you have any reasons why you think it should be revisited?

IMHO, being "principles" they are very good (please read as making a lot of sense) and "universal"

ACPO Principle 1 That no action take is taken that should change data held on a digital device including a computer or mobile phone that may subsequently be relied upon as evidence in court.

ACPO Principle 2 Where a person finds it necessary to access original data held on a digital device that the person must be competent to do so and able to explain their actions and the implications of those actions on the digital evidence to a Court.

ACPO Principle 3 That an trail or record of all actions taken that have been applied to the digital evidence should be created and preserved. An independent third party forensic expert should be able to examine those processes and reach the same conclusion.

ACPO Principle 4 That the individual in charge of the investigation has overall responsibility to ensure that these principles are followed.

The issues are (still IMHO)
ACPO principle #1 is not (anymore) applicable in all cases (when it comes to phones or encrypted devices) because in some occasions data is actually modified by the method used to access the data, and on this there are different points of view (personally I believe that documented, motivated and *needed* changes to data are not an issue).

ACPO principle #2 is not (anymore) applied in a number of cases (search for "push button forensics" for some takes on the matter).

ACPO principle #3 is not anymore applicable due (mainly) to the issues seen above, and this is also intertwined to the validation (or actually utter lack of it) of tools mandated by - besides "common sense" - ISO 17025.

ACPO principle #4 remains applicable, but is undermined by the (IMHO common) viloations of principles #2 and #3.

jaclaz

ReplyQuote
Posted : 05/11/2019 11:29 am
Rich2005
(@rich2005)
Senior Member

I think it depends on your point of view of the term "principle".

If we think about the common usage of the word principle then I think it helps.

So point one is essentially saying "In principle we should not modify the original data". Nothing wrong with that in my mind.

The second principle goes on to say that "In circumstances where a person finds it necessary to access original data, that person must be competent to do so and be able to give evidence explaining the relevance and the implications of their actions." Again that's perfectly sensible in my view and is really dealing with your concern over principle one. In that, generally speaking, you should seek to avoid accessing/modifying original data, but if it's necessary, the person should be competent enough to know and explain what they did, and why it was necessary. I think this would cover lots of things - mobile extractions probably being the most common example.

Principle three is still applicable in my view (in the broadest sense). However I would remove the final sentence, especially with respect to collection of data that cannot be repeated at a later stage, and this was probably written mostly with something like disk-based evidence, rather than a collection of data that may not be repeatable later. I'd also focus this principle more on documentation of interaction with the source evidence and properly documenting findings. Ie there could be circumstances where, as an example, someone could use a proprietary "black box" piece of software to locate data, and justifiably not reveal the method, or not know it, but be example to point to the raw data on the disk, it's location/details, etc.

Principle 4 - I understand why it was written, and generally is little more than a heads-up, that the "officer in the case" is essentially in charge of what happens, in law enforcement matters. However in practice they're not going to be sat on the shoulder of an examiner directing them or in a position to ensure "the law and these principles are adhered to". This could be rewritten to reflect more that an examiner should consider all legal and evidentary implications of actions they intend to take and, where necessary, consult the person in charge of the investigation for authorisation to proceed with a course of action, particularly in circumstances where data may be being modified or accessed outside the scope of a physically seized item. Or something like that 😉

ReplyQuote
Posted : 05/11/2019 12:41 pm
jaclaz
(@jaclaz)
Community Legend

In that, generally speaking, you should seek to avoid accessing/modifying original data, but if it's necessary, the person should be competent enough to know and explain what they did, and why it was necessary. I think this would cover lots of things - mobile extractions probably being the most common example.

Yep ) , but in this I am seemingly a little stricter in my interpretation, in the sense that even if no data is actually modified, the person should be competent enough anyway, i.e. not a "button pusher".

Anyway, the whole point I was trying to make still revolves not on the (undoubtable) validity of "principles", I don't think anyone can seriously be against any of them in theory, the issues are only about how they are put (or often actually not put) in practice.

Which is not entirely unlike the majority of critiques on ISO 17025, there is nothing actually "wrong" in their provisions, the issue comes when/where yje norm is applied to digital forensics and with the actual ways it is put in practice by real world labs/investigators on real world cases.

jaclaz

ReplyQuote
Posted : 05/11/2019 1:55 pm
Share: