Some additions with warnings for confidence measures and catches for testing/ validation and competence. -
its just a draft, i can see typos in this
Taking into account the feedback which I have gladly received, this is the completed draft -
Im hoping this captures everything involved in generic decision making when deciding whether to report something. There are markers for measures of confidence and competence at key markers also.
Hi,
I think the latest one looks very good. I wouldn't have been able to start with a blank page and capture all that information in a flow chart in the first place. Capturing decision making in a flow chart is hard to do.
I suppose the only things I might suggest is the suitability of using the term "Report as Fact", as this might be overstating the strength of certain artefacts. I think there will be measures of confidence for which an examiner may report something. When I report on complex artefacts I may add a weight of certainty. This could include the lack of other artefacts indicating an alternative explanation, but that such artefacts might have existed previously. This leads me onto my second suggestion.
This is around the process for coming to the decision of reporting something. I always cite within my report the artefacts from the device(s) I examined, that lead me to those conclusions, along with any artefacts created during my testing. Perhaps I am reading more into it than was intended, but your flow chart seems to indicate the weight of confidence comes from existing documented material rather than the artefacts seen on the exhibit(s).
Traditionally most examiners gained their knowledge of specific artefacts from training courses and then worked out what had changed in 'newer' versions by reverse engineering the data. In both cases they would comment on the artefacts present on the devices and explain their meaning.
In terms of your flow chart's value, I suppose the question is who is it primarily for? For experienced examiners these steps are intuitive and I would expect examiners at this level to follow this sort of decision making without a flowchart as naturally as they would plan a weekly shop and cook a few meals without a flow chart to help.
For new digital examiners coming into the field, is this beyond their expectation of what the work entails? if so, are there digital units where they are not being shown and taught and need this flow chart to show them how they should work?
If it is to provide information to people outside of our field then it does this job particularly well. Whilst it may look complex, because there are so many parts to it, it is actually simple and straightforward to understand.
My suggestions are more intended if this were primarily to be used as a training tool for digital examiners. You are free to accept of reject my suggestions of course, I just make them because you seem to want opinions and suggestions and that you want to make this flow chart as thorough as possible having considered many views and opinions.
I appreciate what you are doing with this flow chart and as I say at the beginning, I couldn't have started with a blank page and put all this down.
Steve
I suppose the only things I might suggest is the suitability of using the term "Report as Fact", as this might be overstating the strength of certain artefacts. I think there will be measures of confidence for which an examiner may report something. When I report on complex artefacts I may add a weight of certainty. This could include the lack of other artefacts indicating an alternative explanation, but that such artefacts might have existed previously. This leads me onto my second suggestion.
Really? I thought we were tasked with producing reports of fact? Whilst I suppose there may be suppositions, would that be reported? So for example, an image in unallocated - would we really report on 'how' that might have got there? The fact that it was once live even for a short time and is now deleted - is that not as far as this goes? Would it not raise 'uncomfortableness' to be speculating about how it got there?
Maybe Ive got the wrong end of the stick - do you have an example of when a confidence measure might be used and what it typically looks like?
Only other thing I can think of as a rubbish example is recovered Internet history after a browser was uninstalled? - I would probably report that the history comes from artefacts belonging to X browser (if I can actually identify this with certainty - making it a fact). If I couldnt identify the browser I would state that its structured as IH but no associated browser can be identified - still a fact. Does that kind of make sense?
This is around the process for coming to the decision of reporting something. I always cite within my report the artefacts from the device(s) I examined, that lead me to those conclusions, along with any artefacts created during my testing. Perhaps I am reading more into it than was intended, but your flow chart seems to indicate the weight of confidence comes from existing documented material rather than the artefacts seen on the exhibit(s).
I am hoping that the practitioner comes with their inferences from the investigation and then the framework helps them to establish valid conclusions in 1 of 3 ways. Reference to past case precedents, testing and published material.
Traditionally most examiners gained their knowledge of specific artefacts from training courses and then worked out what had changed in 'newer' versions by reverse engineering the data. In both cases they would comment on the artefacts present on the devices and explain their meaning.
Ah yes I had missed training, thanks! when a practitioner works additional content out, this would be the 'testing' route i would say.
In terms of your flow chart's value, I suppose the question is who is it primarily for?
Good point, I suppose the answer is anyone that it would support. I agree, experienced examiners should do this already. I am not brave enough to say that they all do. I suspect that lesser experienced individuals may get the most from it. Maybe even labs as part of quality management.
Just trying to help I guess
And thanks for your detailed input its very helpful
Only other thing I can think of as a rubbish example is recovered Internet history after a browser was uninstalled? - I would probably report that the history comes from artefacts belonging to X browser (if I can actually identify this with certainty - making it a fact). If I couldnt identify the browser I would state that its structured as IH but no associated browser can be identified - still a fact. Does that kind of make sense?
The issue here is what is a fact. So yes you have recovered data which is consistent with the data produced by browser X when a user accesses websites.
However, I could fabricate the same data manually. Therefore this data exists but doesn't represent user activity.
The issue with digital is that everything, from the file-system to user data, is an interpretation of a series of 1's and 0's.In theory, if I created a truly random generation of bits, I could eventually create and Indecent Images in JPEG format for example. Being extremely pedantic, you could state that you located data which can be interpreted as a picture file.
It's this issue that causes a lot of difficulty in Digital Investigations as opposed to wet forensics where DNA and fingerprints physically exist, nothing really exists and it is just an interpretation.
Only other thing I can think of as a rubbish example is recovered Internet history after a browser was uninstalled? - I would probably report that the history comes from artefacts belonging to X browser (if I can actually identify this with certainty - making it a fact). If I couldnt identify the browser I would state that its structured as IH but no associated browser can be identified - still a fact. Does that kind of make sense?
The issue here is what is a fact. So yes you have recovered data which is consistent with the data produced by browser X when a user accesses websites.
However, I could fabricate the same data manually. Therefore this data exists but doesn't represent user activity.
The issue with digital is that everything, from the file-system to user data, is an interpretation of a series of 1's and 0's.In theory, if I created a truly random generation of bits, I could eventually create and Indecent Images in JPEG format for example. Being extremely pedantic, you could state that you located data which can be interpreted as a picture file.
It's this issue that causes a lot of difficulty in Digital Investigations as opposed to wet forensics where DNA and fingerprints physically exist, nothing really exists and it is just an interpretation.
I see your point and totally get the idea of random data capable of formatting something legit. I guess there are levels of fact.
Fact Level 1- Data appears to be structured as IH.
Fact Level 2- Data cannot be 100% attributed to a browser
Fact Level 3- Data is in a format that is comparable to browser X despite X not being on the systemwhen does it not become a fact……'I believe IH to be attributed to X', 'IH appears to belong to X'
…back to a fact …..' IH is formatted in a way consistent of X's storage of IH. (providing it has been tested and validated properly etc)
I think you can factually state limitations in analysis, providing they are worded properly. Its wording like ' i believe', ' i consider', 'they seem' etc that to me, would raise concerns of opinion.
Just my thoughts, could be rubbish
The issue here is what is a fact. So yes you have recovered data which is consistent with the data produced by browser X when a user accesses websites.
However, I could fabricate the same data manually. Therefore this data exists but doesn't represent user activity.
The issue with digital is that everything, from the file-system to user data, is an interpretation of a series of 1's and 0's.In theory, if I created a truly random generation of bits, I could eventually create and Indecent Images in JPEG format for example. Being extremely pedantic, you could state that you located data which can be interpreted as a picture file.
Though - honestly - when it comes to a complex format such as a JPEG it is improbable that it was "randomly" generated.
I find much more likely (again in edge cases, but still more likely) that reconstruction of text files carved from unallocated may cause a "spontaneous" (and "bogus", but readable) text to be generated.
Still, IMHO adding a reasonable evaluation/description of the possibilities that could have lead to the creation of an artifact is important, as a matter of fact it is vital that the experience of the "expert witness" is *somehow* expressed[1], particularly regarding three main points
1) how technically (and logically) an artifact may have been generated
2) how likely it is that the artifact may have been generated involuntarily or by automated means without the knowledge of the user
3) how well the artifacts found on an examined device (as a whole) fit a (again technically and logically) possible scenario
All in all we are back to the base concept of a "full timeline" and placing the findings (wherever possible) in their context.
Without the experience and knowledge of a human expert, we would be back to the issue about one button forensics, which again can be a good triage method, but nothing more than that.
And now as a side-side note (and I understand it is not a common-common case, but I suspect it will become more common) there could be an added provision somewhere in the flowchart related to language proficiency 😯 of the examiner with the language[2] used on the device and by the user.
There have been more than a few cases lately in Italy (actually AFAICR related to telephone interceptions, but essentially the matter is not so different) where the misinterpretation or mistranslation of something said in either a foreign language or a dialect or a slang of some kind has led to investigating errors.
jaclaz
[1] as long as it is clearly separated from the actual "fact" reporting, and clearly designated as an opinion
[2] as sometimes a same sentence may be read getting a wrong meaning, a re-known example being "Edwardum occidere nolite timere bonum est"
Hi,
In relation to measures of confidence in artefacts there are many possible examples. A simple one is a Windows registry, which may report the date of installation, but it isn't a fact it was installed on that date. Other than the clock may not have always been correct the OS may have been installed onto that disk but in another PC and then transferred over. Or it could be a clone of another disk, either created in that computer or in another computer (and so on…).
I might say that program A creates these artefacts in this folder with these properties but it doesn't rule out the possibility they were created by another program or process and then placed there. The likelihood of such an occurrence is likely to be very small. It will depend on the program and the possibility the program allows or used to allow the syncing or importing of certain files, which might have been a user or automated action and for which different options might have been available to apply.
If we're talking about a red herring being created by a user then it might be a user who works in IT and clearly displays the requisite knowledge based on what else they've done on their devices, is more capable of doing so than a user who barely knows how to use MS Word.
I'd also be want to be a little cautious around artefacts which might have been impacted by privacy and security programs, (or settings), anti-virus software or even significant changes in the OS.
I'm not saying I never report things as fact or that my reports contain nothing I can nail down, but reporting something as fact invites the opposing barrister to ask the question "is there no other possible way or means this artefact……..?".
The 'is it possible….?' question does come up in cross examination from time to time. The answer may be 'Yes', a strong 'Yes but……'.
It is possible I could win the lottery jackpot every Wednesday and Saturday for the next month from a single ticket each draw, but is that actually going to happen? Yes everything is possible.
But the artefacts present, the lack of certain other artefacts and a number of other considerations explained in the report leads one to the conclusion that the only reasonable explanation is the one being proposed, but that's not quite the same as 'fact' in my book.
What you're doing is a good thing and I think someone should be capturing the process of an examination in terms of decision making. Nobody else has come forward and offered something for the community to look at and comment on, including me.
In some ways this thread has begun to encompass a discussion on how too much data, inadequate budgets and other factors are resulting in an increase in a move away from examiners providing detailed examinations and interpretation.
Perhaps a companion flow chart for this one would be one based on competency. Defining what training, certification and experience qualifies the examiner to make the determinations they make.
Steve
In relation to measures of confidence in artefacts there are many possible examples. A simple one is a Windows registry, which may report the date of installation, but it isn't a fact it was installed on that date. Other than the clock may not have always been correct the OS may have been installed onto that disk but in another PC and then transferred over. Or it could be a clone of another disk, either created in that computer or in another computer (and so on…).
I might say that program A creates these artefacts in this folder with these properties but it doesn't rule out the possibility they were created by another program or process and then placed there. The likelihood of such an occurrence is likely to be very small. It will depend on the program and the possibility the program allows or used to allow the syncing or importing of certain files, which might have been a user or automated action and for which different options might have been available to apply.
If we're talking about a red herring being created by a user then it might be a user who works in IT and clearly displays the requisite knowledge based on what else they've done on their devices, is more capable of doing so than a user who barely knows how to use MS Word.
I'd also be want to be a little cautious around artefacts which might have been impacted by privacy and security programs, (or settings), anti-virus software or even significant changes in the OS.
Three useful examples above about program generated artefacts that reminds me how clarity in reporting is important. Should the author of the report/flowchart define what s/he means by the word artefact/artifact when making a statement using the word fact associated with it?
steve862 your comments remind me how the word 'artefact' might be defined. In Barbara Ann Kipfer 2007 book 'Dictionary of Artifacts' (defining terms for archaeology), Barbara refers to artifacts generated due to human agency
"artifact any object (article, building, container, device, dwelling, ornament, pottery, tool, weapon, work of art) made, affected, used, or modified in some way by human beings." ….[artefact].
and
"artifact type a description of a category of artifacts that share a set of somewhat variable attributes…" ….[artefact type].