Digital Evidence Admissibility – Exploring Best Practice And Compliance Frameworks

Si: Away we go. So, friends and enemies, welcome to the Forensic Focus Podcast. We are here today with Ryan and Robert, and we are going to be talking, I believe, about standards and standards in forensics … So, guys, do you want to introduce yourselves, and we’ll kick off the conversation and complain massively about the implementation of standards in forensics? Because I know it’s a bug bear for me, and I’m pretty sure it’s gonna be a bug bear for you as well!

Ryan: No doubt. Rob, go ahead.

Rob: Yeah, so, Rob Fried, I am Senior Vice President and Global Head of Forensics Investigations for a firm known as Sandline Global. We do eDiscovery services. I was brought on board about two years ago to lead the forensics practice, where we since built up the lab and the practice, in terms of resources and the team members, and all is great. Background is I’ve been doing this over 21 years now. Started off training law enforcement at National White Collar Crime Center, also known as NW3C, and I’ve been in professional forensic consulting since.

Ryan: Yeah. And Ryan Parthemore. My job here today is to make Rob look old, because I was actually one of his students at the National White Collar Crime Center in lovely Fairmont, West Virginia. And came from law enforcement. About 20 years in law enforcement. Heavily into digital forensics because I was the IT geek. So when you put a badge on an IT geek, you send them to digital forensic schools like NW3C and others, and they handle all of that digital forensics stuff that nobody else understands, but nonetheless needs. And then I came to Cellebrite.

And Cellebrite’s where I am today. I have an interesting title as Product Evangelist, which I like to explain as basically taking everything that the field is telling us that we’re doing right as well as even more so, everything we’re doing wrong, and steering that to product management. And then when product management comes out and says, “we got this right”, I then take that information back to the customer and say, “look, we listened and we delivered. Here’s how you can put this to work for you.” Been at Cellebrite now for two years and loving it.

Si: Nice. Fantastic. I have to say the ”I used to be in law enforcement and I knew how a computer works” is a remarkably common career path, it seems into forensics. The amount of people who I’ve met who have justified this as…I was talking to a to a guy, a police officer the other day, and he was like, “I was so bad at writing up my statements that I bought a computer just so I could type them up”, and then people were like, “oh, you know about computers, here, you are a forensic expert”. And he’s now got a position lecturing in the university.

So, obviously it all pans out in an amazing way! So yeah, no, that’s really cool. And I do love the title of Product Evangelist. I think that’s such a fantastic thing. I want to be an evangelist for something one day, I think.

Ryan: The nice thing is it’s not clearly defined, so really anything that you want to get your hands into you can sort of loosely put it under the umbrella of evangelism.

Si: Excellent. Good stuff. So, I mean, we had penciled in for this, and you guys came to us with the suggestion of talking about standards. We’ve got some serious grief with standards in the UK at the moment, thanks to our forensic science regulator. But tell me a bit about why you guys wanted to bring that to us. Where are you coming from on this point?

Rob: Yeah, I’ll start off with that. So, my academic background is that I went to University of New Haven and studied forensic science. Got an undergrad and then a master’s in it. I don’t have law enforcement background, although I trained police officers for almost three years. So, you know, a lot of my studies and also exposure in my first few years of being involved in the industry was around ex, you know, military law enforcement at all different levels: federal, state, local.

And so that’s kind of what I, you know, was starting to learn and hear about all the different things going on in the different police departments. Also, at the same time, the RCFLs when I was at NW3C started to pop up all across the country. And those are the FBI, you know, labs where they have individuals working on digital based evidence. So, my exposure has always been to, you know, law enforcement standards, how things work in a lab, and then now transitioning into the private sector, you know, using my background as a forensic scientist to influence and also try to bring some of those principles and standards that that I’ve learned from day one into the work that we’re doing every day.

Si: So, as a forensic scientist, is that (and forgive me if I’m using terms that perhaps aren’t properly defined yet), but is that meatspace and wetwear kind of forensic science as opposed to digital?

Rob: Correct. Yeah. It’s anything that involves crime scenes and, you know, evidence collection, evidence examination beyond, you know, just looking at it specifically in targeting forensics.

Si: Cool.

Ryan: Yeah, and my background’s somewhat different, but dovetails in directly. I was tasked with starting a digital forensics laboratory for a district attorney’s office in Pennsylvania back in 2007. And then it was just all about what you could beg, borrow or steal in order to get the data off the phone. We used tools like BitPim. We used tools that were really designed by telephone handset manufacturers to back up data. In fact, if you go to the origins of Cellebrite, that’s where Cellebrite started, was the ability to move your data from one device to your next phone. And then the laboratory, this was a multidisciplinary laboratory, so it wasn’t just digital forensics they’re doing, you know, blood and biology, chemistry, all of these more well established, I would say, forensic sciences, more tenured forensic sciences.

And then here I am in digital forensics trying to put the square peg into the round hole and look at digital forensic science through that lens of the more tenured forensic sciences and apply it to things like ISO 17025. And that compliance framework was a nightmare for me initially. But coming out on the other side, what I realized is through these compliance standards, it actually made my life a lot easier. Now, not upfront, because there are a lot of hoops to jump through, but once you get your system in place, what you end up with is a highly consistent outcome.

And I felt so much better when I would pull a case folder and head to court to take the stand and testify to a case that I did using our compliance framework than on a case that I did before. And I don’t know if everyone’s freely going to admit this, but I can tell you that it scared the hell out of me when I needed to take the stand, because I felt like the weight of the entire case was on my shoulders, and I was going to forget to say something, I was going to mischaracterize in some way. I was going to screw the case up. And even though I had that prep with, you know, the prosecutor on my direct testimony, I’m also worried  (more so, in fact), what the defense is going to ask me. I’m going to look stupid. I’m not going to have the answer, and it’s just going to jeopardize everything.

And what I found was that through the application of these established forensic science standards, like, you know, ISO in its various forms (a couple of different ones come to mind), but when we apply those, and we do this consistently, we don’t have that fear anymore. We can take the stand and we know that there’s not a question they can throw at us (that’s relevant anyway), that we won’t have the answer to.

Desi: So, I’ve got a question. So, I come from a background that’s not law enforcement or digital forensics, more from an incidence response background. So, when these labs are being set up and you’re following that standard to set up your system to kind of have the data go through all the way until it’s hitting court, is there sometimes room for interpretation and then maybe different labs are using different standards? Like, how does that fit in…? Like, is it cohesive, I guess is what I’m asking? Like, so if you are on the other side where you are questioning the evidence that’s being presented in court, is that something to take into consideration, like what the other lab is doing?

Rob: Yeah, I mean, I can say that I’ve worked in several labs and, you know, labs that are also multinational, based on the company I’ve worked for. And there is a push within an organization to have everybody using the same standards across the organization. But in the different companies and firms that I’ve been a part of, we’ve always done things a little bit different than, you know, the other firm that I was at previously.

And, you know, I’ve had the opportunity now to set up a lab and do that from scratch. And it was important for me to incorporate a lot of the standards and principles because I think Ryan’s exactly right that you’re on the stand, it’s you, nobody else is there. You don’t get a phone a friend, a lifeline in any way, and you’re being asked questions, and the first thing that they ask you right out of the gate after you’re, you know, accepted as a witness, is to talk about your chain of custody documentation and what steps did you take to make sure that, you know, this is the evidence that, you know, you were provided.

And, and it’s got here a chain of custody to show, you know, how you obtained it and what it’s, you know, been through in terms of being connected to machines, being, you know, passed around while different people do different types of analysis. And that’s always something that everybody is curious about, is the life cycle of the evidence. And so, starting a new, fresh lab thinking through this, it was important for me to think through the various standards that were important knowing that, you know, somebody would have to speak to exactly, you know, the life cycle of a piece of evidence.

Si: Is there, in your opinion, a point at which standards should stop, in terms of being presented as evidence? I mean, collecting data I couldn’t agree more. In fact, I think that they’re essential that you have a documented understood process for, you know, collecting data. But after that, we are into the position whereby things are so fluid in the industry, we are looking at new applications, we’re looking at new uses of existing applications. And certainly from my perspective as a forensic examiner, I’m finding that I need to be more creative, I feel, than the standard allows me to be if I have to follow a process. Is that something that you could…would…how do you feel about that, basically?

Ryan: I think the fundamentals matter. In fact, I know the fundamentals matter, because regardless of what you do, two things are true: the rules of evidence, be it a civil proceeding or a criminal proceeding, they’re not going to change. The second thing that is true is that you are responsible to justify every action taken or not taken. So, there is this element of justifying your actions, right? Because the standard doesn’t go in and tell you, you know (let’s take the various ISO standards, for example), they don’t go in and tell you how to do your job as a forensic examiner. What they do tell you is that your outcomes need to be repeatable, right? They need to be reproducible.

The difference is that you can do it again. Reproducible, meaning someone else can follow your documentation and they can do it again. And you need to be able to justify, again, every action taken or not taken. So, as long as that trained digital forensics examiner can take the stand and say, “here is what I was presented with, these are the steps that I took, and this is why I did it”, then the version of the app, or the type of the app or the type of case doesn’t matter because, it really comes down to those fundamentals

Si: At that point, then (playing devil’s advocate entirely in this conversation), one would say, “okay, well, I do that. I write my documentation in a way that means that it’s reproducible. I can justify everything that I’ve done. Why am I spending $10,000 on getting my lab accredited?”

Ryan: Well, lab accreditation is definitely a, you know, a case by case evaluation of circumstances, and I’m going to hand it over to Rob in just a second to talk about that evaluation for his organization. But whether you go through the actual accreditation or not, you know, that’s an individual decision. It’s a business decision, really. Whether we’re talking public sector or private sector, it’s still a business decision. But putting in place best practices is a far more significant issue in my view. So, Rob and I had this conversation before where we talked about, you know, if…it’s very easy, especially in high volume, to stray from your best practices when it’s not a requirement.

So, for example, a hot case hits the door. Again, it doesn’t matter whether we’re talking civil litigation or criminal proceeding, but the hot case hits the door, and if you don’t have that standard in place that, you know, basically that handcuff that says that you must do it this way, it’s very tempting to just say, “look, I’m going to do the acquisition quick. I’m going to get at the data that’s critical to this, and I’m going to get it back to the investigator. No problem. I’m gonna backfill the paperwork later.”

But in a high volume lab, it doesn’t happen, because the phone rings, the next case comes in, and you’re doing the same thing. And before you know it, you’re taking the stand with an incomplete case folder because you didn’t follow that process. Now, Rob, as far as the accreditation, you’re going to be the man here to talk about that.

Rob: Yeah, I mean, I think it was a no-brainer coming in through a clean slate of building a lab. I couldn’t agree more with, you know, sometimes when you have a high volume of work or, you know, you’re under the time pressure that we’re currently under all the time to get things done and get it done quickly, I wanted guardrails on our process. I wanted to make sure that everybody did the same procedure throughout the course of the life cycle of a piece of evidence, and even just the matter.

You know, I also wanted to make sure transparency was there at every stage and throughout the organization, throughout my team, so that we can all log into a centralized repository and see where we are in the process at any given time. You know, there was a lot of consideration about different evidence management solutions that are out there, but I think that for me, that is something that I can speak to my clients, which are relatively, you know, larger law firms, multinational companies that want to know that you’re doing the right thing with their data. We’re talking about, you know, secret sauces of, you know, what people’s entire business is based on, HR information, the most sensitive information in an organization. The clients ask, “how do you deal with securing your evidence? How do you deal with logging things?” You know, it’s not just about doing the collection, it’s about all the documentation that’s associated with it. That’s a part of it. And that’s key.

And that’s really why in the, you know, the real time situation we were being faced with the pandemic, we still made it a point to have a physical lab, because there still needed to be a place to send evidence to do the analysis. And that was important to have the standards. We built our entire lab, the design of the lab based on ANAB standards, on security, on designing the walls and the entire infrastructure. And this evidence management piece was a huge part of that as well.

Ryan: Yeah, and it’s important to note that, you know, Rob, you’re not a one man band. When I was in the digital forensics lab getting started, I was there and occasionally I had a little bit of help, meaning one or two people that stopped in occasionally, maybe they’re working their own case or what have you. But it’s not scalable if you don’t have processes in place. And we need to understand that we’re all humans and our personal memory of a case is perishable.

And then when we start adding multiple people in with our own strengths and weaknesses, maybe Rob has a few, you know, a few members on his staff that are just amazing report writers, and maybe he has a few members on his staff that man, they can really get the job done, but when it comes to writing that forensic narrative, it’s just not what they like to do. They’d rather just, you know, put it on hold. But the framework of compliance is what steps in and is the great equalizer. It ensures that everything that’s coming out of a lab is consistent regardless of who did it.

Rob: Yeah. And I also want to say…talk about different, you know, ways that people do things. Every, you know…most of these teams are comprised of people in all different backgrounds. You know, you may have been in the military, I may have been in academics and coming out of school, you know, Ryan’s in law enforcement. And so we all have a different perspective. Some people may like to actually print up forms, other people may like to fill them out electronically. Think about the data collection log versus a forensic acquisition form.

I’ve had discussions about wanting to just have a forensic acquisition form to fill out over the data collection log, but then the client needs a data collection log. And then also your preference. I prefer to print up my forensic acquisition form to be able to focus on it and make sure that I’m doing it in my own handwriting and I’m putting down all my thoughts and notes. Some people prefer to do that on the computer.

Again, this is all personal, you know, we all have gotten to this a little bit from different backgrounds as we started off with. And this is a way to standardize and streamline the process in a way that I know that if I’m going to be the one testifying, I know what Ryan’s, you know, process is, and Ryan knows what my process is. Because again, you’re on your own when you’re testifying on behalf of your employees and colleagues.

Si: Yeah. No, I mean, I understand the con…I mean, and again, you know, back to…I’m playing devil’s advocates to a certain extent here. I completely understand the principles behind best practice is unassailable as a concept. Accreditation is an implementation of best practice. What we’ve…and, you know, what you guys possibly are aware of, because it’s reasonably public knowledge, is that in the UK they are forcing accreditation on us.

Ryan: Yes.

Si: And this is something that is causing some issues (it’s causing a lot of issues!) for the one man bands. It’s great if you are a lab who has, you know, multiple people. And in fact, I think accreditation actually has a greater value in those scenarios, whereby you are trying to standardize an output across multiple people. But where we have a single person whose output is…okay, it is variable, we have to face it, we’re all human, that we don’t do the same work on every day that we would necessarily always hope to! But we do, you know, a reasonably standardized thing. What’s your feeling about compulsory accreditation?

Ryan: You know, I don’t think it’s a one size fits all. And when I talk about compliance, I’m not necessarily talking about accreditation. I think that there are, in fact, I know there are many labs that are saying, “look, we are gonna take 90, 95% (the number isn’t exactly important), and we’re going to adopt that level of compliance with accreditation standards, but we’re just not prepared right now. Maybe it’s because we don’t have a compliance manager.

Maybe it’s because we don’t have compulsory, you know, accreditation requirements on the books right now.” And they’re just going as far as they’re comfortable. And I support that if it’s the right decision for them. The reason I support that is because, to your point, accreditation is really just a collection of best practices. And as an evangelist, for me, it’s important to advocate for raising the bar on digital forensics as a forensic science.

So, as long as we are trying to improve ourselves, do better, manage our workflows, you know, and it may even be that we adopt a system that makes our productivity increase, makes us more productive, you know, that the situation I found myself in with accreditation, where at first it was very daunting with all of this, the administrative overhead, I’m rolling my eyes being like, “I’m going to spend more time dotting I’s and crossing T’s than I am actually solving cases”.

And through the adoption of a system that allowed me to, you know, to first of all make sure that I was dotting I’s and crossing T’s, but relieving me of the administrative overhead, that was the game changer for me. And that’s what led to me pulling a case folder with absolute confidence that I wasn’t going to take the stand and tank a case because I put everything on a sticky note and it fell off.

Rob: You could put Scotch tape on that though, Ryan?

Ryan: That would help..

Si: That’s best practice there. It’s double taping it then…

Rob: I think it really comes down to having the internal support. It’s an expensive endeavor. Let’s not forget about the number of man hours that goes into that, the perseverance that goes into writing. But it’s also a lot of push from the clients. I know a lot of clients that they ask, you know, when they hire you for a job, it’s also an interview process as well to get the job. “What are your standards? What are your security standards?” And so this all takes time. It’s not something that, you know, you can wrap up in a few weeks.

It’s something that’s…it’s a process, but it’s also now getting to learn habits, right? Better habits. And that’s something that’s not for every lab or every, you know, every company has different ideas, but there should be a consistency and a push always trying to do good and do better and look for opportunities where you can, you know, be part of a process that is going to only amount to success, future success for you, because people will see that you took that extra step or those extra steps and put in the effort to attain such a well-rounded and and respected accreditation internationally.

Ryan: I think that anyone listening in law enforcement understands that when the word compulsory is used, it almost always originates from somebody on midnight shift that did something stupid!

Desi: Yeah. It’s like the…

Ryan: That’s where policies are born!

Desi: Yeah. It’s like that sign in the radioactive forest where they’re like, “don’t lick the trees”. And, you know, someone has licked the trees in the past.

Rob: You know, it’s also…it takes a lot of effort to write, you know, as examiners. We’re good examiners, but putting down all those processes is a bit of a challenge. And, you know, sometimes I’ve seen that work best in a situation where it’s a bunch of colleagues doing it. But again, there’s a balance between the projects that you’re getting in and the internal initiatives, and that’s always something that we always have to take into consideration. You’ve got to keep the lights on, but then you also want to keep to the best standards. So, you’ve got to balance that time out.

Ryan: You know, I’ve dealt with law enforcement agencies in the UK and the US, and I do have to say that, you know, with that compulsory accreditation it has allowed the UK police forces to have standards in place. And when we walk in there and talk to them about their, you know, lab workflow, they do have a very set process. And it’s not like that in the rest of the world. So, you know, there may be some good and some bad there. But I think one of the other things that’s important with compliance and having things written down to Rob’s point, is that, you know, in digital forensics, we do have this dirty little secret.

And that is that for all the highly technical approaches and strategies that we use to get at evidence, we very frequently return that back to the client, be it an investigator or a private sector client, on external media. And if you don’t have a process in place to manage exactly how that’s done, you know, we talk a lot about chain of custody, getting it in the door, the proper steps of forensic science. What we tend not to talk about as much is, “okay, what next, what do you do with the digital evidence that you’ve recovered? And how do you get that back to the stakeholders?” And we talk to clients worldwide and two answers come up most often, and those are Blu-rays and flash drives, right?

Si: Are we not moving away from lab accreditation here into, you know…from the sort of 17025 and the digital forensics into just sort of standard information security stuff, though?

Ryan: I think it’s all part of the package. I mean, it’s not necessarily compliance in the sense of how you’re getting at the data, but it’s the entire process that has to be controlled. You know, there’s not a single event. It’s a series of events that are part of a process, and that process has to be managed because at the end of the day, that is what’s going into court, and that is what’s going to jam a lab up if that process isn’t managed correctly.

Desi: In that example, then, are you saying that when you are passing data back to the customer and there needs to be more process around that, are they the ones then taking it to court, not the labs?

Ryan: Well, it’s both. So, the lab is sending data back and both are testifying to the end result. My point being though, that as part of our process, we need to look at things like the durability of media. You know, how…what are we doing with that evidence? How is the stakeholder (be it an investigator, a prosecutor, a private sector), how are they handling it?

Because that is what the courts will look at when they’re going to determine the admissibility. Is it relevant, you know? And in order for it to be relevant, it has to pass the test of relevancy in order to be admissible. So, my my point in bringing this up is that we, we can do everything right, you know, 80, 90% of the way, but if at the last, you know, 10% we’re doing something that’s substandard, then we’ve jeopardized all of the work that we’ve put into it.

Rob: Yeah, I think that’s a good point just because, and we’ve spoken about this, you know, what’s the shelf life of, you know, USBs and thumb drives? But also what’s the best, you know, scenario for putting evidence into a locker. If the evidence is a, you know, a cell phone, you know, we have to make sure that that’s in airplane mode first. We also need to make sure that, you know, there’s power to that device if necessary. There’s all these different criteria.

And also temperature and room control, environmental control needs to be taken, you know, taken into consideration. We actually have a supplemental HVAC unit in our suite in a skyscraper. On the outside, it actually has a little balcony just because of the fact that we can’t rely on the building itself to keep the air going, so we pump additional supplemental air into that lab 24 hours a day to make sure that it’s a good, you know, situation for electronic devices. And also filtering out the air, you know, the warm air from all the workstations. So, there’s a lot that goes into evidence handling and durability and things like that that, you know, right out of the gate designing the lab, all this came into consideration.

Ryan: Yeah, the digital evidence may be ones and zeros, and that’s awesome, but in order for the rubber to meet the road, there has to be some kind of physical vehicle involved. And it doesn’t matter what you choose. You know, you may decide that, you know, some kind of redundant on-premise storage is right for you. Someone else may choose, you know, that cloud is right for them. And maybe flash drives are, you know, are okay.

But the point is to understand what the limitations are of anything that you have in place. And if you do use something like flash drives, you know, I would be looking at things like, well, the warranty on these things are generally, you know, three to five years, and my case may last longer than that. If it’s a homicide case, I’m keeping it for a lifetime, what do I do when that physical body, that vehicle breaks down? You know, have I thought about that in the terms of my overall compliance and digital evidence handling? At the lab, and also, you know, with the investigative stakeholders.

Rob: Yeah, we’ve had situations where the hard drives are shipped back to us and they’re in pieces, you know, but that was the original evidence. But we also, you know, obtain a backup if necessary in case that is going to happen, right? So, there’s all these things that you have to think about the durability, the resilience of the hard drives, right? It can impact a lot of things.

And if that’s your best evidence and you only have one copy, you know, the first order of business when we get data in the lab, besides entering into our evidence management solution, is to create a backup up on the network, verify that it’s the exact same and move on. But, you know, that’s part of that workflow that we were discussing, and that all needs to be, that everybody needs to be in sync, not just from the check-in process, but overall how this data moves around and who’s touched it. So, this way we can speak to the chain of custody at any point in that, you know, during that time.

Si: Yeah. I mean, one of the fun things about ISO standards as opposed to any others, is actually that the requirement is less “you must” and more “you must consider”. Certainly my experience of them anyway. And yes, to the extent that at certain times you can actually go, “yes, I’ve considered, I’ve decided I don’t wanna do anything about it. And that’s fine, thank you very much. But, yeah, you know, at least I’ve considered it.” So, yeah, I totally hear what you’re saying on that front.

Ryan: It definitely feeds into the overall, you know, compliance issues. Because, like, Rob’s talking about making a backup copy, and I know Rob, so I’m sure that he’s treating that backup with the same due regard as the original. My experience is a little bit different in law enforcement where, you know, we have that investigator that’s saying, “okay, I got this evidence back from the lab, I’ve got to lock this thing up in the evidence room. That’s gonna be super inconvenient for me. So what I’m going to do in the meantime is I’m gonna make a copy and I’m gonna put it on a flash drive, stick it in my desk drawer or I’m going to, you know, put it on my, on my desktop. That way I can say, “yeah, I checked this thing into evidence”, but every time I want to review it, I’m not going have to take it back out of evidence because I got this working copy.”

And then when we look at it…a civil proceeding or a criminal investigation, we know that other folks are going to need copies of that. And how is that managed and what does that look like? You know, frequently what that looks like is that investigator is just making more flash drives, and those are all unmanaged copies of the evidence. And from a compliance standpoint, it’s a domino effect because what…this is data, this is digital evidence. And, you know, well, I’m going to give Rob some of the files, but he doesn’t need all the files, so I’m just going to give him, you know, the folder that he wants or the cell phone acquisition that he really needs. So now I actually have variations of copies. They’re not even all the same anymore because I have subsets that are being duplicated. And you can see from a compliance standpoint how things can break down very quickly.

Desi: So how is…like, in that example, like that sounds super worrying. How is that being seen in courts once it reaches the court? Is it just they, there’s not an understanding and they may have history bringing evidence to court, so it’s just not questioned or it’s just not thought of because the judge or the prosecutor doesn’t have experience in the digital forensic space?

Rob: In my particular case, because, you know, we’ve dealt with some high profile celebrity cases where you take a full preservation of their phone, for example, and then they want to do a subset of the data just because they don’t want to produce anything beyond what’s being requested, right? So, even that has a workflow. But if you’re starting to create the subsets of the data, you know, as long as that workflow is documented and everybody’s on the same page, there’s generally not going to be a lot of arguments about, kind of, why that was decided. Let council speak on behalf of their client, you know? The data custodian is what we call them. And then, you know, I can speak to the fact that this has been, you know, part of the request. We still took a forensic preservation, but we created a subset because that was, you know, the requirements of the matter. We could always backtrack based on our notes and documentation.

Ryan: You have the documentation. And that’s different, you know? So, in the law enforcement arena there was a time, you know (after graduating Rob’s class of course), but I could take the stand and say, “you know, I went to X number of hours of digital forensics training. You know, I’ve personally executed search warrants and arrest warrants, and I’ve imaged X number of hard drives and I’m an expert”, and there would be no challenge, you know. And I would testify and I would answer all the direct, and I’ve had cases where defense counsel stood up and said, “no questions, your Honor”. Those days are over.

I mean, maybe on some minor cases that they’re just, you know, they’re not looking to challenge the evidence, they’re just trying to get the best deal they can or a recommendation or something like that. But when you have a criminal case now and, you know, you hire a defense counsel, it’s basically the expectation that they are then going to hire a defense expert who is going to look at all the data and come up with a way to attack your process. So, again, going back to the old days where you could just stand up and say, “I’m an expert”, and everyone in the room agreed because you seem to be smarter than they are, that’s over now. And we need to, as digital forensics practitioners, we need to prepare ourselves for that. That we are going to have challenges that we do need to have documentation. And this isn’t like computer hard drives, right?

That we have the almighty hash value. We could bring the original hard drive in. We would…we could hash it immediately (connect it to a write blocker, of course). And we would have our MD5 or our SHA-1, or SHA-256, whatever. We would have that. And we always had, you know, the northern star that we could look at and say, “look, nothing, we did create any changes. Here’s a forensic duplicate.” It’s not the same way when we’re talking about smartphones, you know. We necessarily boot them, changes are made. And we’ve always said that the rule is that you can take actions that change the evidence, but they must be documented.

Si: Right. Yeah, I mean, speaking as a defense expert who comes in and challenges your processes…to be honest, I tend not to challenge processes because, you know, processes are good, it tends to be the logic that’s applied after the processes that isn’t usually so hot. But what I’m finding is that subsection of information that you’re talking about being handed over, I’ve seen…I’ve got a case at the moment that I’m arguing about because I’ve got…I did a calculation on it, I think I had 25% of the evidence that they’re using to prosecute with.

Ryan: No, that makes no sense. Yeah. I would absolutely push back on that.

Si: Exactly.

Ryan: There could be subsets. So, you know, if I create, for example…you know, I have 100% of the data and I create an exhibit. It’s work product. You know, it’s something that I wish to expound upon. You know, it doesn’t mean that you can’t do the same thing, right? But if you’re deprived of that 100% data set, well that makes no sense because that’s…to me, that is the absolute right, that that needs to be turned over as discovery. I don’t have to turn over my, you know, my work product, something that maybe I’ve written a Python script that can extract the data from the database and show it in a visually appealing way. I don’t have to share that with you, but certainly I need to share the underlying database with you to allow you to do your own.

Rob: I just had a case where the whole case was about metadata of documents. And so we were given the documents and I said, “hold on, I can only tell you so much. I need to understand where these documents came from”. Well, they came from somebody’s computer. Well, let’s get the folder structure, because the folder structure will tell me when the doc, you know, when the folder was last modified, maybe when the documents were put in there. And then, you know, that came into play. And, you know, now we need more than the folders because we also have to look at the registry to see, you know, what’s going on in the registry and see how the file was recorded in there.

And, you know, these are all very important things, but the context is lost when you do these subsets of data to really understand, you know, all that’s left out there. And so, you’re right, I’m running a lot of requests right now for additional pieces of evidence to be provided so that we can do a full forensic analysis and give the whole story, versus what everybody is comfortable providing at this point, because that may have some really important pieces to to talk about, you know, in that additional evidence that you would’ve never even known was out there.

Ryan: And it comes back to documentation too, because if I document exactly what I had, what I did and why I did it, then you’re going to look at that and say, on the receiving end when it’s turned over on discovery, you’re going to look at it and be like, “ah, okay, I understand, you know, this is what they did”. And either there’s either you agree with it or you disagree with it, but with that documentation in hand, you understand why you got the data you did. And maybe your strategy at that point is to challenge the process.

Maybe it’s that they, you know, they did a…somebody did a great job of turning over all the inculpatory evidence, but what wasn’t turned on turned over is exculpatory, you know? So but, if you have it documented, then you know exactly why you ended up with what you did.

Rob: Yeah. What files do you want to provide me so I can tell you whether or not that’s going to give me a complete picture. And if you are providing me certain files, and I feel that that’s still not enough, I can then say, “I need these additional, you know, forensic artifacts to be able to look at this in a little bit different way for, you know, for our intent, our goal in all of this”.

Ryan: And it comes down to transparency, really. And in, you know, on the criminal side with law enforcement, transparency is…has always been important and is more important than ever before, because if law enforcement is not being transparent, then the suspicion runs rampant. You know? We…law enforcement needs to be running compliant, defensible investigations. It needs to be beyond reproach. Everything needs to be out there, laying our cards face up. Because in today’s environment of ‘defund the police’ and everything associated with that, it is so much…is so critical that we have a proper investigation that’s well documented and there’s nothing to hide.

Rob: Sure.

Si: If I told you that the current UK stance on guidance given to the crown prosecution service is that they only have to give over the things that they consider to be relevant and that’s it. And they can ignore us. What would you say to that, as a defense?

Ryan: You know, I don’t think it’s something that could apply in, you know, under US law for rules of discovery and just from…if we take jurisdictions out of it, to me that sounds like a recipe for disaster.

Rob: Yeah. And also, you know, what are you being provided? Are you being provided active data, deleted data? You know, what else is potentially out there that may be relevant that you’re losing the opportunity to review?

Ryan: Yeah. The courts are supposed to be…they’re designed to be an adversarial process, and they’re designed to be, you know, equal footing if not actually that the prosecution has, you know, has the burden. So, it would sound very…it doesn’t sound like it’s in the spirit of the way the system was designed to be able to subjectively say, “you can have this and you can’t have that”. Absent investigations that are contraband related, and of course, when we’re referring to the digital arena in contraband, we’re most commonly we’re referring to child sexual abuse material, otherwise known as CSAM.

So, granted there needs to be a departure where review of CSAM is under controlled circumstances, you know, monitored. Maybe it’s the defense counsel and defense expert comes to the prosecution or the investigating agency, and they review it there as opposed to, like we talked about, burning them a Blu-ray or giving them a flash drive. But, you know, aside from that, I think that the systems were completely designed, and again, not a call about, you know, constitutional issues, but I do believe in the investigations being defensible, transparent, and there being equal access to the evidence.

Si: Yeah. It’s a contentious point in the UK at the moment, as I’m sure you can imagine that we are being put into this position along with, you know, along with the certification and, you know, compulsory certification and certain statements like that make it seem very unbalanced in favor of the prosecution as opposed to being a fair adversarial system as you describe. If…as we’re going to have to go into compulsory certification, how would you guys recommend that we start? How do we go about it? What’s the best tactic that we should do to approach this? I mean, you must have done a few by now and have some lessons learned for the rest of us who are about to experience it.

Rob: First off, I would look at all your existing documentation and see what’s already been developed to see if your, you know, processes and workflows is something that you can bring into that process. Number one, because it is a very time intensive, labor of love type of project. Overall, there’s a big time commitment, you know, internally. Understand, you know, if you have the proper resources to be able to get this done. See what the timelines are. But also see what additional support you may need, you know, even outside of the organization. I will tell you from going through and applying for an accreditation here, what we’re going through, there’s not a lot of, you know, consulting that can be provided by the accreditation board.

They’ll give you some reading material and some kind of, you know, information that you could kickstart with. But ultimately you have to be able to interpret what’s being requested and, you know, come up with your own documentation for your own organization. Every organization’s run a little bit different. So, it’s not a cookie cutter template type of…you know, they’ll give you the standards and the way to, kind of, look at things and you have to move forward and write the documentation tailored to your organization keeping to their, you know, their requirements.

Ryan: I would start with taking a deep breath! And then preparing yourself to take a unbiased 30,000 foot overview of the way you do business right now. And what I mean by that is, I would say, “okay, in order for a device to hit our front door, what does the investigator have to do, or the submitter have to do? And what do we have to do in order to receive that submission in and start that process of work? What do we…are we asking our submitters, be it investigators (private or public sector, doesn’t matter), are we asking them to put pen to paper and fill out a form? Are we asking them to fill out some kind of web form that we have that ties into a system of some sort? Is it an email?

You know, what does that look like from our, I’ll call it the customer’s perspective, the submitter’s perspective? What do we do on our end in order just to get the thing in the door?” And then once we have that, we have a good idea of what that looks like. What happens next? Does, you know, it comes in the front door? Do we have a person who then puts that on a shelf making it available for an examiner? What does that look like, you know? I would want to start asking about, you know, what’s our compliance frameworks? Are we required, for example, is this a public sector law enforcement in the United States? If it is, we’re going to start thinking about criminal justice information security policy or CJIS. We’re going to start thinking about compliance with that private sector.

We don’t have to worry about it. UK? Different requirements, right? And then when we say that person puts it on the shelf, is that documented? How is it documented? What about when the examiner goes and pulls it for work? Is that documented? What about documenting what they did? Is there a tool log? Did they know the software that they used, what version it was? Did they validate that software? If they did validate it, maybe they went and they downloaded the latest version of whatever forensics, you know, automated forensics tool they wanted to use. They got the latest version.

Did they…do we have policy around how they validate those tools? Did they go to NIST and download a control image and then use that control image with the brand new version so that they could determine that what was found was what was expected? Nothing more, nothing less. It was in the right place. And then did they document it? How did they document it? Am I providing a tool log to aid in that validation of new versions? Do I want them to validate every version? Do I only want them to download…I’m sorry, validate the major releases that they’ve downloaded? In other words, version 7, version 8, version 9, not necessarily 7.5.2. And do I have the policies and the tools in place, whether it’s a logbook, a spreadsheet, a solution that runs my whole system, whatever it is, do I have it? And then how are their reports written? Do they have thorough documentation of everything they did so that they can justify what they did and didn’t do? Are my reports consistent?

And do they require…do they have all the data in them, all the elements that if we’re looking at ANAB, ISO 1725 do all the elements that are required in that report are listed? Do my reports have that? And then look…what are my review processes? Do I need to be doing things like technical review? Do I want to do technical review 100% of the time or do I want to do it 10% of the time? And, you know, my administrative review, are all of those things in place? So, we really do have to do an outer body experience, that 30,000 foot overview and take an unbiased look of, “let’s follow it”. You know, what was the old children’s book of “how a bill becomes a law”? And it talked about the, you know, a democracy and how a bill is presented and it moves through. You have to do the exact same thing but you have to be a cell phone, or you have to be a hard drive or whatever it is that’s being submitted in.

You have to think about the route that it takes and what the requirements are. And you can immediately find problems in your process. No matter how good you think it is, when you look at it objectively, you will find room for improvement. And that’s not bad. It’s not bad to aspire to improve the processes and raise the bar on the forensic science.

Desi: So it sounds like we’ve got the opportunity to look forward to your new children’s bestseller there, Ryan.

Ryan: Yeah!

Desi: How to get a mobile phone through…

Si: How does a disc become evidence? Yeah!

Ryan: I can tell you it will help children fall asleep. That’s all.

Desi: Alright, guys. Well, we’re coming up to the top of the hour. I’ll leave you guys with, kind of, the hardest question we always ask everyone, and because we’re always so busy. But, what do each of you do to unwind from work?

Ryan: Go ahead, Rob!

Desi: More work!

Rob: I actually do a lot of industry thought leadership through publishing different articles and different books on the subject. I do have three children all under the age of seven. So, I have a lot of home stuff going on. But really it’s about trying to figure out a way to continue to balance both my professional ambitions as well as my family obligations. And doing that both while I can…early in the morning for writing, and all day long, otherwise with the kids.

Ryan: The gym and bourbon for me!

Si: A good combination.

Ryan: That’s balance. It’s just what Rob said, it’s balance!

Desi: Well, Rob and Ryan, thanks so much for joining us on the podcast today. We’ll have transcripts and links to anything we’ve spoken about down in the show notes after this. As always, if you’ve liked what you’ve heard, like and subscribe, give us some suggestions on what to listen to next. But again, thank you gentlemen for your time, and we’ll see everyone again next time.

Ryan: Thanks everyone.

Si: Cheers, guys.

Rob: Take care.