Christa: As technology’s advancements continue to outpace practitioner’s ability to keep up, standardizing digital forensics processes, practices and tools has never not been a hot topic in the community. The response ranges from the United Kingdom structured efforts to much more fragmented approaches, including in the United States. The result: few labs are fully standardized.
On this week’s Forensic Focus podcast, we’re talking to the supervisor of one such lab. Jesse Lindmar works at the Virginia Department of Forensic Sciences Central Laboratory, supervising the digital and multimedia evidence section. I’m your podcast, host Christa Miller and welcome Jesse.
Jesse: Thank you for having me.
Christa: So let’s… I’d like to start with a question. How long has the Virginia DFS lab been accredited?
Jesse: So they were first accredited in 1989, so 22 years ago.
Christa: Wow. So was the digital forensics portion of the lab always a part of this or was it accredited much later on?
Jesse: So, the lab was accredited as a whole in ’89. It’s a traditional… what I call a traditional forensic laboratory. So it has all the traditional forensic disciplines there, for example, your latent prints and impressions and firearms, and all of those things. But the digital multimedia evidence section, that didn’t get added in until around 2005.
Christa: So, I mean, digital forensics had obviously been an industry by then. Maybe not quite to the extent that it is now, of course. But what was important to the lab about being accredited? And is there anything about the accreditation that’s unique in the country?
Jesse: So, the core aspect of accreditation is just, it allows a laboratory to demonstrate their commitment to quality. I mean, that’s the core aspect of it as far as the standard that we’re accredited to anyway. Some things unique about the Virginia Department of Forensic Science, it was the first state laboratory in the United States to offer DNA analyses to law enforcement agencies. It was also the first to create a DNA data bank, which included information from previously convicted sex offenders.
Christa: How about with regard to digital forensics? Is there anything particular about that?
Jesse: I think, like most laboratories and law enforcement agencies (and other digital forensic service providers), it develops out of a need. And that was kind of the case here. I wasn’t here back in 2005 when this section originated and was built, but it kind of originated from a photography section that was in existence in the section. And you had requests from agencies that were asking for digital forensic-esque services and it just kind of morphed from there, pretty quickly I would say.
Christa: So, given that, do you think that standardization and accreditation are appropriate for every lab? Why or why not?
Jesse: Well, I mean, two part there: standardization, I would say yes to a degree. Accreditation, you’ve got various types of accreditation, so it really kind of depends on what you’re looking at. So, my laboratory they’re accredited under ISO 17025, the 2017 version is the latest we’ve been reaccredited over… or under, I should say.
And it really, it’s designed to help you develop and implement and adhere to a quality system. Your overall goal is to perform quality work and make sure the people that are doing the work are qualified and all of that aspect. So your end result of your analyses is a quality result.
It’s not getting into too many, I would say, specifics as they relate to digital forensics. So it’s not telling you, “This is how you must collect evidence, and this is how you must preserve evidence, and this is how you must analyze evidence.” And I think that’s one of the big misconceptions of accreditation, at least under 17025, is that it’s, “I’m going to be locked into this framework that is going to be difficult to work within, and I’m not going to have somebody tell me what to do if it doesn’t necessarily make sense.”
And that’s not at all what it is. You will basically be your own worst enemy when it comes to accreditation because you’re following the guidelines of the standard, but a lot of it is left up to you to develop your approach to that standard. And if your approach is too strict, well, that’s your fault, that you can’t then adhere to it, because you wrote it too strict. So that’s usually the biggest misconception of it. And then that’s really what I’ll tell people. It’s just about quality and you are going to create the problems for yourself, not the accreditation standard that are there.
Now standardization. (So, I mean, that’s discussing just kind of the accreditation, at least under that particular standard.) From the standardization side, now you’ve got a much kind of bigger discussion, as far as well, what do you want to standardize?
So under 17025, you’re trying to standardize quality and you’ve got various aspects of that you’re trying to follow. But just the general concept of standardization in general, you could get into, well, “Do I need to have a standardized approach to how I handle evidence? Do I need a standardized approach to how I acquire data from certain evidence? Do I need a standardized approach to how I conduct certain analyses of certain data?”
And that’s where the spider web of standardization could get a little crazy. And that’s the discussion I think that needs to have is: what can we realistically standardize that the majority of digital forensic service providers can follow?
Christa: So I’m curious about that. Because I know in the UK, there’s the Forensic Regulator’s Codes of Practice and Conduct. And I think here as well as internationally, the Scientific Working Group on Digital Evidence has a number of best practices documents. So what do you follow when it comes to standardizing those kinds of processes?
Jesse: So, you’ve got the SWGDE (the Scientific Working Group on Digital Evidence), and they’ve got probably the greatest repository of documents. So what’s great about them is they’re quick. They can have an idea and start developing a document to push out pretty quickly. They’re a well-oiled machine.
And then you’ve also got the Scientific Area Committees under NIST that then try to at least for the digital side (because I’m involved with that group as well, although I’m not going to speak for them), they’ll look at those SWGDE documents and then those try to get transformed into ASTM standards. Although now there’s a look at trying to take the SWGDE documents as their own and try to roll them into some sort of standard.
So, there’s those documents. And I will say whether it’s SWGDE or ASTM or any other body that’s trying to draft a standard, you do have the ability to comment on draft policies and things like that. But even with that ability, sometimes these documents come out and you may not necessarily want to follow all aspects of them. Maybe you don’t necessarily believe (for lack of a better word) in some of the guidance that they’re providing, or maybe that guidance doesn’t necessarily fit with how your lab or your section or how yourself, how you operate there.
So that’s a tough thing. It really is kind of taking into account the best practices. You’re looking at these documents for guidance and then trying to adhere to them. But sometimes you can also kind of have to cherry pick the things that actually fit with how you work. And then of course, being able to justify why you may be deviating from these standards.
But again, these are things that nobody is required to adhere to. They’re great guidance and they’re resources that I think most of us in this industry look to, but there’s nothing, there’s no overarching guidance or legislation or anything that says, “These are the things you’re going to follow as gospel.”
Christa: So you’ve mentioned a couple of times about quality and I think really what it comes back to, is the quality of evidence that ends up being presented at court, at trial. Nobody is required to do any of what you’re describing, as you said, but what kind of guidance do labs get? Especially the smaller labs that may not have a lot of resources from, for instance, prosecutors or from case law that maybe was unfavorable to what they’re trying to do?
Jesse: I mean, it really is kind of like the Wild West a little bit in the US. There really is nobody you’re looking to for, again, this gospel of guidance, kind of thing. So everybody’s kind of left up to their own to figure out how they want to work. And again, we do have these documents from SWGDE and ASTM and things like that.
And you’ve got experts in the field that you’ll look up to and you’ll listen to. And there’s some collaboration. Not as much as I would like, because, especially, I would say in both public and private sector, people don’t necessarily want to be scrutinized, they don’t want to share their kind of trade secrets, if you will. So you don’t have a lot of that sharing of methods and problem solving and kind of research and development and stuff like that. And I would love to see more of that.
But so, like in the example, you would say, maybe it’s a small department and someone’s working with some sort of digital forensic case here, and they’re like, “What am I supposed to be doing here?” And that’s where the standardization and the accreditation would be kind of great, if it was universally acceptable or university applied, universally accessible, I guess I should say. Because then it’s like “These are the things I’m going to follow, and these steps that I follow or this general framework that I apply is the same thing that everybody else in the US or the world doing this type of work is also following.”
And we’re not there yet. It really is that person at that agency or department, they’re going to do what they can based on their own kind of knowledge, skills, and abilities, what they’ve been trained to do, what their agency dictates that they do.
And all of that is going to affect the quality of that work that comes out at the end, and then what they’re then able to testify to. And none of us are created equal. None us are robots. So you get varying degrees of capability and varying degrees of accuracy and everything else along those lines, in those situations, as far as the result that’s generated from an analysis and what you’re testifying to in court.
And that, that is where it gets difficult because that’s, again, we have this antagonistic court system where you’ll have an expert, you’ll have an opposing expert. And sometimes they agree on findings, and many times they don’t. Because they’ve got different backgrounds and different skills and different abilities and different ways of understanding, which is great. But again, that’s where it would be nice to have some general plan that everybody is following, at least so there’s some basis you know, some baseline that we can adhere to.
Christa: So your colleague, Amy Curtis Jenkins, spoke at the American Academy of Forensic Sciences Conference this year about labs needing in-house counsel, which ultimately not just be independent of law enforcement or prosecutorial functions, but maybe offer some of that oversight or at least some of that sort of guidance. What do you think… do you agree with that? What do you think it would take for that to happen in terms of resources and oversight and whatnot?
Jesse: So like oversight from in-house counsel, is that what…?
Christa: Yeah, I think just having… again, given that the quality stems from the need to have admissible evidence coming and being introduced at trial, to to be able to offer that… I guess, smoothing over some of those rough edges that you’re talking about.
Jesse: Well I guess I would say, like the in-house counsel’s going to be one element of your quality management system. So like in general, you’re going to have (like we do at our laboratory) you’ve got different people with different roles in that system. And including a quality manager.
So everybody kind of plays a part. So you have my examiners that play their part and then me as the supervisor and I’m playing my part in that system. And then I know it’s going up to the laboratory director and then the director of the laboratory system. And then we’ve got our, what we call our technical services team, which is our program managers, and they oversee certain sections. Everybody plays a part in that quality system.
So the in-house counsel is one aspect of that, and there is definitely a benefit to having that if you can, it’s definitely a plus. I would say they’re there to help obviously guide us in the legal arena. So we’ve got somebody on our team, if you will, there to guide us through certain things and advise us on certain things and help us if we’re dealing with court orders, or just interpreting court documents in general.
And of course they understand what’s required, the best ways to present evidence and things like that in court, because they’re all coming from that background. So it’s definitely beneficial to have that capability in house.
Christa: Have you noticed, by the way, there’s some statistic out there about, I think something like 95% or 97% of all federal criminal trials end up plea bargaining. Do you think that that impacts that sort of push towards quality, or no?
Jesse: Well, the way I look… regardless of where a case or an investigation, or court proceedings, wherever they’re gonna end up, our goal is still to just provide the most accurate and transparent examination of results that we can. I mean, that’s the idea. So, we have the benefit (my group, my laboratory) obviously we’re involved in that whole process, the whole legal process, but we’re not usually paying attention to the end game, where things are going.
And that’s kind of also the benefit where we’re not directly affiliated with law enforcement or anything like that. That’s by design. So, I mean, we play our part in that process and that’s what we get to focus on. We’re not worried (for lack of a better word) where things are going to end up. We just want to make sure we can do the best work that we possibly can to aid in this entire legal process.
Christa: Okay. So I guess I’m going back to the notion of standardization itself and/or accreditation, where is the line? We’ve talked a little bit about labs that are varying sizes, the populations that they’re serving, whether they’re larger or smaller, the types of cases that they’re working, etc. How do all of those factor in to the decisions that they make about to what extent they’re being accredited or not? And to what extent they’re standardizing or not?
Jesse: Well, I mean, I think a lot of people — and it’s fair — a lot of people are gonna look at it (the standardization and the accreditation aspect) “It’s going to cost me time and money to actually develop and implement and adhere to that type of thing. It’s going to take away flexibility.” And those are fair thoughts. And I would say, for both of them, yes there’s an upfront time and cost in being able to develop your own process to follow some sort of accreditation or standard process.
But once that’s in place, the machine runs, you need to be part of making sure that machine continues to run. And it isn’t a cut down on… the end goal here is you don’t want any problems. You want me to mitigate or minimize problems that you’re having as far as your forensic aspect there.
So whether you’re a large agency or a small agency, the accreditation can be applied at scale. It’s obviously going to cost a little more to implement on the large-scale side, because you got more people you need to kind of onboard and bring into that fold and educate and kind of get up to speed. But once that initial cost (time and the financial aspect) are done again, it runs. And the goal is that you want a quality team generating a quality product and even on the small scale, it could be… I mean, both large scale, small scale, it’s important.
Because the more people you have working in a laboratory, the easier it would be to kind of get away from the standards that you’re trying to follow. Because you’re trying to make sure everybody’s following everything that you’re supposed to be following. Easy to do that on a small scale: you’ve only got two people in your lab, or two people in your section, it’s real easy to check whether they’re following the rules. So not as much to maintain there.
Now, as far as the end results, whether it’s one or two people working a case or 40 people working a case, I would still want to be confident that they’re doing everything that they can to make sure that the work that they’re doing is of a certain standard, and that ultimately, if they’ve got to go testify about this, they understand what they did, and why they did it, and they could explain what the result that they achieved, what it means.
Christa: So to that end, given the rapid pace of technological change, is true standardization even possible, or is it more needed now than ever?
Jesse: Yes and yes! This is kind of where it’s… trying to figure out what you can standardize. And what my approach has been (because I think about this all the time), how far can you actually take it? Because again, I don’t want a team of robots. I want people that do this, that can think and can kind of step outside the box, if you will, and look at something and try to figure out a solution and things like that. And again, not get bored out of their mind because they’re so regimented.
But I like to focus on the basis, the baselines, the core aspects of what each and every type of case we work contains. And I want to make sure that my people understand those core function and have those core skills.
And also, for example, I mean, just understanding (and sometimes this isn’t something you can teach) but I would say, having an inquisitive mindset and knowing like “Okay, I don’t know what that means, this thing that I’m looking at,” or “I don’t know what my approach should be for this, let me do a little bit of research.”
And we do that all the time. There’s always something where it’s like, “Okay, let me figure this out, let me read a little bit more about it, let me see what I could find, and then start doing maybe some testing or getting into my analysis or whatever.” Like that’s an important aspect. But again, I want people to be able to do that. I don’t want to be like, “This is for every single scenario, this is what you need to do.” Because again, there’s no free thinking there.
So it’s kind of figuring out what’s that line of things that I can say, “This is the way we’re all going to do it, and then this is where you have the ability to kind of adapt and pivot and figure out what the best approach is going to be.”
Now, the nice thing about that (this is getting into another topic) the peer review aspect, is I can have somebody do that. I can have it to where, we’re all doing this, this thing this one way, and now you’re able to kind of diverge and do your own thing as the case requires.
But then someone’s going to come behind you and they’re going to review what you did. And they have questions or maybe if they feel like the approach you did wasn’t necessarily the best approach and they can articulate why, and you can talk about that. This is how everybody gets better.
But going back to where people have the core understanding of things: are you comfortable navigating raw data? That the plain text or the hexadecimal, are you comfortable working in that environment and decoding data? Do you understand if, let’s say for example, that the majority of data that we’re analyzing is within a database, do you understand that database structure and format? Do you understand how it’s designed? Do you understand how to read its data or query its data?
And if you do, if you understand that core aspect and we all understand it the same way, and we can all work with it the same way, then you can apply that same knowledge and skill to any other data set we have that has data from a database.
So we’re understanding the concept of searching for keywords or phrases or regular expressions across a dataset. It’s things like that where I like to have these core understanding, or have my people have this core understanding, so they can fall back on the basics, if you will. If it’s like, well, “I don’t know where… I know what kind of information I’m looking for in this case, I just don’t know where to find it.” “Well, what’s a way you could look for that?” “Well, I could probably do some searching, cross that data to see what’s responsive.” “Yes, yes you can.”
So you’re falling back on that base level understanding of conducting searches and then reviewing the results and then interpreting those results. And then you can apply that at scale across any of the data that you’re looking at.
So it’s not necessarily like I’m going to say, “There’s a standardized approach to analyzing web browser history or internet history, and this is the way you’re always going to do it.” It’s more of having a basic framework that you can follow, that you can apply, that we all follow and understand, and you can apply that to that type of analysis, if you will.
And I think that’s the kind of thing that’s kind of lacking across the board for everybody, because nobody knows where to start, where to pick this up, how to develop something like that. And then of course, sharing it so we’re all on the same page there.
So those things are important regardless of the size of your laboratory, and it’s easy to implement if you’re just focusing on the basics, then as you get into more kind of specialized analyses, well, that’s maybe where you have to focus a little more attention.
Christa: So I guess I’m curious, in terms of having that inquisitive mindset and taking the time to go through and do the research and figure things out, there’s still gotta be an opportunity cost with incoming cases. And how much time do you spend on doing that research versus, cutting through the backlog, right? Which I know is a problem for a lot of labs in the US and worldwide.
Jesse: That is a tough thing, and this is where you’ve got a multitude of approaches. You’ve got laboratories that are going to say, “Okay, to this, this is your approach. Every case, this is all you do.” And because they’ve got… they know, like they have to have… from the day a case is received, they maybe have to get it out the door within five days.
And it’s a tough balance there where you’re trying to balance speed with thoroughness and accuracy. And that’s really a tough thing. And I would love to know if anybody’s mastered that yet, because there’s probably shortcomings with almost any approach.
So with my group, we try to balance that as much as we can, but I’m kind of a real strict proponent of the thoroughness and the accuracy. That’s kind of riddled throughout our procedures. We’re constantly verifying the equipment that we’re using (so for us equipment and software and hardware). We’re making sure that the output that we’re getting is expected and it’s accurate and we’re being thorough.
Because we want to make sure that this end result we’re sending out to our submitting agencies, and those investigating officers, that it’s going to have the information they need to kind of continue with their investigation. And it’s a tough balance.
If you’re spending that time, making sure everything is thorough and accurate, obviously it’s taking time and the clock’s ticking and you’re not getting things out the door in five days, in most circumstances doing that, because you’re trying to, again, cross all the T’s and dot all the I’s.
And with the accreditation aspect, you’ve got a lot of documentation. The idea is that somebody else can come behind you and read your notes and reproduce what you did and achieve the same result. And you can’t do that without any notes, or very minimal notes, or cryptic notes. And all of that takes time. But the end result is we’re very confident with what we’re producing, what’s going out the door.
Christa: How are you dealing with some of the newer extraction methods, parsing methods? So again, going back to the idea that technology is advancing so rapidly, novel methods that are not generally accepted yet, they haven’t had a chance to undergo that testing. Maybe the tools haven’t been either validated against them, or it’s difficult to verify the data. How do you deal with that kind of thing?
Jesse: So it’s kind of going back to two concepts: you got validation and you got verification. And those are two terms that are actually defined in the 17025 standard. And they’re clear as mud! They try to be. And the kind of the way I look at it, and I think other people when they see the word validation, maybe they think of like a formal process because really it’s what it’s supposed to be. It’s supposed to be this planned out formal process now that you’re adhering to, and there’s all this documentation and it’s this kind of robust, thorough testing.
Now, the tough thing with applying that concept to digital is most of the time we’re using tools that have many capabilities built in. And you’ve got this isolated dataset or device with a specific data set on it, and you’re running this through this specific software suite, and you’re looking at the results and you’re saying, “okay, based on these results, I’m basically blessing this, this equipment, this software to be used in all cases, in all scenarios”.
And validation is supposed to be like in other disciplines, it’s a very specific thing. Is this function, can it be used in this particular scenario? And that’s a tough thing to do in the digital world. It’s tough thing to say, “Hey, this piece of software that’s designed to just allow you to analyze various types of data, because I validated it in this certain situation, it’s now good to go in all these situations.” And so I really… this is just a thing that I can’t really stand, honestly. I mean, you’ve got this validation requirement, we have these requirements, to validate things.
Actually let me specify. So like under 17025 in there, even then they’re clear: you need to validate laboratory developed methods, something you develop in your laboratory, a standard method that you’re basically using out of scope, or not the way it’s supposed to be used. Those are the two main areas in which you’re validating something.
Most of the stuff we use in the realm of digital, it’s standard. It’s not something you’re developing internally in your laboratory. This is a commercial product that you’re buying. And there’s many non-commercial products as well that are out there that are amazingly useful.
There’s even some language in 17025 about commercial off-the-shelf software being considered validated by the developer. So these specific formal validations, I don’t think they really get us anywhere, unless they’re almost always very targeted. Unless you’re validating something that has a very small use case, very specific function, and you’re only using it in that specific use case, formal validation is great.
But when you look at some of the validation things that are out there publicly facing that you can download, they’re trying to test a lot of different things, but even then it’s not every single scenario and it never fails. You might validate something, do this formal drawn out validation, and then the minute you start using this piece of equipment in casework on a different dataset than what you validated against, it doesn’t perform as you expected.
And then does that nullify the validation you just did? Well, no. The validation is supposed to be designed. You’re using it in a specific case. You didn’t account for that case in your validation. And you’re not going to be ever able to really think of every single scenario that you’re going to use that piece of equipment in.
So now for me, that lends into verification, which is basically… you’re just figuring out does this piece of equipment perform a certain function that I need? And those are the types of things that, at least my group, we try to do that in every single case, every single time we’re using a piece of equipment.
Which again, this does take away from being able to get things out the door very quickly, because I’m trying to be mindful of, “Okay, in this particular scenario that I’m using this equipment, I now have a potentially a unique device with a unique dataset. Is it performing as expected?” And the only way to confirm that is to go through and have some checks and balances. Is everything this tool showing me, is it thorough? Is it accurate?
And that’s something we work at in every single case. And you almost always come up with some sort of shortcoming or some unexpected result that you then have to compensate for. And it’s completely understandable. I mean these products we use, we use and we rely on, they can’t be programmed to do everything. And none of them market themselves as being able to do everything. You can make it in a lot of ways, do what you want it to do, you just have to understand that.
So having that idea of “everything needs to be validated,” I don’t think that’s really fair to try to apply that concept in all aspects of digital forensics. It might give you the warm fuzzies that “I did this validation”, but I don’t think in a lot of ways, it’s not inclusive enough. It doesn’t answer all the possible things. And that’s where that verification side, I think is more important, where you’re constantly kind of spot checking things to make sure you’re comfortable with the accuracy of what that piece of equipment is doing.
Christa: So I’m kind of, as you’re talking, I’m kind of thinking in terms of the criticism that ISO 17025 has received for not being fit for purpose for digital forensics. It doesn’t sound necessarily though, like that’s the case.
Jesse: Well, like I said, I understand when people are like, “Down with accreditation” and “Never me!”, and “Oh my God, the cost.” And again, I would say, I can’t even comment on the cost because I came into a laboratory system that was already accredited. So things were already established. I just had to take what was established for my team and work at following that and then trying to evolve it and make it better and adapt it to the new ways we operate and things like that.
And I’m in the midst of revising our procedures right now, trying to make it a little more in line with how we need to operate today. So it is kind of a cat and mouse thing there. But again, at least as far as 17025, it’s just designed to give you this quality framework to follow.
For example, it’s like, “Okay, you need to establish training guidelines for your people.” It’s not telling you how to train your people. It just says you need to have a training plan, period. So it’s like, “Okay, I need to develop the training plan. I will make the training plan what I need it to be. If I’m not happy with it, I revise it.” It doesn’t get as specific as people…it might say, “Okay, you need to retain certain records.” Okay. It’s not going to tell you how to retain them. In some places it’ll tell you at a minimum what you should retain, but you are going to develop those guidelines on how you’re going to retain that.
So if you make it too complex, that’s on you. You’re going to have certain requirements that your examiners have to meet. It just says, “You need to have requirements.” It doesn’t say what those requirements really need to be. You establish that. So again, it’s giving you the framework to follow.
And as I mentioned earlier, I think people think it’s going to tell you how you’re supposed to operate, it’s going to give you some general guidelines. You’re going to develop the documents that tell you how to operate. So the more involvement you can have in that process, the better your quality system is going to be.
So like in my laboratory, we have overarching documents that the entire laboratory has to adhere to. So those are things we call our quality manual in our laboratory. So those are the things everybody’s got to follow the same, like evidence handling guidelines and chain of custody guidelines.
So those are like overarching concepts that it’s written to adhere to the framework of 17025, and then we made it also fit the way we operate so it could be universally applied to our entire laboratory system. And we have four laboratories in the state of Virginia that all follow those guidelines. And then you’re going to have more specific laboratory-based ones. And then you’re going to get into your section-specific.
So like I’ll have procedures, that the digital multimedia evidence section, that we are responsible for following. And that’s going to dictate the things that we can all follow, and then it also is going to allow some room for areas where people need to be able to do things independently, because there is no way to make a solid rule for that particular thing.
So these are things really that everybody can follow and even without the accreditation, even without that seal of approval, if you will, you can model yourself after a laboratory’s quality management system. And we publish almost all of our documents on our laboratory website. There are several other labs that do the same.
I’ve had some discussions with people about, “Hey, wouldn’t it be great if we could give some guidance to people on how they could apply, for example, 17025, those guidelines within a digital forensic environment?” Because there’s nothing like that out there right now. Everybody’s kind of trying to figure it out on their own. And sometimes we all naturally end up doing the same kind of thing, and then sometimes people come up with really interesting ways of applying a guideline.
I have the benefit of being a technical assessor for an accrediting body. So I get the occasion of sometimes going out and assessing these other labs and really seeing the amazing ways that they implement these guidelines, these standards.
Christa: How so? Do you have any examples that you can share?
Jesse: Well, it’s like, you’ll go, and you’ve got 17025 in front of you, and you’ve got whatever the accrediting body that is issuing the accreditation under that standard, they might have their own guidance documents. ANAB is who I’ll get to assist on occasion, and they’ve got a document that is in kind of supplement to 17025.
So you’ve got those in front of you. And then you’ve also got that laboratory’s documentation in front of you and you’re trying to go through and you’re trying to figure out, “Okay, how is this particular laboratory implementing the guidelines and the standards and the supplemental documents? Where is that in their written documentation?” And then when you go on site, can you see them actually practicing what is written in their document?
That’s really the core thing. That’s an external assessment that we do, which is something that may not happen if you’re not, let’s say, formally accredited. You might be loosely following some sort of guidance, but do you have somebody then coming in and checking routinely to make sure you’re following that guidance that you’ve implemented?
And that’s one of the key components of most accreditation is: you’re going to be checked! We want to make sure you’re actually doing what you’re saying you’re doing. Not only are you following the standard, but you’re following your own specific procedures, or in addition to whatever are in the ISO guidelines there.
So, that’s kind of my take on that there. And it’s partly specifics. I mean, a lot of times, you’re just paying attention. You go in there, you’re trying to separate yourself from how your laboratory operates. So you’ve got to go in there with kind of a clear mind and it can’t be a lot of, “Well, I do it this way at my lab.” It’s always in the back of your head, and a lot of times it’s just kind of… especially if you’ve encountered an issue with how you’re handling an aspect of accreditation, you’ll maybe pay more attention to that in another laboratory to see well, “How are they handling it? Is that accepted here? Okay. Maybe I could kind of implement that back at my laboratory.” So it’s a lot of that kind of stuff, which is great.
It’s always interesting to see how is somebody else handling the same problem that I’m dealing with? And you have the occasion of having some really nice conversations with people when the day’s over and your creditor assessor, I should say, “hat” is off. And you could really kind of get into it and say, “Hey, I really liked what you were doing there.” Or “Can you explain that one a little bit more to me? I’m not sure I’m getting why you’re doing that,” kind of a thing. And it’s just a nice thing to be able to do.
Christa: So again, going back to the pace of technological change, how often is this kind of process going on? Is this an annual thing? Or is it like every five years, or is it sort of arbitrary based on…?
Jesse: So, I mean, we have annual internal assessments that we do, which is also another requirement there. You’re looking internal to yourself to make sure you’re following what you’re doing. And there are different ways of approaching that, but you’re trying to take a close look at yourself, stepping outside of yourself, if you can there. And looking across things and say, “Hey, are there things we should be doing better? Are there things that we say we should be doing that we’re not doing? Are there things that we’re saying we should be doing that we’re not doing, that we really can’t do, and we really shouldn’t have written our procedures that way?” kind of a thing.
So it’s a constant thing. So, I mean you’re on paper of maybe having to do that annually, and then you’re going to have your external assessments where somebody, an assessor from other laboratories, are going to come in and review your system. That’s time-based as well. It just kind of depends on your accreditation cycle. Every few years there in your cycle, you’re going to have this external assessment there. And that’s where you’re going to get reaccredited, ideally, because you’re following everything you’re supposed to be following there.
But in between those points in time, you should still be having a constant look at your procedures. Again, like I mentioned, I’m in the midst of updating our own procedures here now. That’s just, now was the time. It’s not because I’ve got a specific deadline where I’ve got an internal audit or an external assessment or something like that coming up. It’s just, “Okay, we’ve been discussing this particular thing for a while now. As a team, we’ve reached a decision, let’s try to implement some change in our procedures now.”
And I mean, that’s usually always the goal there, is to constantly be better, to adapt. There’s things that we decided we could do a few years ago that we realized now, “Okay, we can’t really do that anymore. So it’s time to remove that from our procedures, and it’s just not something that fits anymore.”
So it’s always being open to change. And I think anybody that works in digital forensics is used to having to adapt and pivot, because nothing’s really ever, “That’s the way we always do it” kind of a thing, because things like you said, things change so quickly. New technology, new capabilities and things like that.
Christa: All right. Well, great discussion, Jesse. Thank you again for joining us on the podcast.
Jesse: Well thank you for having me. Great topic, and I can’t wait to kind of hear more about it.
Christa: Yeah. Likewise! Thanks also to our listeners, you’ll be able to find this recording and transcription along with more articles, information and forums at www.forensicfocus.com. If there are any topics you’d like us to cover, or you’d like to suggest someone for us to interview, please let us know.
