The following transcript was generated by AI and may contain inaccuracies.
A quick word before we get into the episode. The Forensic Focus State of DFIR Survey is open now, and we’d love to hear from you. It covers caseloads, AI, tools and careers. It takes about eight minutes, and it’s anonymous. Everyone who completes it can get early access to the report, and you can enter our prize draw too. Take it at forensicfocus.com/survey. It closes on the thirtieth of November.
Desi: Welcome, everyone, to the Forensic Focus Podcast. Wherever you’re listening to this from, you can grab us on any podcast platform, on YouTube, and on our website as well, forensicfocus.com. You can also grab the transcript from our website if you enjoy reading along or you want to look into something interesting that we’ve said.
Any links or anything that we talk about will be in the show notes wherever you are, so you can check those out while you’re listening or after the show. This week, Si and I are joined by our good friend Rob again. I was initially disappointed, because I thought he was coming on with a new book, but…
Si: I know. This is a huge loss.
Desi: I know.
Si: First of all, my bookshelf is lacking. The space up here! So get on with it.
Desi: So no new photos of new locations, unless it’s from the old books. But we’ll get into what we’re here for today. And we can see them in the background – we’ve got miniature versions.
Si: Oh, you’ve got mini ones! Talk about false perspective – I just thought those were further back. That’s brilliant.
Rob: My dad got me these as a gift, and I told him, “Those are magnets. Those aren’t good for a lab. They stay far away.” But my colleague Andrew is on, and he helped write the intro for this particular book here, the last one. So we have a little bit of a celebrity here with Andrew being here as well.
Desi: Yeah, it’s fantastic. Before we jump into the specifics of why we’re here today, Rob, Andrew, why don’t you give us a bit of background and introduce yourselves for the people that may be listening for the first time, or who may not have caught one of the last episodes we’ve had? Rob, why don’t you go first?
Rob: Sure. Thanks again for having us on today. Really appreciate it. I’m Rob Fried, and I’ve been in the industry for approaching 25 years very soon. Historically, I’ve done e-discovery for the majority of my career. I started off teaching at the National White Collar Crime Center in West Virginia, then got into the e-discovery world back in 2005 and have been doing it ever since.
Most recently, I’ve been at the firm Page One for the last year. As you guys have said, I’ve done a lot of writing, thought leadership, speaking at conferences, things like that, and I’m excited to talk about my newest creation with you all today. I’m based out in New York, and happy to be on with you guys – we’re truly global today with where everybody is.
Andrew: Absolutely. Hey, everybody, welcome. My name’s Andrew Milauskas. I’m the COO of Page One, based in Southern California, so we’re literally all over the world in many different time zones here.
My background’s mostly e-discovery based, so dealing with data and electronic cases. I’ve been in the space for a little over 20 years. I’ve worked in paper shops and been part of e-discovery teams for pretty close to 20 years, and I have lots of Relativity experience.
For those in the forensics world, Relativity is the largest legal platform used for data. We work with a lot of forensics data coming into the e-discovery realm for law firms, corporations and investigations. With Relativity, I’m a Relativity Master – without going too far into it, I have lots of different certifications there – and earlier this year I was named an AI Visionary by them. I’m using lots of different AI tools within Relativity every day. So that’s my background.
Desi: Awesome. Thanks for that. I don’t think the episode’s come out yet, Si, but we were talking maybe two episodes ago about how, a year ago, we were saying AI’s not going to come into digital forensics. “No way is it touching anything we’re doing.”
Si: This is my predictions being so deeply wrong.
Desi: Yeah, we’ve been truly wrong. And it’s been very interesting to see how everyone is using AI, which leads into what we’re talking about today. We were having a bit of a chat beforehand, but I think a good place to start is – Rob, do you want to give us that initial rundown you were explaining to us about what this project is, why you started it, and why it’s exciting for you guys?
Rob: With all the AI models and solutions that are out, it’s obviously caught my attention from an investigative standpoint. I was a little bit iffy. I’m old school in a lot of ways – I like to see the locations and see everything for myself. But then I started teaching at the university level, and I saw the students starting to use it, and it got me thinking: “How can I use this in a way to give back a lot of the information?”
I’ve written five books, and I have lectures that I’m giving in these classes. When I was teaching, I was getting emails and phone calls, and throughout my career it’s always been repetitive types of questions. “Explain to me how a remote collection’s done.” “Is there a good possibility that we can get deleted data off of cell phones?”
As you’re writing these workflows and these responses over and over again – that’s partly why I wrote my books. But not everybody’s going to be able to have a copy of the book 24 hours a day, and they’re not always going to be able to read a book.
So how can I make this a living environment, where people can go onto a website and use that website as a portal into my brain and my content? So I created forensics.expert, and forensics.expert costs you nothing. You put in your email address, you click a couple of buttons, and you’re seconds away from getting an answer 24/7. You can even try it now.
Si: We’re all typing now!
Rob: It’s been an amazing thing. There are a lot of ways to do this these days, but a lot of times you get hallucinations and everything else, so I really built the model around keeping it to my content. I put it out there and immediately started getting feedback from the community about some of the hallucinations that were still there, and I listened to that.
I love critical feedback, because that’s the only way we can get better, and it allows me to sleep better at night knowing I’ve put something out there that people are going to want to come back to. I’m able to understand who’s using it – what sectors people are coming from – based on the email address they provide.
The feedback’s been super, because people are giving it a rating: how’s the response? Do they feel like it’s giving a valuable response? It can do a lot of things, but it stays within my content. It doesn’t go beyond the content.
I’ve seen government agencies use it, law firms, practitioners and students – and I’m proud to say also our competition. That’s the highest level of compliment. People are coming back – enough that I had to put a limit, unfortunately, on how many questions you can ask, because it’s coming out of my pocket.
I wanted to write another book, because I always want to impress you, Desi, with some more content, but this is the next best thing.
Desi: I love to see the workflow and where your ideas went. You said not everyone can have the book on them all the time, so you tried mini books first in the background. And then you were like, “Okay, mini books don’t work, because they’re too small to read. I’m going to do a website instead, because everyone’s got a mobile phone.” It’s great to see that evolution. The innovation there is fantastic.
Rob: Absolutely. There’s a lot of technology out there now that lets people do this type of thing, but it’s about putting the time and effort into building something that is solid – something people can go to, knowing they’re getting my brain. I trained it, so anytime I give a talk or create really good content that I feel is worthwhile to share, I take those snippets of information and put them into the model so it stays current.
I brought up Flock cameras a few weeks ago, because that’s another hot topic. I’m speaking at a lot of PI associations, and I got on the cover of PI Magazine with it. It’s just taken off, and it’s another example of the different ways people are using AI.
I love educating people, so I’m using AI not only to help answer people’s questions, but also to give back and have an educational resource that’s free of charge. There’s no fee.
Desi: It’s one of those cross-domain things – it’s hard to jump from digital forensics to creating something like this. Now that you’ve shown me, I’m like, “This makes complete sense.” Put all your knowledge – your brain, your books, your lectures, everything – into a database that can be queried by AI.
I’ve seen this done so many times by people in marketing, who say, “All my sales calls, everything I do in marketing, I’ve put it all into an AI engine and you can query it.” And they sell it as a product – that’s essentially what you’ve done.
I just never thought of doing that for professionals in the forensics or incident response field, where it’s curated, you’re working out the hallucinations, and you’re trying to make it a source of truth – if someone asked you a question, this is what you would send back in the email. It’s like an advanced FAQ.
Rob: That’s right. And I personally use it sometimes when I get those questions, because it is my content and I know it doesn’t go beyond my content. If you ask something off-topic from forensics, it’s going to say, “This is not within Rob Fried’s content, so I’m not going to be able to go there.” Look at Si trying this!
Andrew: Let me try to break this on this call.
Rob: Be careful, Si.
Si: Well, the thing is, if it’s filled with the content from your books, it should know something about you. So I’m going to see what it knows about you.
Rob: There you go. It will tell you. It won’t give you my home address, but there are a lot of smart people out there who make this stuff.
Desi: I’m interested – have you fed it the transcripts from all the podcasts you’ve been on? Because then it’ll have information about Si and me.
Rob: It depends. What I do is extract major concepts and fill in gaps from them. I don’t really like to go off a lot of my written content, but I do keep current with topics that are trending. Right now, a lot of people are talking about the transition from digital forensics to e-discovery.
That’s why my colleague Andrew and I have been collaborating on a lot of different ways we can leverage AI beyond what the forensic tools are giving us, because of the sheer volume – and the expertise Andrew has to really make this happen in a very short period of time.
We just had a major project where the timeline got really condensed, so we had to shift gears a little bit, and having that expertise in-house to say, “Hey, look, we need to pivot,” was a perfect example of AI in use. We’re so busy talking about AI in terms of digital forensics.
My opinion on the digital forensics side is that there’s a lot of software being developed right now, but there aren’t a lot of focus groups. We’re hearing about AI being built into the tool after it’s already done. Everybody wants feedback, but let’s talk about how to use it today – all the different use cases, the public sector versus the private sector, what the differences are, and how this can be applied. It’s a very worthwhile conversation. It’s not going away.
It’s just about how you want to use it and what you feel comfortable with. I thought forensics.expert was a really good way to give people a sense of using an AI model built by somebody who’s been in the field long enough to put out really good information as a starting point.
This is not where I want you to end your research, but it’s a good way for you to at least learn. When I started in this field, I wasn’t law enforcement. I was told, “Hey, figure this out.” There were no books on e-discovery when I got into this field. So this was a way for me to put my books in an environment that gives anybody access. We even have our own clients using it, and everybody’s thankful for it, because it’s something that hasn’t been done yet, and I’m proud to bring it to the table.
Si: It’s really cool. I’m going to show my age here massively. I actually read artificial intelligence at university, but long enough ago that all of this was a pipe dream.
One of the things we did focus on a great deal at the time was what were called expert systems. We viewed them as this amazing thing whereby, instead of going to talk to a doctor, you would type your symptoms in and it would tell you that you’ve got the plague. Hot topic at the moment if you’re Russian.
What we didn’t have was the large language models to take in questions and put out responses. If you asked the right question, you got the right answer in exactly the same way every time. What we’ve seen is this fantastic leap to the point where we have an expert system populated with your knowledge that you can actually ask a sensible question of, and it will tailor a response to your question, rather than generically spouting some rubbish about something it thinks you might have asked.
Perhaps this is where ChatGPT and Claude still aren’t great for medical things, because if you say, “Have I got cancer?” it’ll tell you 15 ways you’re going to die before breakfast. Whereas this condensed, properly limited set of knowledge will give you such a good answer within the scope of what you’ve set it to do. I think that’s a really cool thing to take forward.
Rob: And I’m really thankful to the people who have approached me and said, “Hey, I used it and I’m concerned about this.” One example was paths. If you’ve ever read a lot of the content I put out there, I don’t like to talk about tool versions or paths of things. I write things so that they’re timeless.
Desi: Mm.
Rob: It’s about the concept, because everybody has to learn how to drive their tools and understand how they get you from point A to point B. You give people the foundational knowledge, and I’m all about the foundational.
That stuff’s going to change with every version of the tool. So putting tool names or specific data paths in there – that’s for you to be in the moment and appreciate where we’re at with the operating systems and things like that. Foundational knowledge will always give you the insight to be able to pivot as you need.
Si: It’s funny you use the word “drive,” because it’s exactly that, isn’t it? We’ve all gotten in a car and learned that this pedal does this and that pedal does that, and then it doesn’t matter – well, it might in some cases, but generally it doesn’t matter what car you get into, you’re still able to put those concepts to good use and not kill yourself or anybody else.
Again, showing my age, I learned to drive over 30 years ago, and I drive a car now that wasn’t even invented at that point in time. But I can still drive it. So the foundational concepts are so important to have.
Desi: I wanted to jump to Andrew. What kind of background did you bring to this project, and what were some of the challenges that came up in introducing this?
Andrew: From my perspective, this is mostly Rob and what Rob did. I tested it a few times with some of the questions I ask Rob all the time – he probably gets sick of me asking him repeatedly, which is probably why he created it, let’s be honest. “Andrew, you’ve asked me this question 30 times. You should know the answer by now.”
But no, it’s phenomenal. You can type in anything forensics-related and it comes back with an answer. To your point about hallucinations: in the legal space, attorneys are using AI to write briefs, and they’re getting citations and getting fined by court systems – I’m sure all over the world, but certainly in the United States, which I know of – because they’re filing briefs that don’t have real citations or real cases in them.
So, to your point on the medical, the legal, the forensics – the problems are the same. At the end of the day, somebody has to verify that the information is real and that they’re getting the right results back.
Rob: It’s a big part of it. We see a lot of thought leaders out there speaking about verification and being able to look at things. For example, there are so many disclaimers too: don’t put real case data in here, be careful about what you’re getting back, it’s truly just educational content.
But I also built in some of the things that come up in conversation when we’re looking for AI content on devices – like, did you try to replicate what the AI model was giving you? If you put in a question, it will log your session: “I put this in at whatever time,” and it continues on and gives you that information. So at this point, when you’ve used the portal, all your stuff is documented.
And then I went a step further. You can put it into a PDF, you can copy it as text to somebody, and you can send it off in an email, and it’s all branded with that information already – who you got it from, what time you did it, and everything else. Look at me, like a kid in a candy store.
Si: No, I’m terrible for this sort of thing. Whenever Desi and I are talking and he mentions something, I’m off on it. I had to get a special shock mount for my microphone so you can’t hear me typing so much. It’s terrible.
Rob: These are all things where, if somebody has a question for me, I’ll be able to respond and put it in an email. But if somebody’s using this at 2:00 in the morning because they’re trying to wrap something up and research something – you can even use this to prepare to testify and look at things in different ways – then you can send it off as a PDF. And you can reference where the information was gathered from in the content I uploaded and based the model on.
So everything is referenceable. It’s also recorded – when you’re using it, I keep the documentation in mind, which I always speak about as part of the process of doing any of this work.
Desi: I’ve got some feedback and then a follow-up question, because you mentioned case details just there. You do have a disclaimer on the site, but it’s in the Connect / Speaking Engagements section, so maybe move that up above the chat prompt, so people see it as they’re using it, because it’s all one page.
My question, though, is: if someone does inevitably put sensitive case data into this chat prompt – which someone at some point will, because we all don’t read at some point – what is the back end of this? What happens to these chats? I can see that if I ask a question, it gives me an answer, and if I ask a follow-up, it goes into the same chat. Is that chat cleared or not saved if I go away from the page? What happens with that data?
Rob: It’s out there – it is currently saved. If we want to clear that, I can certainly put something in there to do it. That’s a good point, and these are good feedback points. That way, there’s no record of it. But again, you should have a record of it, so there’s a trade-off.
If you want to keep track of something for this session, it’s got to be put somewhere for you to get it back. If it gets cleared at the end of your session – maybe that’s something I can build in. This is why this feedback is so great. But this is where we’re at with what people are doing right now.
I had the same thing with students. Students are putting course questions into this and giving me back answers to their prompts, and you can clearly see there was content there that they probably gathered from somewhere else. So I can definitely appreciate wanting to limit how much is stored out there.
But there’s also the benefit of being able to capture it in a way that your session is logged, so you can document your steps – otherwise you’d be taking screenshots.
Si: The PDF download is very tidy, I have to say. And it does come with a disclaimer at the end that says, “For informational purposes only, not legal advice. Use at your own risk,” which is very reasonable. It’s very tidily done.
Rob: I even had a colleague from the UK, now living here in the States, ask, “What about GDPR or privacy?” So I had to go back and do all that too. Again, this is all coming out of my own pocket. I’m trying to give back to the community, and I’m 100% passionate about keeping it free, with no cost associated with any of it. But I definitely love the feedback.
Desi: Do you need a “Buy Me a Coffee” or Patreon link at the bottom? While it’s free, people could still chip in. There’s Eric Zimmerman’s tools, for example – you can use them completely for free, but if you’re a commercial organization, cough up some money, kind of thing.
Rob: I got you.
Si: ExifTool’s another good example of that.
Desi: And WinRAR. I don’t know how they’re still in existence, because no one pays for that. But they sell shirts now, so maybe you can sell T-shirts. Have an Etsy store.
Rob: There we go!
Desi: I do like the saying you’ve got: “Data points to the truth.”
Rob: Yes. The idea there is that it’s really data points. You have all these data points that you’re trying to put together when you’re doing investigations, and taking all that information – all the volume of data right now – and putting it together helps point to the truth. Sometimes that’s truth we want to hear, and sometimes it’s truth we may not want to hear.
But all together, that’s where it’s going to get us. This all comes back to validating your work, and taking all the knowledge in the books, the lectures and the presentations over time, and helping people connect the dots to hopefully give them a successful outcome.
Desi: That’s awesome. There’s also a quiz on here – Si and I are both perusing through. It’s right at the top. You can do a knowledge assessment.
Rob: I think Si’s taken that a few times now – that’s why he keeps going back.
Si: I didn’t even notice it. Sorry, I jumped straight down to the meat.
I went to the F3 conference – I was going to embarrass some people by saying this – last year, or was it this year? I can’t remember. Time is a social construct that has no meaning anymore. I sat at a table with the chairman and a few of the directors. F3 is the UK’s First Forensic Forum – it’s an organization for digital examiners, police and so on.
I sat with some quite senior people and we did a quiz, and we didn’t do that well. It was quite embarrassing, but we won’t talk about that. I do like a good pub quiz, though.
Rob: Test your knowledge. You want to be grounded with all this stuff and know that you have a lot of ways to get information right now, but test yourself. Figure out if you’re getting the basic concepts down, because that foundation is what’s important. I always look for ways to stay current on everything by taking information like this and still going out and looking at another source.
So take what I gave you and take it another layer. Look at some of the other books that are out there as well. My interpretation may be slightly different, but like I said, it’s a good starting point. Test your knowledge, test the responses you get from here, and go forward in your efforts to either work your case or pursue a career in digital forensics – whatever you’re visiting for.
Si: That’s pretty much a good statement for AI in general, though, isn’t it? Where we’re at with it is that it’s not a definitive answer, but it’s such a good starting point now. We used to do a Google search, click through 50 things to find the information we were after, and then cross-reference it with another 50 things.
And now we type in one query and somebody does that for us – somebody, there you go, personalizing it. Something does that for us and gives us those results upfront. We still have the obligation to go and verify it, but it’s streamlined that process so much. I think that’s really incredible.
Rob: We’re seeing it across the board in all segments of our operation. Andrew can tell you how many times it’s saved people so much time, because we’re all working to deadlines, and like I said, this gives you access 24/7. In the practicality of it for investigations, we’ve seen it pay dividends.
There may be some hesitation initially, but then you’ve got people like Andrew and other members of our team who have used it and have seen really good responses from it.
Si: Managing large data sets is definitely something it’s hugely valuable for. I still have reservations about AI – many reservations – in terms of forensic investigation. And that’s actually where people are using it for transcription.
To explain it in a slightly different way: Amped FIVE – the video analysis software from Martino Jerian’s company, Amped – has a tool for decoding quite corrupted number plates. It’s really good at it. I’ve seen it demonstrated and I’ve seen the work on it.
But the thing is, it’s a guess, and it’s a percentage probability of it being right. They’re very, very clear that you should have an idea yourself of what you think it is before you look at the response it gives you, because you immediately bias yourself the second you see what it thinks it is.
So you remove that by not biasing yourself. When you’re doing transcription, if there’s a word which is slightly unclear or dubious, or possibly has synonyms, and that’s transcribed for you, then when you read the transcription, you’ve introduced bias into your reading of it.
This is really well demonstrated by song lyrics, because some of us consistently sing along to the wrong lyrics, because that’s what we think they are. It’s only when somebody explains it to you that you go, “Oh, shit, I’m such an idiot.”
The risk for audio transcription is very, very real there. I’ve seen some tools that put automated subtitles on videos and things like this, and I’m very concerned about that going forward. And I’m going to slightly pick on you, Andrew, but you did say that you were using it for video analysis – big, bulky video analysis. How are you dealing with that sort of risk of bias?
Andrew: You’re absolutely right, and you have a few different options to look at it. You say transcription – I’ll even throw in medical records. Handwritten medical records.
That’s something we’re coming across too. If you look at a typical medical record here in the United States, you have check boxes, “circle all that apply” – check all of these things that you have: high blood pressure, cancer. All these check boxes.
So the question we’re coming up against is: what are your options for all of these? Your options are either to review it all manually – and to your point, if there are five videos and five medical records, sure, go ahead and review them. But what do you do when you have 50,000 or 100,000 records?
We’re working with cases where it’s not just one person’s medical records. It’s a hospital system’s medical records. Hopefully most of them are electronic now, but going back 10 or 20 years, they’re scanned and handwritten, and the handwriting’s awful.
OCR obviously isn’t going to work on handwriting, so you have all these options, and it’s really tough to make that call. Do you go through everything manually? Do you try to leverage AI?
To your point on transcription, let’s take the word “window.” You use the search term “window” to look for these files. Maybe you use AI to come up with synonyms – glass, windows, or many of the other words you could come up with to describe that pane. Maybe they just used “pane” without “window.”
So you can use AI to catch those variances. It’s a way to get through things that we didn’t really have answers for, where the only answer was, “Hey, it’s manual.” There’s definitely risk in it, and nobody should tell you otherwise.
But if you have mounds and mounds of it, I think it’s one of the answers, or at least you can use it as a starting point. With all that said – and I’ll get off my soapbox in a second – go back and try to validate as much as you can.
Test documents. If we have 50,000 medical records we’re looking at, maybe you grab a sample so you can try to catch how people speak or what types of forms you have. And there may be other outside-the-box solutions you can use.
Si: And I imagine in medical records there’s some mileage in the fact that the language used is probably a reasonably finite set, generally speaking. So a squiggle that looks like “paracetamol” is probably paracetamol, rather than whatever else it could be.
Andrew: Oh, for sure. As an example, we had some medical records, and we used an offline LLM to generate text from them, and we were analyzing the results, especially the first time we did it. With some of the handwriting, I can’t even read it just looking at the page.
Si: I was going to say, medical records is unfair. You picked doctors as your sample set – you’re already at a huge disadvantage.
Andrew: You’re like, “Is that a three? Is it an eight? Is it a zero?” The handwriting’s so bad. Sorry to any doctors out there listening – terrible handwriting.
Si: That’s brilliant. I’m going to pivot ever so slightly. Obviously, you’re running a local LLM. At the moment, as we’re all painfully aware, hardware is astonishingly expensive to buy and fund. Desi is crying because of his need to move computers from Australia.
The idea of selling them and buying new ones isn’t a viable option, because it’ll cost twice as much as shipping them, which is daft. But when you’re talking about 50,000 or 500,000 medical records, what class of machine are we talking about for you to actually do this?
Andrew: That is a great question. I have somebody on my team who’s much more knowledgeable about this and would be better placed to answer it. I know we’re building a VM through Azure, and as far as RAM, GPU and CPU, it’s up there, and it’s not cheap. But I don’t look at every bill that crosses my plate, so we’ll pretend it’s high, but not terribly high.
Si: You’re still making a profit – that’s the important thing.
Andrew: That’s the one that matters. For sure. Absolutely.
Si: Well, I’ll direct this the other way, then. Rob, what’s forensics.expert running on in terms of hardware?
Rob: It’s all up in the cloud, and it’s got a back end. I pay, essentially, per query. All the heavy lifting is done in the cloud, not on my own hardware, fortunately.
Si: Right.
Rob: Anytime anybody puts in anything, it’s pennies on the dollar, but that’s why I’m trying to limit it, especially offering it free of charge at this point. If it grows where there’s a need to up it – if the threshold gets to a certain point – then I may put up a paywall. I’m hoping that I don’t need to. But for my stuff, it’s all cloud-based.
Si: You heard it here, ladies and gentlemen: please do not abuse Rob’s system, so that we can all keep it free, as opposed to having to deal with a paywall and/or read through 50 adverts.
Rob: You may force me to write another book sooner than anticipated! I still write articles, which will probably be put out there. If this thing blows up, it’s great. But like I said, I just hope it’s useful for people. I want to get the word out, because it’s something I spent a lot of time on – I was in the doghouse at home a little bit: “What are you doing? What are you doing?”
But to come away with another thing I can put out to the industry that’s useful to people, and that people are using – and like I said, the biggest compliment is when our competition is using it and coming back more and more – makes me feel like it’s accomplishing what I set out to do. That’s to help raise awareness, and to give people a solid place to look as a first step in their investigative or digital forensic informational journey.
Si: I think it’s an interesting sign of the current world that there is an almost infinite amount of data, but not necessarily that much information. A trusted source is far more valuable than as many untrusted sources as you like, and I think that’s where you stand apart: you are a trusted source.
People will come to you either to get started or perhaps to verify something they’ve read elsewhere, to make sure they’re in the right place. That’s down to your own skills and your own reputation, which have taken years to build, and you’re giving them away in a very generous way, which is very kind of you.
Rob: Happy to do it. When I was in school – and I can’t age myself too much, I still have a baby face – I was taking organic chemistry and sitting in those labs. They’d give you the formulas, and you’re like, “Okay, what’s the derivative of this?” You actually had to do the lab.
Si: Yeah.
Rob: Now you just need a screenshot, unfortunately. The world is different right now, and a lot of information’s at our fingertips. But if I can give somebody the encouragement to see that this is a discipline, more so than just question and answer – that there’s reasoning behind things – that’s the goal. It doesn’t answer every question, because there’s stuff I haven’t put in there yet, but at least it will show you the discipline in my head on how I would think about next steps.
Si: Hmm.
Rob: And I have mantras in there, like “No documentation, no evidence,” to keep people honest with themselves about how they’re seeking out information and how they’re using it. My mantras are right there with me.
It’s almost like, “Hey, you can use this, but just remember it’s on you. You’re going to be up on the stand. You’re going to be the one asked by your client, ‘How did you get to this conclusion?'” So I make sure that’s ingrained in some of the responses.
Desi: This is the last question I have for this session. How are you finding the students you teach at university are using tools like this, and what’s the level of critical thinking?
We see professional lawyers use these tools and not apply critical thinking to what they’re putting out, because they’re not cross-checking or referencing, even though it says, “This may hallucinate. Check the answers.” What’s your experience been on the front lines of seeing people use it?
Rob: All my tests were handwritten. I remember walking out of class and telling Andrew, “Hey, they’re getting a sheet of paper. This is not online.” Because I think a lot of that is starting to go off the rails a little bit. I think people are too dependent. When somebody asks me a question, my first instinct is not to go and look on Google or ChatGPT.
It’s to go into my memory banks, and the way I do it is to visualize being in that moment before and how I addressed something. With a lot of the kids, as soon as I gave them assignments, you could tell – there are bullets, it looks like an outline. Or when I ask a discussion question, five of the 10 answers all start off with the same thing, maybe slightly tweaked.
Maybe they’re putting in, “Take this answer and make it a little bit different.” I’ve also heard of professors putting white text in their assignments, so that people cut and paste without noticing it, and then hand in the response – with “Mary Had a Little Lamb” embedded in it, and they didn’t even know.
So I think this generation’s in for a little bit of a surprise: there are critical thinking aspects that you’re only going to get if you’ve actually delved into the material.
That’s why practical exercises are really important. I looked at how many people were doing the practical exercises that were part of the class – actually having to go through the steps. Having friends who are professors right now, I empathize with them having to do that full-time.
I did one class and I’m like, “Wow, this is an eye-opener.” But this is the next generation. We really have to think about the skill sets people are walking out with, and how they’re actually showing that they’re gaining knowledge from these courses and the material they’re walking away with.
Si: It’s interesting, because it’s just come out under EU law that you have to watermark AI-generated products, and that includes text. ChatGPT and Claude are the two I use, and therefore the two I looked into, and they’re both going to watermark generated text in some way, shape or form.
They’re willing to share the detectors with authorized people, and universities qualify as authorized people, so that’s up there. If nothing else, it’ll teach the students the skill of setting up a local LLM!
Rob: I’ll be honest with you. When I had to do my syllabus and I was given the template, I said, “I remember a syllabus being two pages.” This syllabus was 15 pages. I did the syllabus and I was all proud, and they said, “You need to choose your standpoint as a professor on the use of AI.” I said, “Oh, I thought that would…” “No, you have to choose what you want to allow.”
It was a very interesting thing. It used to be whether or not kids could use a calculator – that was the kind of support we were able to get. Now you really have to put your stance on everything, because everything is an open book right now. All they need to do is go to different areas of the internet, and they have all these different tools at their disposal.
But my friend told me he creates assignments where, if students use AI, they’re not going to be able to get the response based on how he teaches something, because he teaches a certain method. It’s like when our kids come home with homework now. I thought two plus five is seven. No, you’ve got to show how you got there – you’ve got to do all these branches. I can’t even do my eight-year-old’s homework right now, because they’re teaching it differently.
Si: Yeah.
Rob: They’re keeping us all stressed out at home.
Andrew: You just take a picture and ask ChatGPT to do it for you, Rob.
Rob: That’s right! My son said to me the other day, “Dad, this is such AI slop.” And I was like, “Wait, is that a new word?”
Desi: Yes!
Rob: These kids are learning it. He’s 11 years old.
Desi: “Slop” was Merriam-Webster’s word of the year in 2025.
Rob: I’m getting old myself. I’ve got to keep current. I’m joining the club pretty quickly, Si.
Si: Oh, dear. That’s good stuff.
Desi: Si, do you have any more questions for these gents?
Si: Oh, millions, and I could keep talking for hours. But I think I’ll draw a line under it today. We will get you back, though, and interrogate you about AI poisoning, detecting AI when it’s been used by other people, and a whole raft of other stuff. We’d fill another hour easily, so let’s call it quits today and arrange another time for you to come back and cover the rest.
Rob: Absolutely. Always a pleasure, guys. Thanks for having us.
Si: Always.
Desi: Definitely appreciate it. All right, Si, do you want to take us out?
Si: Oh, yes. Well, everybody, thank you very much for coming along and listening to the Forensic Focus Podcast. As Desi said at the beginning – although you’ve listened all the way through – we’re available on a number of platforms, one of which you’ve found us on. But if there’s something you prefer to listen to us on, or you happen to have stolen somebody else’s podcast player, you can find us on any of the good podcast platforms, and probably some of the bad ones as well.
It’s been an absolute pleasure to have you both on. Rob, again, thank you so much for being so generous with your time and your knowledge, and for sharing it with the world through Forensic Expert – “Forensics Expert,” he says, making sure the transcript gets it right. Dot expert: forensics.expert. Because if it hears you wrong, people will be off somewhere else!
Thank you very much indeed. We look forward to speaking to you again in the future. Desi, have a great day, and I’ll talk to you later.
Rob: Thanks, guys. Cheers.
