Hello all,
I notice that people are using a number of common tools to perform data carving (PhotoRec, MagicRescue and Scalpel or Foremost). However, they appear to be using a specific tool on the basis of the file type and their own preference.
I've been looking for a script or wrapper that combines the functionality of all three tools, enabling a user to run Scalpel, MagicRescue, etc. on a disk image in sequence, exporting the output into different sub-directories and attempt to perform some type of de-duplication (e.g. perceptual comparison of images using the pHash library, text comparison, etc.)
I've done the usual Google/forum search and can't find anything that takes this approach, even the first part of concatenating the output of various tools. Is anyone aware of anything that performs the above described tasks? If not, would this be something that would be worthwhile to produce for the OSS forensics community?
Greetings,
This is about 20 lines with your favorite scripting language. Maybe 30 if you add deduping by hash.
-David
Fair enough. I didn't think it would be complex, but was curious if anyone actually did this.
Kovar, what type of deduping do you perform? Is it the standard MD5 hash comparison or more advanced analysis based upon the content?
Edit Corrected error in text
Greetings,
By the by, I am doing something similar with Python, using subprocess.Popen to execute native utilities.
I'd start by deduping by hash, yes. Given that you're doing file carving, and than different tools may recover fragmented files differently, this is probably only a starting point.
-David
I've been looking for a script or wrapper that combines the functionality of all three tools, enabling a user to run Scalpel, MagicRescue, etc. on a disk image in sequence, exporting the output into different sub-directories and attempt to perform some type of de-duplication (e.g. perceptual comparison of images using the pHash library, text comparison, etc.)
My father used to say "You can probably get the postman to deliver the milk but it'll probably cost you a lot more."
There are at least a couple of very good open source de-duplication systems out there. Why expect that a data carver will do the same?
You could create a script to do this, but I am not how sure it would be, generally, useful. I look for carvers to be as sensitive as possible (no false negatives) which means (especially in Windows systems), that I'm likely to get near duplicates. The fact that they have been recovered from different areas of the media is, itself, significant.
I see de-duplication as being a fundamentally different process from carving since you de-deduplicate only after you have determined that the duplicates have no significance. That is not a decision that I would want to have automation make.