±Forensic Focus Partners

Become an advertising partner

±Your Account


Forgotten password/username?

Site Members:

New Today: 3 Overall: 33814
New Yesterday: 10 Visitors: 190

±Follow Forensic Focus

Forensic Focus Facebook PageForensic Focus on TwitterForensic Focus LinkedIn GroupForensic Focus YouTube Channel

RSS feeds: News Forums Articles

±Latest Articles

RSS Feed Widget

±Latest Webinars

Processing thousands of images

Computer forensics discussion. Please ensure that your post is not better suited to one of the forums below (if it is, please post it there instead!)
Reply to topicReply to topic Printer Friendly Page
Forum FAQSearchView unanswered posts
Go to page 1, 2  Next 

Processing thousands of images

Post Posted: Mon Jun 23, 2014 5:05 pm

I have a hard drive with tens of thousands of image files on it. I processed the entire drive with FTK.

There are about 4-6 copies of each image on the drive. I need to find a way to filter and sort images. Ideally, I'd like to produce a report or a folder with a single copy of each image. I'd also like to do things like filter out the jokes, meme images, etc.

It would also be nice to have that report show the image, and give the 5 places where that image was located on the drive. FTK can do some of this using the PhotoDNA feature. File hashing will not work, because the images are not exact copies.

Is there any good program to help me do this?



Re: Processing thousands of images

Post Posted: Mon Jun 23, 2014 8:00 pm

With my software I just deduplicate the images based on hash values. This may remove a large number. Obviously this will only detect exact matches and on most drives there are many such matches.

The next stage is harder.

1) I use my software to skip known files, based on NRSL hash values. This will remove some.

2) As a rather vague stage, one can often skip small files, or files without embedded dates. (Again, CnW software adds in the date of the file from the internal metadata).

If you are only interested in certain files, it should be possible to filter by date, camera type etc
Michael Cotgrove

Senior Member

Re: Processing thousands of images

Post Posted: Tue Jun 24, 2014 1:18 am

Sometimes the hard way is the only way Smile

If this is a CP related job then I would just bite the bullet and start reviewing with minimal exclusions because you never know what will be relevant.

You may even be able to exclude some locations and concentrate only on user folders.

Depending on what your goal is tens of thousands of images is not really that big of a job, it's when they get into the hundreds or millions that you have a hard time.

If you are just looking for particular content (CP or regular porn) then you should be able to get through 30-40k in a single day quite comfortably.

If the pics need more close examination then you are in for a long haul because no software can reliably cut out too much of the leg work here.  

Senior Member

Re: Processing thousands of images

Post Posted: Tue Jun 24, 2014 5:56 am

That is the kind of job Netclean was designed for. It uses PhotoDNA to group images that are visually similar into a visual group. The reporting isn't as good as I'd like, but when it comes to image and video review and categorization, it is top notch.

And, best of all, it is free to law enforcement.  

Last edited by twjolson on Tue Jun 24, 2014 8:40 pm; edited 1 time in total

Senior Member

Re: Processing thousands of images

Post Posted: Tue Jun 24, 2014 6:39 am


you can use Vizx2 from ZIUZ or LACE from Bluebear. These soft are not free but you can ask for trial version. ther are designed to analyze large amount of picturze and video. you can use hash values.
Vizx2 has the ZZ40 hash that detect ‘near duplicates’. Working closely with Microsoft, ZiuZ has integrated their PhotoDNA technology with VizX2 to create an additional means of matching. PhotoDNA is a little more tolerant to changes in an image than ZZ40.



Re: Processing thousands of images

Post Posted: Wed Jun 25, 2014 7:09 am

Reconnoitre has a very advanced reporting engine and can create very customisable reports (which can be saved and re-used or shared).

I have just modified an existing indecent images report template (took about 5 minutes) such that it compares images by hash and only reports the full details for the first of a series of duplicate images. Subsequent duplicate images have just summary information displayed, i.e. Filename, path and VSC number (Reconnoitre automatically processes VSCs and images within them.

A screenshot from a page of the report is shown and the full report (in this case a PDF) can be downloaded at the link below, the PDF shows a few more of the summary features in this report

Summary images report
Paul Sanderson
Forensic Toolkit for SQLite

Senior Member

Re: Processing thousands of images

Post Posted: Wed Jun 25, 2014 1:27 pm

I'll second the Netclean recommendation.

If your not LE and therefore can't get a free copy, I would say C4All is a good option just for viewing the images.

Ten's of thousands of image is actually a fairly low number, the average job in our unit it 150,000ish after known hashes have been removed.  

Senior Member

Page 1 of 2
Go to page 1, 2  Next