The question.
Exactly the right question to ask. The answer is, the question. The question, is the answer.
mrgreen
Naah, the answer is 42. wink
It's the question that is tricky….
jaclaz
Paul, good points. However on the "mud sticks" level alone why not use SHA256/512 where practical to do so to save you having to answer these questions again?
Nowhere did i say that I would use them in preference to SHAxxx, what I said was that I would be happy to use them. As most hash libraries are predominately MD5 (still), the point is moot in that I either make use of the signatures or lose that intel.
At the end of the day they are a tool and if use of that tool leads me to a solution then fine.
If I find enough evidence using MD5 (asume that I have also used SHAxxx and it has not found the same evidence), and I can verify it (i.e. the pictures are illegal, the file is malware…), then I would be happy with the results - why wouldn't I be.
In the ideal world we would convert our libraries to SHAxxx, but we dont live in the ideal world (oh that such a conversion was possible, or that we access (and time) to re hash all our librraries) and I am not going to turn my back on an extremely useful tool based on an almost non existent possibility that one, or a small handfull, of files have been maliciously altered. Lets face it, in 99.9% of scenarios the perp is unlikely to have the skills anyway. In the other .1% of cases are hackles will probably already be raised and we will be looking harder.
Lots of replies! Many of them with thoughtful responses. Thanks for everyone's involvement - I'm relieved that it's being discussed, rather than just seeing a series of votes with no responses. )
I mean, it seems you could equally well say 'I have yet to see data manipulated in a significant way, and yet produce the same CRC32'.
Sorry, I think I worded my post a bit incorrectly. I meant I have yet to see "scientific" proof (not anecdotal proof) that MD5 can be manipulated in such a way. As I understand it, currently the attack requires a new MD5 to be generated for both files - the "good" file and the "evil" file. You cannot, for example, take the MD5 of a "known good" file and change your "evil" file in such a way that it will give you the same MD5.
I was thinking further about this last night, and I guess that ultimately this may become a moot point as more forensic tools include SHA. I wonder if, in future years, we will naturally migrate from MD5 as it drops from being the default option in imaging tools.
Files identified by MD5 should always be checked, not for collision problems but for errors further down the chain.
I've seen a case taken to court where a hash from the UK LE C4P database has identified a level 4 IIOC picture that turned out to be a picture of Buzz Lightyear (fully clothed).
The chances of this being a collision in MD5 are 1 in N Million, that it was down to a bored investigator pressing the wrong button, 1 in, er, 1000 ?
Even today, I've got the email from Guidance about the update of Encase to verison 6.19.6 the release notes lists the following
51247 Reuse of the encase.hash set results in incorrect hash matching.
There are plenty of other things to worry about before MD5 collisions become an issue.
MD5 collisions in cryptography are a problem, in forensics, not so much.
Great post!
I had a case last year where the other side said our conclusions were invalid because MD5 was " broken" and they threw the DOD cryptography papers at us. We sucessfully argued that while MD5 was cryptographically broken it was still relevant for file comparison. There is a big difference between the two issues. Ofcourse, the only reason we still use MD5 is because so many of the tools and hash sets on the forensics and litigation support side have not been updated to use the new algorithms.
And i'll reiterate MD5 is fine for what it's supposed to be used for, low-risk data integrity - it's not a HMAC. MD5 is a standardised algorithm, it is well spread and is ubiquitous in some situations.
Yes, MD5 has flaws (already pointed out earlier) - if you want to protect my creditcard with MD5 Certs, i'll take my business elsewhere.
One thing you can do to increase resiliency when using hashes is to produce 2 hashes with 2 different hashfunctions, i.e. MD5 & SHA1 (a lot of downloads nowadays do that) it is VERY hard to produce a legit hash that collides for 2 different hashfunctions that use different algorithms and modes of operation.
Personally, i would never implement MD5 in a NEW piece of code (i got Sha256cng for that), but scriptable tools like MD5/MD5sum are still applicable in a low security world where your only enemy is data corruption. Some (older) platforms does not support Sha256 and newer algorithms, so you have to roll your own on in that world. That makes the easiest way to get something rolling is to use MD5 to get cross platform applications quickly.