Join Us!

SHA-1 SHA-256 SHA-5...
 
Notifications
Clear all

SHA-1 SHA-256 SHA-512??  

  RSS
mscotgrove
(@mscotgrove)
Senior Member

The software I have developed has MD5 hash values. I have received requests to include SHA-??.

Does anyone have views on which would be considered most useful. SHA-1 is very common, but is the extra security of SHA-256 and SHA-512 actually worth the effort.

Academics seem keen to point out that MD5 and SHA-1 have been broken, but has this ever been critical in a court case?

Quote
Posted : 04/03/2011 3:28 pm
azrael
(@azrael)
Senior Member

Although MD5 and SHA-1 have been "broken" - the ability to create two objects with the same hash that have any beneficial meaning is still infeasible. What you gain by using the larger numbers is a greater time/chance/distance between collisions in the hash space, thus meaning that any two objects are less likely to have the same hashed value - the downside is the required additional computation time of the larger hash values.

To the best of my knowledge, all attacks at this point have been purely academic, and like all pure academia - have eff-all impact on the real world except to worry people 😉

ReplyQuote
Posted : 04/03/2011 4:28 pm
joe_bowman
(@joe_bowman)
New Member

Personally, I would say it is important to differentiate between flawed and broken in this sense. If either MD5 or SHA-1 had in fact been properly broken, we would not be using them - they would be worthless and each and every time MD5 or SHA-1 was mentioned in court, it would be shot down in flames by the opposition expert / legal team and the examiner's position would be indefensible.

Both functions have however been proven to have weaknesses in the algorithm itself, and if the algorithm of either function is ever proven to be sufficiently weak that it is a trivial matter to alter the contents of a file such that it is possible to generate the same hash value, then it will be broken, and forensically speaking, worthless.

On the matter of SHA-256 and SHA-512; NIST have recommended that as of 2010, the SHA-2 family (SHA-224, -256, -384 and -512) should be used in preference to SHA-1 and MD5 hashing functions. NISTs recommendations always have been ahead of the game (I guess they need to be), but my point of view on the matter is such

Suppose that you undertake a case today, and everything is hashed using MD5. It goes to court and the suspect is convicted. In 3 years time, a cryptographic breakthrough finds a critical and indefensible flaw in MD5, and it is proven trivial to generate files that match a required hash-value. As such, MD5 at this point becomes worthless. On the back of such discovery, an appeal is launched, focusing on that critical bit of evidence whose reliability can now be brought in to question.

Given the cryptographic attacks on MD5 and SHA-1 in the past, and their known weaknesses, the hashes will not last forever. As the judicial process can be quite drawn out, is it worth taking a risk that the hash function you are using is not going to be forensically reliable when the case reaches court? In my eyes, we need to look ahead too, and what the effect on us as examiners will be if MD5 or SHA-1 is no more…

I believe that it will be a few (5 - 10 years) before MD5 or / and SHA-1 are proved to be sufficiently unreliable. My advice is to use as generous a hash as you can afford (thinking that larger hashes = more computation = greater number of files on computers now = far more time to hash a case) or alternatively, use more than one hash. If it is likely that one hash will be broken, it is less likely that 2 will be, or even 3. Even on the collisions front, it is highly unlikely that if a collision is found in MD5-space for two values, it is unlikely due to the vast differences in the algorithms that the same two files will collide in SHA-1 or the SHA-2 family.

Sorry for the essay, and I hope this helps )

ReplyQuote
Posted : 04/03/2011 4:43 pm
joe_bowman
(@joe_bowman)
New Member

In the time it took to write the essay, the same point was made far more succinctly)

ReplyQuote
Posted : 04/03/2011 4:44 pm
mgilhespy
(@mgilhespy)
Active Member

Although MD5 and SHA-1 have been "broken" - the ability to create two objects with the same hash that have any beneficial meaning is still infeasible.

Azrael, given that you've made that statement, I'd be interested in your comments re this

MD5 collisions

The work of Daum and Lucks which is referenced is no longer available at the original link, but is still online here

Daum and Lucks

and here

Lucks webpage

Will be great to hear your take on that.

ReplyQuote
Posted : 04/03/2011 7:07 pm
mscotgrove
(@mscotgrove)
Senior Member

Thanks Joe_bowman. Based you what you said, and NIST recommendations I am going for SHA-256. This should be safe until I get my bus pass, even though that has moved back 6 years now.

ReplyQuote
Posted : 04/03/2011 8:41 pm
azrael
(@azrael)
Senior Member

Although MD5 and SHA-1 have been "broken" - the ability to create two objects with the same hash that have any beneficial meaning is still infeasible.

Azrael, given that you've made that statement, I'd be interested in your comments re this

MD5 collisions

The work of Daum and Lucks which is referenced is no longer available at the original link, but is still online here

Daum and Lucks

and here

Lucks webpage

Will be great to hear your take on that.

My issue with these is with regards to how contrived the examples are - they have demonstrated a theoretical attack, and, as a proof of concept, I have to agree that they are quite good. But in reality lets look at what we actually are using hashes for in Forensics - we are using them to demonstrate that a given file hasn't been altered in the process of examiniation/transit/handling - or indeed deliberately to incriminate an individual. If I wanted to plant evidence of a given crime during an examination, say, for a change, of a nice neat fraud case - I would have to fabricate, not only a file that matched the hash, but a file that met my needs, and, most likely several more files that (a) matched the hash, (b) matched my needs and, now, © match the other files. If you look at the "recommendation / clearance" example - both of the files contain, in plain text, both of the letters - so whilst you might have fooled someone giving it a cursory view, you wouldn't fool someone who was looking at a byte level - in this case it is more of an exploit of human falability in believing what you see rather than checking further in depth - I am aware of systems that explicitly will flag content that isn't immediately visible to the user in order to mitigate against this risk.

In other uses of hashes, such as passwords, there is no ability for the attacker to craft both parts of the hash, so although the search space in a brute force attack is reduced ( as potentially more than one passphrase would have the same hash ) this weakness is largely mitigated by the fact that, with a long password with a large character set, even reducing the search space still presents a significant problem for a brute force attack - or - the brute force attack is capable of being run irregardless of the search space size ( using significant resources - HPC or rainbow tables or the like ).

I think that the example that scares me most from the ones given in the excellent links, is the idea that two programs might have the same hash - although I have to say, I find it hard to believe that useful function would be possible to get in the same hash, and, in reality, you'd only have one part of the equation that you could edit ( e.g. your code ) as the other program that had been hashed and was already on the system ( protected by say Tripwire or Sanctuary ) was beyond your control. So you'd be trying to manipulate something to meet that hash that would work.

I do work with secure systems, and we hear about these theoretical attacks constantly, and some people spend a lot of time and money trying to make them work - people who are a lot brighter than me mathematically too - and as of yet, I've not see it presented as a real threat as the exploitation is too elaborate and difficult. I'm really not belittling the very important work done by these academics, and, as a professional on the security side, I'd encourage anyone to use the largest levels of encryption and hash complexity that they can without impacting on the performance of their product and/or system - but in reality there is, in my opinion, no real threat _in court_ or _in the real world_ because of these theoretical compromises.

Kind Regards,

Azrael

ReplyQuote
Posted : 05/03/2011 9:55 pm
Jonathan
(@jonathan)
Senior Member

The software I have developed has MD5 hash values. I have received requests to include SHA-??.

Does anyone have views on which would be considered most useful. SHA-1 is very common, but is the extra security of SHA-256 and SHA-512 actually worth the effort.

Academics seem keen to point out that MD5 and SHA-1 have been broken, but has this ever been critical in a court case?

What's the software designed to do? How does it use hash values?

Without knowing it's not easy to say which would be most useful.

ReplyQuote
Posted : 05/03/2011 11:27 pm
indur
(@indur)
Member

In forensics, what you're usually worried about is a preimage attack. That is, you take the hash value of a file or a disk at one point in time. What you want to show is that no reasonable manipulation of the bits in the file or disk will result in the same hash value that was previously computed.

The weakness that has been demonstrated for MD5 and SHA1 is a collision attack, which is the ability to create two files that have the same hash value. This means being able to manipulate both files. (The collision attacks that have been demonstrated require being able to insert arbitrary, variable-length binary data. In most well-defined file formats, such as JPEG images, this is easy to detect forensically.)

So the risk to forensics of MD5 weaknesses is negligible (other than the risk of having to explain this to a layperson), since the way that the hash functions are used is different than the way they must be used for the weakness to be exploited. The major risk is that, since a collision attack has been found, a preimage attack will be discovered in the future.

ReplyQuote
Posted : 05/03/2011 11:55 pm
mgilhespy
(@mgilhespy)
Active Member

Azrael thanks, I found your follow up post very interesting.

ReplyQuote
Posted : 06/03/2011 1:12 am
markg43
(@markg43)
Member

DOJ currently only requires MD5 hash as sufficient. If you want to future proof a little, then SHA-1 and 256.

In my work, the software I use makes MD5/SHA1 automatically. Works for me, in order to invalidate my evidence you would have to break both algorithms on the SAME file.

To my knowledge, the eggheads have not done that yet. Please correct me if I am mistaken anyone. Preferable with a link.

ReplyQuote
Posted : 06/03/2011 11:44 am
mgilhespy
(@mgilhespy)
Active Member

Can any Australian members comment on whether this ruling back in 2005 has had any ongoing influence?

The NSW Roads and Traffic Authority (RTA) concedes that a court's decision to throw out a traffic infringement case had created "some uncertainty" about speed camera detection.


RTA lawyers told the court they could not find an expert to prove the authenticity of mathematical algorithms published on each picture.

The algorithms known as MD5 are used as a security measure to prove the pictures have not been altered after they are taken.

Link MD5 doubts in Oz?

ReplyQuote
Posted : 06/03/2011 12:17 pm
Patrick4n6
(@patrick4n6)
Senior Member

Aussie and although living in the US I'm au-fait with Oz law and was working LEO forensics in another state in Oz when that case occurred. That case doesn't prove anything other than if you don't have your witnesses lined up, don't bother going to court. I don't see in the article that it was dismissed with prejudice, which means that the govt can get their ducks in a row and do over.

And back to the MD5 issue as posted

Hashing has 2 functions in forensics validating your images haven't broken / been tampered; and KFF. There is no protection in the world that's going to defend you against a malicious examiner, especially not MD5, since if one were to plant or falsify evidence, they would merely need to do so before the initial hash. MD5 is more a check that your images haven't been corrupted, requiring the examiner to go back to their original image and re-make another working image. As for KFF, only an idiot would submit evidence based purely upon a match and not hit it with a Mark 1 Eyeball.

Hashing is always in my experienced combined with another method to verify, e.g. source hash with a good chain of custody process, and KFF with a visual inspection, meaning that more than a possible exploit of the MD5 algorithm is needed to invalidate it.

ReplyQuote
Posted : 08/03/2011 3:34 am
Share: