zero fill and check...
 
Notifications
Clear all

zero fill and checksum  

Page 1 / 2
  RSS
twjolson
(@twjolson)
Active Member

I'm playing around with doing a zero fill. I've done a 500 gig hard drive and a 2gig flashdrive. However, after I do the zero fill, the checksum doesn't add up to zero. The MD5 hash, for instance, returns a non-zero result. I was under the impression that a zero'ed out drive returns a zero hash. Looking at the disk with a hex editor confirms the drive was zero'ed out.

Assuming my information is wrong. How do you ensure the drive is all zero's, without having to pour through pages and pages of Zeros?

Thanks!

Quote
Posted : 22/12/2009 8:21 am
mscotgrove
(@mscotgrove)
Senior Member

An MD5 hash will be different for every length of file, even though the contents may all be zero. A CRC sumcheck will also produce different results.

The only sumcheck that will return zero is one that justs adds each byte to a number.

The only way to check that it is totals zeros is to add every byte into a long number, ie int64 to make sure it does not overflow. This number should be zero

ReplyQuote
Posted : 22/12/2009 1:31 pm
twjolson
(@twjolson)
Active Member

So, how do you go about proving that in court?

Here is how I heard it. You image the drive, but after doing your examination on the image you want to start up the computer as the suspect did, same hardware and such. You can't just put the evidence drive in, so you do a copy of the image to a new, similiar drive. However, the defense might balk, how do you know that the new, similiar drive did not have old data on it, data that you are now using against their client.

I might have heard wrong, but my curiousity remains. How do you go about proving a zero filled drive is actually zero filled? A program or something?

ReplyQuote
Posted : 23/12/2009 9:50 pm
Patrick4n6
(@patrick4n6)
Senior Member

Pretty much all the live forensics CDs out there implement a checksum that will return a string of zeros if your drive is wiped. Get one, validate its results against a known wiped drive, and then use it every time and you are fine.

ReplyQuote
Posted : 23/12/2009 10:08 pm
ba2llb
(@ba2llb)
Junior Member

So, how do you go about proving that in court?

Here is how I heard it. You image the drive, but after doing your examination on the image you want to start up the computer as the suspect did, same hardware and such. You can't just put the evidence drive in, so you do a copy of the image to a new, similiar drive. However, the defense might balk, how do you know that the new, similiar drive did not have old data on it, data that you are now using against their client.

I might have heard wrong, but my curiousity remains. How do you go about proving a zero filled drive is actually zero filled? A program or something?

Compression of the zeroed out storage device should produce maximum compression ratio and you can prove this for any given-sized storage media. On another note, the Secure Erase utility is fast and completely compliant with U.S. gov standards and practices.

ReplyQuote
Posted : 23/12/2009 10:25 pm
4n6art
(@4n6art)
Active Member

If I am not mistakenā€¦

A CRC32 check of the entire drive that is completely Zeros will yield a checksum of all Zeros.

I don't have my info in front of me but I seem to recall that may be the case. You may want to test it out. This is assuming that the drive has been zero-wiped and not reformatted after that.

Happy and safe holidays and New Year to everyone!
-=ART=-

ReplyQuote
Posted : 24/12/2009 1:23 am
armresl
(@armresl)
Community Legend

If you are wanting to see things like the user did and aren't going the virtual route. Use Safeback and backfill.

Something that is easily forgotten is that any objection has to have foundation, someone can't object just to object. If there is an objection raised you can intelligently say you did xyz and why xyz was the best choice for your sceneraio.

Safeback is a great tool and rarely mentioned anymore, but as long as you get a hard drive which is close to the same size as the donor drive, you will be A O K.

So, how do you go about proving that in court?

Here is how I heard it. You image the drive, but after doing your examination on the image you want to start up the computer as the suspect did, same hardware and such. You can't just put the evidence drive in, so you do a copy of the image to a new, similiar drive. However, the defense might balk, how do you know that the new, similiar drive did not have old data on it, data that you are now using against their client.

I might have heard wrong, but my curiousity remains. How do you go about proving a zero filled drive is actually zero filled? A program or something?

ReplyQuote
Posted : 24/12/2009 2:57 am
mscotgrove
(@mscotgrove)
Senior Member

"A CRC32 check of the entire drive that is completely Zeros will yield a checksum of all Zeros."

I am pretty certain that a CRC32 will not produce a sucheck of zero from data which is just zero. I am also pretty certain that the sumcheck will be different for each length of a zero filled file. The sumcheck starts with an XOR with a value such as 0x8005 (for CRC16) and this ripples through to be different with every byte

ReplyQuote
Posted : 24/12/2009 4:02 am
jaclaz
(@jaclaz)
Community Legend

I am not pretty šŸ˜Æ , but I am certain wink
http//en.wikipedia.org/wiki/Cyclic_redundancy_check

Checksums of a 512 bytes 00 filled file (tiny hexer)
Sum of bytes
00

CRC16 checksum
BB41

CRC32 checksum
B2AA7578

MD5 digest
BF619EAC0CDF3F68D496EA9344137E8B

Checksums of a 1024 bytes 00 filled file (tiny hexer)
Sum of bytes
00

CRC16 checksum
D4BE

CRC32 checksum
EFB5AF2E

MD5 digest
0F343B0931126A20F133D67C2B018A3B

jaclaz

ReplyQuote
Posted : 24/12/2009 8:40 pm
pwakely
(@pwakely)
Junior Member

In my opinion CRCs are not ideal for verification of all-zero space, for two main reasons
(1) while it is possible to select a CRC with properties that generate a zero result for all-zero data, they will do it for any length of data (so does not verify that the full data length has been examined)
(2) CRCs are easily manipulated (i.e. I could easily insert data into the middle of all zero space and simply add a few bytes of extra data to generate a zero CRC). Equally CRC-checked data itself is often designed to generate all-zero result for error checking.

My preferred method therefore is either a direct check (effective programming check of values at all locations are ==0), or if wanting to use tools for ease of use, then to perform a standard hash (MD5/SHA1/SHA256) and compare against the known result for a hash of an all-zero sequence of bytes of the same length as the data under consideration. The "known result" can be generated either by creation of a file of the required size of all-zero binary data and performing the hash on that file, or programatically.

Since I had a software method I wanted to test, I have created this as a simple windows utility, the "all-zero hash calculator" (am sure I should have thought of a snappier nameā€¦) which will provide the "known result" for any specified length of bytes, and included acceleration tables so that the result can be calculated in less than a second for any length up to 2TB. I have made this freely available here. Any feedback welcome as usual, and hope some people find it useful.

Phil.

ReplyQuote
Posted : 28/12/2009 4:08 pm
roncufley
(@roncufley)
Active Member

If you want to prove that a drive is all zero then scan it with a hex editor for a non-zero byte, if the scan fails to find a result then the drive is all zeroes.

A checksum that simply adds all bytes will roll over at some point so a zero result will not prove all zero bytes although you would have to be very unlucky to get a zero checksum with a drive with non-zero data.

ReplyQuote
Posted : 28/12/2009 8:25 pm
pwakely
(@pwakely)
Junior Member

> If you want to prove that a drive is all zero then scan it
> with a hex editor for a non-zero byte, if the scan fails to
> find a result then the drive is all zeroes.

Agreed that works fine; but for very large files can take a long time. The main reason for the app was that if for example you've already captured a drive/device, and when you look it appears to be all-zero, you can check the already-generated-during-imaging hash against the allZeroHash from the app to validate that all is zero in seconds, rather than the hour(s) it would take for a [^\x00] to run across a terabyte image.

> A checksum that simply adds all bytes will roll over at
> some point so a zero result will not prove all zero bytes
> although you would have to be very unlucky to get a
> zero checksum with a drive with non-zero data.

Yep, same is true for the carefully selected CRC - you'd only get zero result if unlucky or from deliberate manipulation. Another option I did consider was doing a multiple precision sum (using the GNU bignum library, offering multiple precision) which would allow summation across 2TB within a 50-bit accumulator without overflow. However while this would be a little quicker than the hash, I like the other benefits of non-manipulation of hash property, and the ability to "quick check" for existing images and non-DD format.

BTW, I'm not really trying to champion a methodology here - everyone's preferred method will vary, and some more suitable in different circumstances; but having had the issue of people not understanding the properties of CRCs across large data blocks on a few occaisions, I just thought I'd share my thoughts, and the app, in case it helped others.

Phil.

ReplyQuote
Posted : 29/12/2009 2:22 pm
Jonathan
(@jonathan)
Senior Member

So, how do you go about proving that in court?

Here is how I heard it. You image the drive, but after doing your examination on the image you want to start up the computer as the suspect did, same hardware and such. You can't just put the evidence drive in, so you do a copy of the image to a new, similiar drive. However, the defense might balk, how do you know that the new, similiar drive did not have old data on it, data that you are now using against their client.

I might have heard wrong, but my curiousity remains. How do you go about proving a zero filled drive is actually zero filled? A program or something?

This brings up the old debate of whether each image/case should be made on/stored on a separate device which has been previously securely deleted to avoid the accusation of "cross-contamination".

If you're going to get into explaining hashes to jurors anyway as your scenario implies, then you can state (and show) to the court that the acquisition hash matches the verification hash = no data on the image was added or modified since imaging took place. A simple and established procedure used universally in computer forensics; there's really no forensic requirement to prove a drive is zero-filled in this situation.

There is nothing 'unforensic' whatsoever about placing images from different cases on to the same storage device - be that a HDD, server, NAS or SAN.

ReplyQuote
Posted : 29/12/2009 6:05 pm
indur
(@indur)
Member

No reasonable hash will produce zero for a string of N zero-bytes. I think the direct answer to your question is that you can demonstrate the drive is zeroed by hashing the drive (and recording its size) after zeroing it. Then, hash an equivalent volume of zero bytes and show that hashes are the same. This is straightforward in Linux and fairly easy to program in a scripting language.

ReplyQuote
Posted : 29/12/2009 8:20 pm
roncufley
(@roncufley)
Active Member

> Agreed that works fine; but for very large files can take a long time. The main reason for the appā€¦..
Phil.

Our posts crossed, so I was not trying to compete with your app. I was just trying to give the OP a simple answer to his qusetion using readily available tools without any comment as to whether it was necessary or not.

The question of zeroing forensic media/using a seperate drive for each case is a continual one, there is clearly no technical reason to do it but if it can avoid a line of questioning then it must be worthwhile mustn't it? The only downside (apart from the effort) is if explaining why it is done is more complex for the Court than explaining why it does not need to be done.

ReplyQuote
Posted : 29/12/2009 11:33 pm
Page 1 / 2
Share: