Hashing and wear-levelling

Sounds good to me. )

so I suggest to restrain yourself from providing unsubstantiated opinions

I think I made it quite clear that this was an opinion and opinions are just that - I did not back it up with facts because other than what to me is common sense (based on my experience with media [as an ex engineer and manager of what was the UK's largest data recovery company] and a computer forensic practitioner since 1993 [my programming experience goes back a long way prior to that] - which I highlighted to put my opinion in perspective - not to score points) I do not have any. As an expert it is my duty to differentiate between facts and opinion and I believe that my post was clearly an opinion.

I stated that I am open to correction and I always like to keep an open mind - I have been wrong before and I expect I will be wrong again )

However, I see nothing in your posts or blog that supports the assertion you have made -at the moment it is simply an unsupported claim - I read your November 2008 wear levelling blog and did not see any reference to the two devices that you mention - I did not have time to read all the other material posted.

So in the interests of taking this further and possibly being proven wrong and learning something new, could you please let us know
- which two devices exhibited these characteristics
- whether you have seen this effect in more than one instance of each of these devices
- you mention 1 in 10 devices exhibits this - how many devices have you tested
- why do you think the others do not exhibit this characteristic

Also could you point to the sources that support your contention (not just the sources re wear levelling in general) so that we can independently verify your claim.

As I said - happy to be corrected.

Thefuf You seems to be obsessed with "the device knowing about the file system on a drive" issue. The device doesn't care about the file system. The following links is a good starting point for you
www.dataio.com/pdf/NAN...hanism.pdf
www.bz-com.com/info/we...veling.pdf

The TrueFFS article you post seems at first glance to support your contention with ‘the device knowing about the file system’. However, a very quick google revealed this document from the manufacturers M-Systems http//www.spezial.de/commercio/dateien/produktbeitraege/TrueFFS.pdf this document seems to me to show that for the TrueFFS file system to come into play a driver must be installed into the operating system on the computer into which the device is inserted. This is not quite the same as the device understanding the file system, more the file system (via a driver I have never seen installed) understanding the device.

So for this to be responsible for the OP's mismatching hash problem then it seems to follow that the OP would have to have the driver installed on his forensic computer. Perhaps Thefuf could check and report back for us.

I will emphasis that I have only made a cursory glance at the document and not looked at any other search hits.

To keep the peace, I think that is where this conversation should end

Keeping the peace implies there is a war - not so - I just want to see some supported facts and if I am wrong I am happy to be found wanting in public.

However wear leveling works, the data in a logical sector must always remain the same.

If a disk is hashed by reading logical sectors, with no (TrueFFS) device driver, the results will always be the same, unless the disk / chip has a failure, or a sector has been written to.

The physical location of the sector is irrelevant, the logical location is not changeable.

- which two devices exhibited these characteristics
The devices are mentioned in the blog comments below the post.

- whether you have seen this effect in more than one instance of each of these devices

I calculated md5 hash 10 times for every device, each time these devices produced different hash value. The devices were write-blocked at all times. Further examinations revealed that only unallocated/free space was changing, sector count remained the same; existing data have the same hash value (confirmed with winhex (x-way forensics version) by creating the hash set).

- you mention 1 in 10 devices exhibits this - how many devices have you tested

It is hard to say for sure, I get to examine from 5 to 30 every month.

- why do you think the others do not exhibit this characteristic

Different wear-levelling implementation such as static vs. dynamic, also there is no strict standards really. I even have spoken to USB Implementers Forum, Inc people about this. It is all up to the manufacturers.

The main purpose of this particular post is to inform my colleagues about the need to perform integrity checks of USB flash drives before and after the acquisition, one of this is rarely taken. The standard verification will only do this once and then calculate the hash of the acquired image and compare these two. So, next time someone checks the integrity of the original evidence, the hash value may be different. Ooops.

- Also could you point to the sources that support your contention (not just the sources re wear levelling in general) so that we can independently verify your claim.

If I could find the paper explaining the effect of wear-levelling on forensic investigation, I would have not spent one month of my live reading various documents and examining the drives.

As I mentioned before, my sources are general wear-levelling documentation. I don’t have the links handy, but here are some of these documents

* Forensic Image Analysis of Familiar-based iPAQ by Cheong Kai Wee
School of Computer and Information Science, Edith Cowan University
* Whitepaper San Disk flash memory cards Wearleveling
* System Software for Flash Memory A Survey by Tae-Sun Chung
* Cactus Technologies Application Note CTAN013 Wear Leveling – Static
vs Dynamic

Wear-levelling may be the reason for not wiping the drive correctly and here are some links I mentioned to you in the email. They are definitely worth reading and in my view warrant some further research instead of dismissing the fact and simply blame on faulty wiping software or the device.

http//www.hbarel.com/Blog/entry0016.html

http//episteme.arstechnica.com/eve/forums/a/tpc/f/24609792/m/779008714931

http//kaizen.michaeldundas.com/2008/07/19/wear-leveling-with-flash-drives-and-usb-sticks/

http//isc.sans.org/diary.html?storyid=5213#comment

I should mention it again; there is no bias or profit involved. It is only hard work and no expectation of any kind of reward. I‘d like to include a part of the disclaimer from my blog that outlines the purpose. If one disagrees with something, this person is free to take his time and research the subject, examine the devices and correct me if I am wrong.

Disclaimer
This blog is intended for my digital forensic needs and shared with everyone interested to make our world a little bit safer. While all reasonable attempts have been made to ensure the accuracy of information on this blog…

All,

Thank you for bringing this conversation back on track, I appreciate it and I think it's an interesting subject area to explore further. In the long run who's right or wrong in early exchanges means little as long as we can move things forward and learn on the way.

Jamie

I should mention that wear-levelling is a fairly new technique, I believe (don't quote me on that) it has been first implemented around 2003.

Wear-levelling responds following wear-measurement. Wear-measurement appears to be made up of many parts.

Two of the earliest simple wear-levelling and reclaimation employed, I believe, was the design of Wu and Zwaenepoel [1994] and the other patented by Wells [1994].

*Wu and Zwaenepoel [1994] employ a simple form of wear leveling. When the erase count of the most worn-out unit is higher by 100 than that of the least wornout unit, the data on them is swapped.

*WU, M. AND ZWAENEPOEL, W. 1994. eNVy A nonvolatile,
main memory storage system. In Proceedings
of the 6th International Conference on
Architectural Support for Programming Languages
and Operating Systems. San Jose, CA.
86–97.

**Wells [1994] patented a reclamation policy that relies on a weighted combination of efficiency and wear leveling. The system selects the next unit to be reclaimed based on a score.

**WELLS, S. E. 1994. Method for wear leveling in a
flash EEPROM memory. US patent 5,341,339.
Filed November 1, 1993; Issued August 23, 1994;
Assigned to Intel.

[Updated] Reference sources added

A maybe "stupid" idea. ?

Using dsfo (part of the dsfok package) you get the MD5 hash while "dd-ing" the \\.\PHYSICALDRIVEx. (i.e. hashing the transferred data while transferring it)

IF the MD5 of the resulting image is the same, wouldn't it be correct to state that at the moment the MD5 was the same, and thus that the image is accurate? ?

jaclaz

A maybe "stupid" idea. ?

Using dsfo (part of the dsfok package) you get the MD5 hash while "dd-ing" the \\.\PHYSICALDRIVEx. (i.e. hashing the transferred data while transferring it)

IF the MD5 of the resulting image is the same, wouldn't it be correct to state that at the moment the MD5 was the same, and thus that the image is accurate? ?

jaclaz

Not stupid at all, but
I would still want to know in advance if I am dealing with the device that produces different hash value. It is especially important if you are going to present / rely on the recovered/ deleted files. Some extra work has to be done to make sure the evidence found inside these deleted files is accurate. It may be possible to recover zip, word or rar files and be able to open it on one occasion, and not be able to replicate this after creating another image because of the new free space block being assembled from the different sectors this time. This is not so much an issue with jpeg and other ‘non container type’ files though.

I have found another USB flash drive (pqi 1 GB) with same "problems" (block device gives different hash values every time, but files on a mounted filesystem give the same hash values). I took two images from this drive and hashed every 512-byte block. It seems that data is moving on the drive - some blocks are replaced with zeroes, some blocks with zeroes are replaced with data from other blocks. I can provide diffs if requested. Unfortunately, I cannot reproduce this after wiping the drive with zeroes / random data (everything is OK now, I'm getting the same hash values every time in all tests).

By the way, first diff is at offset (in bytes) 995376640 and the last is at 1010498560. Is it possible, that this area is reserved for wear-levelling and is visible by reading the block device? Any other theories? )

This is the same behavior that was reported at a DefCon talk last year, though the talk was about SSDs and not USB flash disks.

I agree that the only way for this behavior to exist is if the flash drive is made aware of the filesystem on top of the block device.

Some light Googling suggests that compact flash devices have additional commands beyond simply accessing them as block devices – for example, to "erase" whole CF blocks. Since USB should identify the device as a CF drive supporting these commands, a smart-enough driver could use them to provide "extra" information to the device (like "this block is unused now") that the device's wear-leveling can now take advantage of. (I've also seen mentions that the wear-leveling implementations vary greatly between vendors, so it makes sense you'd only see some USB disks have this behavior.) While it's speculation without looking more closely into the compact flash specs, this would be my guess as to how the USB disk is magically getting the information needed to alter unallocated space of its own accord.

As per the DefCon talk, there's no convenient way to stop this behavior – the wear leveler runs whenever the device is receiving power.