ecophobia I had the Intel report, but can you let me know the cost for the portal citation? You may know how it is, can't buy everything.
I started Feb 07 research into flash and embedded systems for mobile telephones (and it still ongoing) - it is all to do with whether extracting and harvesting physical data from mobile phone flash memory produces safe and reliable evidence - I had a link from which to download the flash survey report by Gal and Toledo, two researchers at Tel-Aviv University
Algorithms and Data Structures for Flash Memories
ERAN GAL AND SIVAN TOLEDO
Published June 2005
The link doesn't work for the report I downloaded in April 2007. I have a copy of the report but don't want to get into copyright problems.
ecophobia as you are with ACM can you see if it is still available there please? Here is the ACM reference
ACM Computing Surveys, Vol. 37, No. 2, June 2005, pp. 138–163.
Thanks
Gents,
As I mentioned before, it took me a month of reading and experimenting with the devices before I spoke up about the issue. I haven’t written any technical white papers or dissertations on this topic and I have no plans to do so in the future. The purpose of the exercise was simply to find the explanation of such peculiar behaviour and to devise the procedure to deal with this in court. As you can see, this was done for a very specific purpose and as far as I am concerned, the mission is accomplished. (I briefly mentioned about the procedure in one of my previous posts. LE people can contact me for more information).
I prefer to leave the rest to the researchers and companies with an appropriate budget and R&D departments. I am glad that this topic sparked a lot of interest. I am currently quite busy with other stuff and struggling to find the time even to answer the posts on this great forum.
Trewmte,
I’ll get back to you later re ACM. Sorry, it is almost midnight here in Sydney and tomorrow is Monday -)
I hope you didn't think me too cheeky asking for help. I am of course happy to share information I have if it can assist your own reseach. Thanks ecophobia.
I hope you didn't think me too cheeky asking for help. I am of course happy to share information I have if it can assist your own reseach. Thanks ecophobia.
No problems. I am happy to help when I can.
I was right about and can confirm that ChipGenius cannot read chip info of the USB device if connected via harware write-blocker.
In relation to ACM publication, plz check you PM.
I was right about and can confirm that ChipGenius cannot read chip info of the USB device if connected via harware write-blocker.
Ok, NOW we know. wink
Would the same happen under Linux also? ?
http//
The VID and PID of the devices may be enough to understand what chip is used on them.
I am looking in the (very scarce) documentation I could find about the device thefuf reported, USBEST UT-163, but I seem not to be able to find anything about wear-leveling.
Here is the datasheet
http//
I need to go deeper, however I can confirm that, at least for the other controller in the "family", the UT-165
http//
Flash support
- Supports 4KB and 2KB page flashes, including SLC/MLC NAND, AG-AND and ORNAND flashes
- Integrated ECC circuits for 8-bit/512Bytes and 14-bit/512Bytes BCH error correction
- Support NAND flash word and byte access for both 3.3V and 1.8V flashes
- Dual-channel, interleave and multi-plane mode support to achieve best performance
- Supports flash chip up to 16 “CE” pins
- Embedded “Dynamic Wear-Leveling” algorithm to have flash data WRITE to be evenly distributed over the storage area and the end product life time will be extended
Wear-leveling is advertised as being implemented, which definitely confirms the reports. D
The old Usbest page for UT-163 did not mention this feature, however
http//
?
jaclaz
I don't have specific knowledge about wear leveling, however I'd like to throw the following in the discussion
There are a lot of applications which uses the raw device for storage of data (think of Truecrypt raw container, databases, video apps …), in these cases there are no "allocated" or "unallocated" sectors or clusters.
How would these apps work if a low level mechanism like wear leveling could change the content of an "unallocated" logical sector ?
How would these apps work if a low level mechanism like wear leveling could change the content of an "unallocated" logical sector ?
The "raw" device that any app through the OS (and controller chip) can access, is "raw", but it is "OS raw" only as opposed to "OS logical".
In other words it is not the actual physical device, but rather the representation of the physical device as "filtered" or "rendered" by the controller hardware/firmware.
Read my previous post on page 3.
jaclaz
Likewise, there are generally two senses of "unused" sectors. There are those in the device's reserve pool that are inaccessible to the operating system (not exposed to the device). These are often used for wear leveling, similar to how spare hard drive sectors are used to fix bad sectors. Then there are "unallocated" sectors, which are visible to the OS but not currently being used to store data by the filesystem on the device. It's these that people are clearly interested in.
Chris's statement is really that either the device must interpret the filesystem on it, must use a custom driver, or must be given signals by a common CF driver in order to allow the unallocated sectors to be changed. If you use some filesystem the device couldn't possibly understand (like TrueCrypt raw data, or some obscure Unix FS), on a system that has no drivers that will help it, it cannot change the contents of the unallocated sectors, as it has no way of knowing which are unallocated. (If it does, it is bound to corrupt your data.)
Just a couple of points, and giving you guys most of my "good stuff"
ref the VID and PID
These are generally re-programmable by the 'final' manufacturer, there may be a way to build up a table , see here
http//
However there are "hundreds" of clone chips coming out of China (mainly Shenzhen) And the utilities for the end users to 'change' these PID/VID are freely available online.
On the ACM/ IEEE
Yes most of the forensic papers are available however this one is better
Breeuwsma, M. et al. (2007) Forensic Data Recovery from Flash Memory. SMALL SCALE DIGITAL DEVICE FORENSICS JOURNAL, VOL. 1, NO. 1, Available from http//
also you want to have a look at this
Interface, Open Nand Flash (2009) Open NAND Flash Interface Specification Revision 2.1. 206. [Online]
Available at http//
for background.
and this
Chen, Scott (2007) What Types of ECC Should Be Used on Flash Memory? [Online]
Available at http//
and this
Phillips, B. J., Schmidt, C. D. & Kelly, D. R. (2008) Recovering data from USB Flash memory sticks that have been damaged or electronically erased. e-Forensics '08 Proceedings of the 1st international conference on Forensic applications and techniques in telecommunications, information, and multimedia and workshop, 1–6.
On the 'changing' content , please take into account 'fake' USB sticks from China. (there are millions of these in circulation)
In many cases the contents are 'crosslinked' at the block level, so as the controller chip does it's business, the files will appear to change, because 'n' logical blocks may actually be allocated to 1 physical block.
first thing you need to do is separate out your fakes(loking at the case will NOT tell you), you are then on a level pegging to start looking at the problem.
Also be aware that even internally they may not be what they say.
I have examples of "chip re-branding" , this is where either/both the memory chip or controller have been rebranded, so even when you compare two chips you THINK are the same, they may not be….
To really understand what is going on inside these devices , I'm afraid it's a lot of work and reading.
Oh just to finish off…….
I have at leaset TWO examples of devices that UNDERSTAND and works at the windows file system level. That is to say the internal device has knowledge at a level it really should not have.
I'm currently in the middle of writing a Thesis on this stuff, and joined this forum because I saw an intelligent discussion.
Code…..
Just a couple of points, and giving you guys most of my "good stuff"
ref the VID and PID
These are generally re-programmable by the 'final' manufacturer, there may be a way to build up a table , see here
http//
However there are "hundreds" of clone chips coming out of China (mainly Shenzhen) And the utilities for the end users to 'change' these PID/VID are freely available online.
On the ACM/ IEEE
Yes most of the forensic papers are available however this one is better
Breeuwsma, M. et al. (2007) Forensic Data Recovery from Flash Memory. SMALL SCALE DIGITAL DEVICE FORENSICS JOURNAL, VOL. 1, NO. 1, Available from http//
also you want to have a look at this
Interface, Open Nand Flash (2009) Open NAND Flash Interface Specification Revision 2.1. 206. [Online]
Available at http//
for background.
and this
Chen, Scott (2007) What Types of ECC Should Be Used on Flash Memory? [Online]
Available at http//
and this
Phillips, B. J., Schmidt, C. D. & Kelly, D. R. (2008) Recovering data from USB Flash memory sticks that have been damaged or electronically erased. e-Forensics '08 Proceedings of the 1st international conference on Forensic applications and techniques in telecommunications, information, and multimedia and workshop, 1–6.
On the 'changing' content , please take into account 'fake' USB sticks from China. (there are millions of these in circulation)
In many cases the contents are 'crosslinked' at the block level, so as the controller chip does it's business, the files will appear to change, because 'n' logical blocks may actually be allocated to 1 physical block.
first thing you need to do is separate out your fakes(loking at the case will NOT tell you), you are then on a level pegging to start looking at the problem.
Also be aware that even internally they may not be what they say.
I have examples of "chip re-branding" , this is where either/both the memory chip or controller have been rebranded, so even when you compare two chips you THINK are the same, they may not be….
To really understand what is going on inside these devices , I'm afraid it's a lot of work and reading.
Oh just to finish off…….
I have at leaset TWO examples of devices that UNDERSTAND and works at the windows file system level. That is to say the internal device has knowledge at a level it really should not have.
I'm currently in the middle of writing a Thesis on this stuff, and joined this forum because I saw an intelligent discussion.
Code…..
