Greetings,

This was found on the Wikipedia data recovery page:

""Open source tools such as DCFLdd v1.3.4-1 can usually recover all data, with exception of the physically damaged sectors. (It is important that DCFLdd v1.3.4-1 be installed on a FreeBSD operating system. Studies have shown that the same program installed on a Linux system produces extra "bad sectors", resulting in the loss of information that is actually available.) [1]"

reference: ^ Cyrus Robinson, IXImager Bad Sector Drive Imaging Study. Defense Cyber Crime Institute Cyber Files Reports and studies are available only to US governmental agencies and law enforcement organizations."

Has anyone else heard about this problem with dcfldd? Any more details? Kind of disturbing information.

-David

Okay, so here's a similar paper with similar results:

dfrws.org/2007/proceed...3-lyle.pdf

I've tested this and can confirm their findings (similar - so far). Testing is still underway. I did a presentation on this issue at our annual in-house conference, along with alternatives and solutions. I'll get around to posting them when I'm done testing. I'm not paid to be a researcher, so time is limited for me to "play" with this.

The problem with just pushing out the results right now is that even I find my testing methods "questionable". I don't want to jump to conclusions or make hasty accusations on the accuracy of tools. Coming up with realistic and reproducible "bad sectors" is tough. But using MHDD to create fake ones results in the same findings as the document cited above.

Bottom line is that "conv=noerror,sync" is BAD (for more than just the skipped sectors reason). If you need to use it, you are using the wrong tool (IMHO). I will comfortably say that much. There are better options out there.

My $.02.
Flame away...

Barry

Greetings,

Will your testing include commercial and hardware based tools? The report you linked to is very interesting, but I'd love to know how other tools fared on the same test disks. Going with FTK Imager, or a Talon, based on this report is all well and good until you find out that they have a similar problem.

-David

kovar wrote:

Will your testing include commercial and hardware based tools? -David

Yes. And so far the problem does not occur with tools other than Linux based dd tools accessed through the kernel. Using /dev/raw (or ddrescue with "-d") alleviates the problem, but like I said, I'm still testing.

kovar wrote:

Greetings, Will your testing include commercial and hardware based tools? The report you linked to is very interesting, but I'd love to know how other tools fared on the same test disks. Going with FTK Imager, or a Talon, based on this report is all well and good until you find out that they have a similar problem. -David

I saw an Image Master Solo III lying around our office a while back, and noticed that it uses Linux DD to capture its images.

I'm not saying that product is affected, and at the minute I dont have the hardware at hand, but I'd be interested in testing it.

Ronan

Seems like this issue comes up from time to time. Here is a thread that touches on the subject from early 2007:

www.cybercrimes.it/for...2&topic=24
(Italian).

Here is a more recent posting on the subject matter:

tech.groups.yahoo.com/...essage/82.

ReC

Greetings,

I just skimmed through the NIST reports:

nij.ncjrs.gov/publicat...op&PSID=31

The problem appears to be with the Linux kernel, thus affecting all tools running on Linux. This specifically includes Helix.

It doesn't include IXimager which is running on a Linux micro kernel.

I don't know how it affects tools running on OS X but would guess that dcfldd running on OS X would not manifest this problem.

-David

(It's amusing to note that the message quoted in the Yahoo! group is the one I sent to the CCE list this morning. Why they cross-posted it without replying to the original list is beyond me.)

ronanmagee wrote:

I saw an Image Master Solo III lying around our office a while back, and noticed that it uses Linux DD to capture its images. I'm not saying that product is affected, and at the minute I dont have the hardware at hand, but I'd be interested in testing it. Ronan

The Solo III is not affected. I tested it. Nor is FTK, Encase or IXImager. At least not in the **limited** tests I ran.

This is really nothing to get spun up about. It's no different than the "odd sector" issue that people ascribed to dd in the past. It's only a problem if you run into bad sectors, and if you do, then I would argue that dd (and some of it's variants) is the wrong tool anyway.

Even if you want to use dd (dcfldd, etc.) in the case of bad sectors, there are work arounds for this particular issue. Just like there were in the "odd sector" days. dd/dcfldd, etc. are still perfectly viable imaging tools.

Test, understand, and move on...

Barry,

Does the Linux problem occur even when you specify a block size equal to the device sector size (usu. bs=512)?

ReC.

Rossetoecioccolato wrote:

Does the Linux problem occur even when you specify a block size equal to the device sector size (usu. bs=512)?.

Yes.

Actually, I just got done reading your post on the yahoo tech group and was looking to reply there. The bs=512 option has no effect on this.

I've decided I'm going to post some of my recent presentation on this so that people can see the actual output from the tools and the actual commands that were run. Just to clarify, I had nothing to do with the NIST paper. My testing was independent and was done in preparation for a presentation on media error handling at our agencies in-house conference.

The test drive had 4 bad sectors. All the Linux based dd tools missed between 200 and 232 sectors. Other tools missed just the 4. When the dd commands were run with the device associated with /dev/raw, they correctly reported 4 bad sectors. The only Linux util that did not suffer the problem was GNU ddrescue (not dd_rescue), as long as the -d flag was used ("direct access" - like using /dev/raw).

I really think this has to do with kernel caching. Hence the correct output with /dev/raw. Just a theory. Testing is ongoing.

HTH,
Barry