Join Us!

E01 Image format / ...
 
Notifications
Clear all

E01 Image format / tools  

Page 1 / 2
  RSS
JimC
 JimC
(@jimc)
Member

Please can I ask

Are there any tools that can produce E01 images in reverse sector order? I had assumed XWF could do this but upon investigation I found it was only supported for raw "dd" images.

Jim

www.binarymarkup.com

Quote
Posted : 22/11/2019 6:05 pm
AmNe5iA
(@amne5ia)
Active Member

Just use XWays to do it to DD then convert to e01. e01 isn't AFF4. the e01 format can't deal with out of order sector imaging so you won't get any tool to read in reverse and create a e01 directly

ReplyQuote
Posted : 24/11/2019 2:55 pm
athulin
(@athulin)
Community Legend

Are there any tools that can produce E01 images in reverse sector order? I had assumed XWF could do this but upon investigation I found it was only supported for raw "dd" images.

Before you do anything like that … you need to ensure that that 'image' will never be treated or interpreted as an image taken from evidence disk drives. You may need to prepare the ground for that in some way, to ensure that if the question ever arises later if that image is a 'real image', the answer is emphatically 'no', even if you are not involved in answering that question.

(This probably means, at least, ensuring that the e01 header case information clearly says what has been done to it, but should probably be documented 'in writing' as well, along with some way of identifying the raw data, i.e. without relying on header information being correctly retained.)

Very, very few analysts would even consider the possibility that an e01 image isn't an image, but a constructed/manipulated image. This lays the foundation for the possibility of bad misinterpretation of evidence.

I think it would be safer to choose an image format that does not carry the same 'default interpretation overload' as e01 does, if possible.

ReplyQuote
Posted : 24/11/2019 7:57 pm
jaclaz
(@jaclaz)
Community Legend

@athulin
Do you mean that there is actually any use for an image with sectors in reverse order? ?

I thought that the point was acquiring in reverse order (this helps in some cases of data recovery) but the resultng image is "normal". ?

jaclaz

ReplyQuote
Posted : 25/11/2019 9:11 am
JimC
 JimC
(@jimc)
Member

Just use XWays to do it to DD then convert to e01. e01 isn't AFF4. the e01 format can't deal with out of order sector imaging so you won't get any tool to read in reverse and create a e01 directly

I appreciate this suggestion but agree with @athulin. This would mean that the final image was not a direct image and could thus be questioned. However, it is sometimes necessary to create a backwards image if the source disk is too damaged. In this case, it can be necessary to create a "dd" image in several stages to recover as much as possible. The option to do this backwards can help.

I have now read the spec for E01 in more detail and it seems implicit that

1. The image starts at sector zero - You can create an image of either a physical disk or a single partition but the image itself doesn't record the starting sector.

2. It is implicit that the E01 data chunks are stored in ascending order - Although there is a little wriggle room here because although the "tables" section assumes the data chunks are in ascending order they can actually be stored out of order in the "sectors" section (used by more recent imaging tools)

3. The data chunks each represent a fixed sized (typically 32KB) and, whilst they can be compressed, they must all be present. e.g. The E01 format doesn't seem to support any kind of 'sparse' storage. This seems like a huge oversight since many drives will contain a significant amount of unused storage.

I think this "wriggle room" could be used to create a backwards image but would be limited in practice because the E01 file is typically split into segments (e.g. 2GB or 4GB) and each segment has the same assumption that it contains data chunks in ascending order.

Could any E01 experts confirm if this analysis is correct?

Jim

www.binarymarkup.com

ReplyQuote
Posted : 25/11/2019 10:37 am
Passmark
(@passmark)
Active Member

Surely it is easier to just use a DD raw image for the whole project?
Why do the conversion at all?

There is a very narrow set of disk errors that reverse reading will help with and this set is getting smaller and smaller with modern drives.

And what is meant by a reverse read depends on the tool. I think some that do reverse reading actually read most of the disk in the normal order and only try reverse reading on bad sectors. So 99.9% of the disk is read in the normal order. Thus there is no technical reason E01 output couldn't be directly used (with the right tool). Data is put back into its correct order in RAM, then written out as a E01.

And for the novices, the disk doesn't spin backwards for a reverse read. Nor are the bytes within a sector read backwards. It just means you read the sectors out of order.

If you did convert the DD image to E01 via post processing it should be fairly trivial to check volume hashes to make sure the data is the same before and after conversion. There is no reason to think format conversion would corrupt the evidence. I'd be much more concerned about the corrupt source drive messing up the evidence. Especially if the tool did something like disable CRC checking on the source drive.

ReplyQuote
Posted : 26/11/2019 12:19 am
jaclaz
(@jaclaz)
Community Legend

There is a very narrow set of disk errors that reverse reading will help with and this set is getting smaller and smaller with modern drives.

And what is meant by a reverse read depends on the tool. I think some that do reverse reading actually read most of the disk in the normal order and only try reverse reading on bad sectors. So 99.9% of the disk is read in the normal order.

Well said. )

There is no reason to think format conversion would corrupt the evidence. I'd be much more concerned about the corrupt source drive messing up the evidence. Especially if the tool did something like disable CRC checking on the source drive.

Even better said. D

I would add that (at an "atomic" level) you could well (in theory) create out of - say - a 500 GB disk, consisting of almost 1,000,000,000 sectors, a same 1,000,000,000 files, one per each sector, and re-assemble these sectors in the correct order into a dd-image.
As long as the hash of each sector/file are correct and you re-assemble the image in the correct order, there is no messing with the evidence.
In practice, it would be a hell of a log to check. wink

jaclaz

ReplyQuote
Posted : 26/11/2019 9:23 am
mscotgrove
(@mscotgrove)
Senior Member

If you really want reverse sector order, why not just read the .E01 file in reverse. As with others who have replied, I do not understand the requirement

ReplyQuote
Posted : 27/11/2019 9:05 pm
Passmark
(@passmark)
Active Member

Showing my age, but many years ago there was a theory that by "approaching" the disk sector from different "angles" would allow the disk head to position itself slightly differently over the track and maybe allow reading of otherwise bad sectors. Reverse reading was a version of this, done by seeking backwards. Seeking to the edge of the disk and then back to the desired track was another version. Nothing really gets read in reverse at a low level. But the whole theory of reverse reads doesn't even make much sense (in my opinion) as there is so much data in a single track that most of the time seeking backwards won't even move the disk head. i.e the previous sector is on the same track as the current sector. The effect you actually get is much slower disk reads as the read ahead buffer isn't used and the disk head needs to rest in place until the previous sector rotates under it to read it.

But the whole thing is a complex mess now and it often isn't even possible to know the real low level disk layout for modern drives as they have layers of virtualisation on them.
https://en.wikipedia.org/wiki/Cylinder-head-sector

Won't even mention, SSDs, RAID, SANS, caching, automatic relocated sectors, etc…
Modern drives also position the head much more precisely.
I would be surprised if reverse reading was any better than a simple retry on modern drives.

Not saying it is totally pointless. Just mostly pointless.

ReplyQuote
Posted : 27/11/2019 10:18 pm
minime2k9
(@minime2k9)
Active Member

My understanding of reverse disk imaging is to do with caching of sectors.

AFAIK, When a disk reads sequentially it caches the sectors after it in anticipation. Where a disk is damaged, this can cause the read to hang but more to do with the caching rather than the sector being read. Reading in reverse removes the caching and therefore allows more sectors to be read.

Again, not sure how true this is but reverse imaging has worked for me in the past, though tools like DDrescue (which goes forwards and then back) are usually do a much better job than tools that generate E01 files when imaging.

ReplyQuote
Posted : 28/11/2019 7:24 am
mscotgrove
(@mscotgrove)
Senior Member

If you really want reverse sector order, why not just read the .E01 file in reverse. As with others who have replied, I do not understand the requirement

ReplyQuote
Posted : 28/11/2019 7:53 am
Rich2005
(@rich2005)
Senior Member

If you really want reverse sector order, why not just read the .E01 file in reverse. As with others who have replied, I do not understand the requirement

As minime says above. It was due to problematic drives.

You could image a problematic drive normally, but it would get to a certain point, where the drive would become unresponsive.
If that point was near the start of the drive you might be getting say 1% of the drive imaged.
The reverse imaging option allowed people to image forwards, until it hung, then image backwards until it hung.
That meant, instead of getting the 1% of the drive, for example, people could get perhaps 99% of the drive imaged.

ReplyQuote
Posted : 28/11/2019 8:13 am
nightworker
(@nightworker)
Active Member

use ftk imager as source is image file you can convert image

ReplyQuote
Posted : 28/11/2019 8:58 am
jaclaz
(@jaclaz)
Community Legend

But the whole thing is a complex mess now and it often isn't even possible to know the real low level disk layout for modern drives as they have layers of virtualisation on them.
https://en.wikipedia.org/wiki/Cylinder-head-sector

OT, but JFYI, a nice article about finding out geometry/layout of disks

http//blog.stuffedcow.net/2019/09/hard-disk-geometry-microbenchmarking/

jaclaz

ReplyQuote
Posted : 28/11/2019 10:20 am
JimC
 JimC
(@jimc)
Member

If you really want reverse sector order, why not just read the .E01 file in reverse. As with others who have replied, I do not understand the requirement

I think the point of my original question has got somewhat lost along the way. This is my fault because when I first asked, I didn't understand the limitations of the E01 format. There may be (rare) occasions when a reverse image is necessary but this is actually not relevant to my question which I think I can answer myself now.

The problem appears to be the limitations of the E01 format itself

1. The image starts at sector zero - You can create an image of either a physical disk or a single partition but the image itself doesn't record the starting sector.

2. It is implicit that the E01 data chunks are stored in ascending order - Although there is a little wriggle room here because although the "tables" section assumes the data chunks are in ascending order they can actually be stored out of order in the "sectors" section (used by more recent imaging tools)

3. The data chunks each represent a fixed sized (typically 32KB) and, whilst they can be compressed, they must all be present. e.g. The E01 format doesn't seem to support any kind of 'sparse' storage. This seems like a huge oversight since many drives will contain a significant amount of unused storage.

I think this "wriggle room" could be used to create a backwards image but would be limited in practice because the E01 file is typically split into segments (e.g. 2GB or 4GB) and each segment has the same assumption that it contains data chunks in ascending order.

Yes, you can workaround this limitation by creating a "DD" image first. Sometimes this may be necessary. No, reading the E01 image backwards misses the point - the question was could the image be created in reverse. Based on the above limitations, I think the answer must now be no it cannot. This has nothing to do with if this technique would be useful or why it may help - rather it boils down to a limitation in the E01 format.

Jim

www.binarymarkup.com

ReplyQuote
Posted : 28/11/2019 2:34 pm
Page 1 / 2
Share: