Interesting Ddan !
Meanwhile Carrier's book arrived (mere minutes ago).
I'm certainly going to read the relevant sections once the software has been rerleased, to fix any possible issues I haven't thought of, before the next/final release. For now it seems that what I have is working nicely, although I would like to get some confirmation on the so called negative runs.
Yesterday I could not release the Beta version because I realized I had forgotten to renew my Certificate (doh). I expect a phone call from Comodo today to verify that I am 'me' and hopefully I can sign tonight and release the beta version.
In my software you can list the extents per file, and it would be interesting to confirm if they are correct for files with known negative runs.
I also believe IsoBuster will be a great addition to your tools set. It already does UDF and HFS (and FAT) on HD and flash/usb media etc.
So you might want to check it out.
The beta will not be perfect. I still need to do some coding for Mac resource forks (the MacBinary conversion stuff) and I also don't find deleted files yet. I had expected them to be available like in FAT, but my testing shows that they are removed from the INDX records.
So I need to go read the MFT records that are not in use to find missing files (I guess).
People who have helped me (Dan, Joakim, Paul) send me a personal email and I'll gladly cut you a [Business] type license to the soft.
I'll post here again when the beta has been released. It will depend on the certificate.
Voila,
The beta is online
Get it here
I haven't done the release notes yet, they are for tomorrow.
I note that earlier in this discussion you referred to using the information given at
http//
to implement your run coding. I assume you are aware that this is one page of a much larger document on NTFS? It is by Richard Russon and you can find an English version of it at
http//
I have downloaded the beta version and hopefully will be able to try it out this weekend. Will let you know how I go.
Ddan
Installed the beta version but can't get much further than loading the image without a license. Will send you a pm.
Ddan
I just tried your beta and noticed something buggy.
I took a tiny flash stick and zeroed it completely before creating a fresh ntfs volume on it from XP. I put 1 file on it and deleted it in explorer. Then ran your tool to "find missing files and folders". It found the file but its size is completely wrong. After extraction the 373 KB file was over 10 MB (just a bunch of 00's was added to it). There was only 1 simple run to solve, so nothing overly complex.
Also I see that the name of the file is missing. That is still present in $MFT under the $FILE_NAME attribute so should be easy to add (as long as the record is intact of course).
I tried the beta and extracted the used files from an image. I then compared them using an md5 checksum against the actual drive. It showed two problems.
You still have a fixup problem. The extraction of resident files shows this intermittently.
The second problem relates to the handling of a compressed file. The extracted file looked nothing like the original. Its data run extents were
31 02 96 07 09
01 1E
11 01 20
01 0F
00
Isobuster showed four extents with the correct byte size and lba, except for the last extent which it calculated as 14 clusters instead of 15. It also showed the sparse clusters correctly. I had specially prepared the file to test compression data runs. It consisted of 16 clusters (ie a compression unit) of highly compressible data, plus 16 clusters sparse, plus 15 clusters highly compressible. So maybe your 14 comes from the filesize? My test was basically to show that if the residual part of a file was less than the size of a compression unit (most of the time), then compression does take place and the extent is padded to a compression unit.
I would argue that there are not actually four extents with this file, there are only three
31 02 96 07 09 01 0E
01 10
11 01 20 01 0F
00
and my reasoning would be that the sparse clusters associated with the compressed parts are not used except to signify compression.
Hope this all helps.
Ddan
Just looked at the extracted compressed file in WinHex and realise that you have not done the decompression, which accounts for why it looks different. It does look like you have the right data though.
Have I missed the point of IsoBuster here? The extraction hasn't given me the raw sector data or the 'real' data but something in between.
Ddan
Hi Ddan,
No, decompression is not in there yet either !
NTFS as it is implemented right now is supposed to work on good working (non compressed) NTFS file-systems.
That's the first step. If that works correctly then I can look into deleted MFTs, decompression etc.
> The extraction hasn't given me the raw sector data or the 'real' data but something in between.
I don't understand. What you should get is the data that is recorded. I do not convert (e.g. decompress) that data. So when I get an address (which is hopefully correct) and a length, and/or the extents (runs) and possibly an offset in a block (in case the file is in the MFT itself) then I can extract that data to HD.
If the data isn't what you expected, can you check that IsoBuster is listing the correct address ? Possibly also offset ?
What it gives is a file that is the right size for the decompressed file. There doesn't seem to be an addressing problem as I said earlier. The data in the file though is the compressed data plus the sparse clusters, all in the correct positions. I think this is what you say you are giving.
I'm not too sure I understand why you would give this though. I don't think I see its value, unless I want to decompress externally. As I mentioned earlier, the sparse clusters within a compression unit are not used anywhere. They are not really a part of the file. On the other hand, the fully sparse compression unit is part of the file and showing it as 16 clusters sort of means you have decompressed it!
Maybe if decompression isn't implemented the file should not be extractable. The other details such as extents and sector content are useful though.
Ddan
> Maybe if decompression isn't implemented the file should not be extractable.
Understood. That is something to think about.
I take it, as a for-now solution, compressed files can't be simply decompressed by a third party app ? E.g. if I were to add .zip or something to the extension *for now* so that a third party app can be used to look inside and decompress the file ?
I don't think I will implement decompression myself until version release 3.1 I need a bit more time for that. I still have a ton of things to do for the coming (3.0) release.