Anyone got a bot to...
 
Notifications
Clear all

Anyone got a bot to find deleted truecrypt container header?

46 Posts
7 Users
0 Likes
7,382 Views
(@loonaluna)
Posts: 33
Eminent Member
Topic starter
 

To be clear, have you seen hunt ever output a header? (In brute or chain mode) If you have then you have found the keys you need to decrypt the container. The whole process so far is so that we can get those keys. If you have this please post the outcome here (but not the keys or the password!).

I have seen hunt output a header, but only after the following steps happened First I typed the password three times into truecrypt for the 52gb container. It told me to use the backup header to restore the header, so I did it on a backup of the 52gb container. On this new container, hunt does indeed see the header. However, I've run hunt various times on the end of the 52gb chunk once it's physically separated from the rest of the file and input into a new file, and it's never output the header. I've tested it various times on different smaller files and when the chunk is unlinked from the header, hunt won't see the backup header, and then I've tested in other files where it saw the header and the backup header, separated those files and it would also not see the backup header, however maybe I was doing something wrong, or maybe I had the hash and crypto options wrong at the time (but I doubt it).

If you think it is not AES/ripemd you'll need to use the –all option. This is super slow, but once we have the header we can make it a little faster.

–all is very slow, so what I did was I edited the hunt file so that the –else part said aes and twofish and sha512, as that's what volume tools said it was. That was when hunt started seeing the header of the container that had used this backup to write in a header at sector 0.

I suspect that what might be happening is that the volume header inside the container might be overwritten or in a different location, meaning that windows does not understand it. That's fine we'll just have to work harder to get the data out. What types of files are you trying to recover?

Yeah, that's going to be a tough one. It was an encrypted file that I couldn't remember the password to, but it probably contains important business documents from years ago. When I opened it in peazip it asked for a keyfile, which I might have as I think I saved the file that was sitting next to it and that might be it. And the password I might know if I try a number of attempts while I combine it with the keyfile. Either that or it was just another truecrypt container and that wasn't supposed to be opened in peazip. Anyway, it was a fairly medium file of 1 or 2gb in size, and I was looking to recover the file system to make it visible and easily recoverable, but if I have to go through the same process with it that I'm going through with this, I'll do it. That's why I'm also interested in seeing the second container, in case that one's undamaged and less fragmented.

If TrueCrypt is opening the container and displaying it as raw then you should be good. In TrueCrypt select your mounted container and click volume properties. Post that screenshot as a reply.

Well the 2tb is unencrypted in the sense that I have the password, but it is a 2tb encrypted truecrypt drive though, I'm not sure if I formatted the truecrypt space or not when I started. I'm not sure if that makes it low or high entropy, but an image of the drive is getting copied to hunt for it with chain now that I know the correct password crypto and hash search parameters of the first container (lord knows what the second container will be though).

So to be clear, we are attempting to recover a TrueCrypt container that is inside another TrueCrypt container/volume?

Having the password is obviously useful but doesn't tell us about what the disk looks like. You've said there are lots of 00s which is a good thing. If you can remember when you encrypted the 2TB drive how long it took that will help us understand. If it took a long time the disk is likely filled with high entropy data (unless it has been overwritten by use), if it was very quick it will likely low entropy data.

The container is inside a truecrypt encrypted secondary file storage HDD. I have the password to this HDD

Unfortunately, the 00s aren't that frequent. Maybe they are for some of the linux virtual machines sitting in there, but for the bigger windows vm, there's a lot of random stuff in there, at least according to winhex.

That would be awesome, but first I should try the chain method tomorrow on the entire volume. It will probably fail though, so eventually I'll probably be using –brute. Unfortunately I can't remember the exact combination of crypto and hash that I used so like the -all function in the script, it could take me a very long time to find it. Do you know if there's a tool that can spot backup headers without having to see the actual header, just like truecrypt did on the third attempt?

For the time being, we should focus on the 52gb where we might have a header. I'll do this update at some point. I'll also make changes to brute so that it only tries to decrypt high entropy data and will therefore be faster.

Much appreciated, but I'm not sure what's the matter with the new hunt. Before with big files it would crash on ioerror and with a pickle file that was getting too large. But now with a 2tb image it says there are too many lines, and with 200gb of collected ''free space'' by winhex, it eats up all my ram (I see the ram shootup on task manager) and I have to reboot. Neither chain nor brute seem to work on large files and, as I predict it would be easier to recover the file I'm looking for with an easily spottable file system intact I'm still interested in finding one of the containers intact.

I can run untrue on monday though, on the 52gb file, right now I'm not sure what it means when it says it decrypts, whether it only outputs hex values and if it needs to the container to be intact to output actual files.

 
Posted : 16/03/2018 10:26 pm
(@4144414d)
Posts: 33
Eminent Member
 

I have seen hunt output a header,

Then we're done. Hunt has done its job, we have the keys. It's now up to us extract the container and recover some files. More tomorrow.

 
Posted : 16/03/2018 10:35 pm
(@loonaluna)
Posts: 33
Eminent Member
Topic starter
 

So yesterday I got back on with untrue, to do what I think is decrypt (untrue.exe -p password -i inputfile -o outputfile) and I don't know what to do with the resulting output file. It's just a file that doesn't open anywhere, not in daemon tools, not in peazip, not in truecrypt of course. The same happens when I try outputting an input file that I know is perfectly healthy. What could I be doing wrong?

Hunt will find a backup header without a normal header just fine, that is what example two is.

Also, I looked into what was happening with hunt.py, to make sure it doesn't see the backup header if it hasn't previously run through the actual header. And it doesn't, at least in my experience. It gets the same error any random chunk of files without a backup header gets, a string index out of range issue which I can't quite parse. Even a chunk that starts with a header, but doesn't have the backup header, finishes with this warning.

 
Posted : 20/03/2018 2:43 pm
jaclaz
(@jaclaz)
Posts: 5133
Illustrious Member
 

I believe there has been a communication problem of some kind.

The UNTRUE can be run in three main modes
1) Check a password against a TrueCrypt volume or drive
2) Decrypt a TrueCrypt volume or drive, using a given key
3) Check a password against a TrueCrypt volume or drive, and then use the resultant key to decrypt the volume or drive contents

In order to be able to use it in mode 2) above you need a source, i.e. a Truecrypt volume (which you DO NOT have).
Right now you have a "chunk" of data that you don't know if it corresponds or not to the Truecrypt volume (and probably has a number of corrupted/overwritten sector) so you cannot run it in mode 2).

The modes 1) and 3) can have as argument/source also a Truecrypt header.

You seem to have been able to identify the backup header.

The idea was to try the UNTRUE on this backup header using the –volume_header_file parameter in mode 1).

Check the testcases
https://github.com/nccgroup/Untrue/tree/master/Untrue/TestCases

In any case, if you try mode 2) on a "good" input file (for tests) the output will be (hopefully) a decrypted volume or whole disk image, which you will be able to "open" only in a hex editor (or - it depends - with 7-zip or other tool capable of mounting/accessing volumes/disks image).

Mind you I have no idea if the UNTRUE will actually work because it is limited to a give kind of encryption, and it is aimed to the "main" header, not the backup one, from what I can understand.

jaclaz

 
Posted : 20/03/2018 3:33 pm
(@loonaluna)
Posts: 33
Eminent Member
Topic starter
 

I'm running UNTRUE by decrypting a clean test 5mb truecrypt container that I know isn't corrupted, and still daemon tools can't open it, saying it might be corrupt. Plus, after decryption, the password doesn't work so I can't even access the data anymore. Also, under algorithm options in the UNTRUE wiki, it says that when decrypting based a successful password check, the algorithm specified in the decrypted volume header will be used.

I'm interested in the issues hunt.py might be having when trying to see lone backup headers to increase my chances of finding the other container. But I've copied the 52gb extracted chunk and written the header in with the help of the backup header, as advised to do so by the truecrypt program, in volume tools. And it's with this corrected file, that opens raw and unreadable in truecrypt by punching in the password just once (whereas in the uncorrected 52gb chunk you have to type it in 3 times and though it then opens, truecrypt advises to write the header in again), that I'm running UNTRUE.

 
Posted : 20/03/2018 4:07 pm
jaclaz
(@jaclaz)
Posts: 5133
Illustrious Member
 

I'm running UNTRUE by decrypting a clean test 5mb truecrypt container that I know isn't corrupted, and still daemon tools can't open it, saying it might be corrupt.

Well, Daemon tools is the LAST program you should use for that anyway
You should use a suitable hex editor (or dmde) to open the resulting file.
In any case it is entirely possible that for whatever reasons you are using the tool improperly (with command line tools even a minor typo might cause issues) or maybe it simly doesn't work properly in your OS/setup.

But anyway, this is not what I had suggested earlier, that was only a way to confirm that the header was valid.

… And it's with this corrected file, that opens raw and unreadable in truecrypt by punching in the password just once (whereas in the uncorrected 52gb chunk you have to type it in 3 times and though it then opens, truecrypt advises to write the header in again), that I'm running UNTRUE.

And again this is not what I suggested.

Try going on your machine with the testcases provided with the tool.
This way you can confirm that in your setup the tool works as expected.
Then try using the same approach of testcases on the backup header and see what happens.

It is still not clear to me if you managed to find/isolate the backup header and if you successfully decrypted it, i.e. if you managed to obtain a 512 byte file that in a hex heditor looks similar to the example posted earlier

Sector 28160 Fully valid header found
Hash Option ripemd
Crypto Option ['aes']
Password password

Decrypted Header
0000 54 52 55 45 00 05 07 00 2e c2 dd d8 00 00 00 00 TRUE…………
0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0020 00 00 00 00 00 00 00 00 00 9c 00 00 00 00 00 00 …………….
0030 00 02 00 00 00 00 00 00 00 9c 00 00 00 00 00 00 …………….
0040 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
00a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
00b0 00 00 00 00 00 00 00 00 00 00 00 00 b1 2d f8 8c ………….-..
00c0 f3 00 d5 78 08 30 66 2f 17 99 08 27 28 17 c2 20 …x.0f/…'(..
00d0 b7 2e 9a 14 79 da 01 77 63 98 37 af 75 da 95 41 ….y..wc.7.u..A
00e0 93 1f f6 7e 13 d3 b3 c5 de f2 2f cc 00 b5 98 b9 …~……/…..
00f0 77 55 00 d4 5b b4 e4 7c 77 7e 5e 65 a3 ec 32 c3 wU..[..|w~^e..2.
0100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
0190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
01a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….
01b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 …………….

If you have the above, can you post it's contents?
The values in it may help to determine if the chunk size is correct.

Now, please follow me for a moment, just trying to explain the situation as I see it and (hopefully) save your time in attempts with various tools/programs that won't lead you anywhere.

Let us assume that you found the exact extents of the Truecrypt volume.
Since all that was found was the backup header and *somehow* you need to rebuild via Truecrypt the main header, at least a part (small or large) at the beginning of the volume is corrupted.
Now, even if the sectors inside the volumes are correctly decrypted (by truecrypt, hunt.py, untrue, *whatever*) the unencrypted output will be anyway a (seriously) corrupted disk or volume image.
There is no way any tool will properly access/read/mount the output properly.
The only tools that you can use is a hex/disk editor or a tool like dmde (i.e. a recovery tool).

jaclaz

 
Posted : 20/03/2018 4:53 pm
(@loonaluna)
Posts: 33
Eminent Member
Topic starter
 

Thanks yet again for your remarks jaclaz. For some reason I'd totally forgotten about DMDE. I'll work through your suggestions.

 
Posted : 20/03/2018 7:07 pm
(@4144414d)
Posts: 33
Eminent Member
 

so I did it on a backup of the 52gb container

The screen from TrueCrypt you posted shows that it thinks the volume is 85899083776 bytes (about 86GB) so we're missing a lot of this container to try and rebuild it. Parts of it will have been overwritten now by the other files.

Also, I looked into what was happening with hunt.py, to make sure it doesn't see the backup header if it hasn't previously run through the actual header.

I am quite sure that hunt will find backup headers without seeing a normal header. That is quite literally what example two shows.

https://github.com/4144414D/pytruecrypt/blob/master/examples/hunt.md#example-2--a-damaged-container

The container is inside a truecrypt encrypted secondary file storage HDD. I have the password to this HDD

Unfortunately, the 00s aren't that frequent

This is the reason that hunt cannot find the headers so easily.

Now, even if the sectors inside the volumes are correctly decrypted (by truecrypt, hunt.py, untrue, *whatever*) the unencrypted output will be anyway a (seriously) corrupted disk or volume image.

There is no way any tool will properly access/read/mount the output properly.
The only tools that you can use is a hex/disk editor or a tool like dmde (i.e. a recovery tool).

Yes, this is exactly the problem. We have the keys to open the container, but now we need to reconstruct the container in the right order. Then when it is decrypted, the resulting volume is likely to be damaged.

Even a chunk that starts with a header, but doesn't have the backup header, finishes with this warning.

It looks like you have found a bug the utility that prints information to the screen when a possible header is found. It's not something I've ever seen before. I can't really debug it remotely, I would need you to post sector 3759 of the file you are running against, and the password you are trying. I appreciate you probably don't want to share that though.

I can make changes to the script to handle the error but I don't have the time to look at this at the moment. I see you've edited the script yourself for other things, so if you fix this before me you can make a pull request and I can merge your code.

EDIT Also, given how overwritten this container looks to be it's going to get increasingly more complicated to explain. If you wanted to send me the 2TB image I will quite happily have a go at the recovery job myself.

 
Posted : 20/03/2018 7:59 pm
(@loonaluna)
Posts: 33
Eminent Member
Topic starter
 

Jaclaz and 4144414D,

This week instead of jumping at everything, I paused a little to think about what I was doing. I went through the examples and made sure everything was working properly on my setup. Turns out the 'string index out of range' issue refers to how I was willy-nilly copying parts near the end of the chunk. I should've been selecting an entire sector as the start of the block, but I was just using a random hex value, sometimes even dividing it into further smaller pieces using files that merely consisted of pages in winhex and no longer had any concept of what a sector was. So once I figured this out, selected a beginning of a block that was clearly determined as a full sector, I did indeed get the backup header. It's important to note this before I send 41D on a wild goose chase thinking this is a bug that thinks it's found a header. String index out of range is merely an issue that pops up when we do something as nonsensical as start a chunk in the middle of a sector xD. It happens both when there is and isn't a backup header actually sitting there.

Even with this backup header though, truecrypt claims it's an 80gb container (but when I click volume tools it says 86gb, so corruption, dynamic size expansion I may have forgotten about? etc…), and the chunk is 52gb, so I probably have to assume those 30gb or so are sitting in previous sectors, despite fragmentation, and in the worst case scenario, that they've been overwritten. With this in mind, how deadly is the slightest damage to a container? Does the file table data that upholds data file structure and gives filenames to the hex values, lie at the beginning of the container, the part most prone to getting overwritten if an HDD does indeed write from left to write like humans do? I tried dmde on the chunk but it doesn't see anything that makes sense, just a bunch of insanely huge png files. Whether they signify something more coherent or are they just garbage, I'm inclined to think the latter.

I'm open to suggestions, but my understanding right now is that –chain wouldn't help me much. Most of the other files on the HDD probably have a big entropy, stuff like a windows virtualbox covers half the drive in a fragmented manner. Sometimes there's a file bordering the free space that starts off with plenty of zeroes, like an iso of a videogame, but that's probably a rarity. Btw, I need bitarray to run the new improved hunt.py, and for that I'll download a different python as I'm on windows, so I haven't ran that yet. I tried to run a –chain of the old hunt on the 52gb extent as it sat on a decrypted formatted truecrypt container, but I got the usual memoryerror and eoferror. Only then did I realize that to get the backup header all I had to do was run a –brute of the last .5mb or so of the chunk, as long as the beginning of the .5mb consisted of an entire sector.

I think –brute could help me. Or even better, a program that combines the best of defraggler with the best of winhex. That way I could make extents out of free spaces that are easy to see on defraggler but whose specific sector number can only be found in winhex. Do you guys know of any such contraption? After all, I only know of half a dozen possible passwords and I can't remember the specific crypto hash combination I used, and trying to open extents of ends or beginnings of free space would probably be faster if done straight through truecrypt than in hunt, which goes very slow once we have to start doing different hash crypto combinations. This is bearing in mind that I have yet to find where the second container is sitting, and that despite fragmentation, the first container might be recoverable if spliced up together. Of course I could be wrong on the second count or both counts.

Thank you very much for your offer to do it yourself 41D. I really want to try it myself though and the data probably holds info that can't be shared. I'm not against in the future getting someone else to do it for me, but for now and quite some time, I'd prefer to try it myself.

 
Posted : 24/03/2018 4:22 am
jaclaz
(@jaclaz)
Posts: 5133
Illustrious Member
 

@loonaloona
Yes, what you report is "normal" the whole idea of a "block device" is that the accessible unit is a block, the disk exposes "sectors" so you normally go by 512 bytes and multiples.
I believe that a number of programs that allow read and writing parts of the sector are simply caching the whole sector or using a diffeernt mode of access.

About the volume, 30 GB sound a lot to be overwritten, but of course it is possible.

Until we were in the smaller 2-3 GB range I was more optimistic as normally on a NTFS volume the $MFT, i.e. the main indexing structure is generally (for volumes larger than around 5-6 GB) at cluster 0xC0000 i.e. 786432 which on a volume with 8 sectors per cluster means 3,221,225,472 bytes from start, so likely it could have been recovered, if not fully at least in large parts.

I have no idea if - before and outside the file recovery on the decrypted volume - the exact way the decryption happens, i.e. if - as it seems from the article about UNTRUE and from what I understood of the nice little scripts by 4144414D - the contents are decrypted correctly independently from their relative offfset. ?

I mean, if I got your report right ? , you "forced" a Truecrypt header rercovery on a "chunk" that is smaller than the size of the original volume.
Is this right?
If now you have the exact size (thanks to the decryption of the backup header) you should try making the "chunk" the exact size, by either prepending to the 50 Gb or so "chunk" you have the sectors actually on disk before it or inserting at the beginning a suitable number of empty sectors.

DMDE will (if it can find no usable traces of the filesystems structures) attempt to do a "file signature" carving like Photorec would do (photorec being the alternative tool you may try on the data).

Of course it depends on the actual contents that were on the volume before, but it is unlikely that on such a large container no "valid" file can be found.

That may mean any of
1) ALL files were of types not identifiable by signature or pattern
2) ALL files were fragmented
3) ALL files inside the encrypted container were themselves encrypted
4) the decryption was not successful

Right now I believe hypothesis #4 to be the more likely.

Though it is a lot of work, I would try making a "timeline" map of the disk (of course working on a copy).
I mean, let us assume that what happened was
1) the container was 86 GB
2) the container was deleted
3) parts of the area(s) where the container originally was have been overwritten by new files

The #3 above may happen loosely in two ways
a. a new file has been saved to the disk and happened to "land" on the area previously occupied by the container
b. an existing file has been enlarged to the point that part(s) of it were saved (fragmented) on that same area previously occupied by the container

So, each and every file with created and modified date before the date you deleted the container surely did not occupy the area where the container was.

Of course a number of files created and modified after the same date didn't happen to "land" on that area.

A number of files created before the date and modified later will NOT occupy the ex-container area IF they are NOT fragmented.

But, by making two (sorts of) "maps" before and after the date you might be able to approximately identify the boundaries of the 30 GB "missing" (of course assuming it was a unique 30 GB chunk) or maybe a couple 15 GB chunks.

There is not a "written law" about how a file is written on a NTFS, but it is unlikely that when the 86 GB container was created the algorithms made (say) 1475 fragments of the first 30 GB and then found and wrote some 50 GB in a contiguous set of unallocated space, it is more likley that this hypotherical fragmentation is anyway very low, like 1 30 GB chunk+1 50 GB chunk or 1 20 GB chunk + 1 10 GB chunk + 1 50 GB chunk.

jaclaz

 
Posted : 24/03/2018 10:04 am
Page 4 / 5
Share: