Anyone got a bot to find deleted truecrypt container header?

Yeah apparently this is something truecrypt does when the header is damaged for some reason, I read it in another forum happen there too. You may not be an expert in truecrypt (and maybe 4144 is), but do you know if when a file is saved it can have its first parts stored in later sectors and last parts stored in earlier sectors, or if file saving always goes for the first sectors first?

There is no "fixed rule", it depends on the available space, level of fragmentation, etc., etc., BUT - generally speaking - data is written from beginning to end (if you prefer from left to right) and when one creates a (large) container/file, he/she does so at a time the underlying file system has been recently defragged and has plenty of space available, so it is possible (but unlikely) that the file was saved with a beginning part at a later address.

But the point I was trying to make was another one.

The Truecrypt (at third attempt) must access *something* (either an actual header - even if partially corrupted - or a backup of the header - again even if partially corrupted).

IF what is accessed is NOT the real header, THEN it must be the header backup. [1]

This latter is by definition placed @S-131072 on Truecrypt 7.

So, what happens if you try the python tool on that sector (or on the area around that sector)?

jaclaz

[1] the alternative being that it accesses a sector full of "digital garbage" and attempts to interpret it nonetheless 😯

Yeah apparently this is something truecrypt does when the header is damaged for some reason, I read it in another forum happen there too. You may not be an expert in truecrypt (and maybe 4144 is), but do you know if when a file is saved it can have its first parts stored in later sectors and last parts stored in earlier sectors, or if file saving always goes for the first sectors first?

There is no "fixed rule", it depends on the available space, level of fragmentation, etc., etc., BUT - generally speaking - data is written from beginning to end (if you prefer from left to right) and when one creates a (large) container/file, he/she does so at a time the underlying file system has been recently defragged and has plenty of space available, so it is possible (but unlikely) that the file was saved with a beginning part at a later address.

But the point I was trying to make was another one.

The Truecrypt (at third attempt) must access *something* (either an actual header - even if partially corrupted - or a backup of the header - again even if partially corrupted).

IF what is accessed is NOT the real header, THEN it must be the header backup. [1]

This latter is by definition placed @S-131072 on Truecrypt 7.

So, what happens if you try the python tool on that sector (or on the area around that sector)?

jaclaz

[1] the alternative being that it accesses a sector full of "digital garbage" and attempts to interpret it nonetheless 😯

Well I copied the hex values of the last part of the file into smaller and smaller sizes until I got a 4mb of the last data. Ran it through pytruecrypt and it didn't see anything. What do you think is happening?

Well I copied the hex values of the last part of the file into smaller and smaller sizes until I got a 4mb of the last data. Ran it through pytruecrypt and it didn't see anything. What do you think is happening?

That's exactly what I don't know/don't understand ( and that has my "common sense" tingling 😯 .

I mean, no offense whatever intended ) , but the probabilities that you found an "exact" chunk (exact at least for the beginning OR for the end, and thus for End-131072) are very low.

On the other hand, the probabilities that the *whatever* is loaded has Truecrypt "accept" the password (even if on third attempt) and provide even minimal and partial data (the 80 Gb size) are IMHO even lower.

What happens if you try the Truecrypt password on only the last part of the chunk (or only the initial part of it (or on *any* chunk that is a part of the original)?

Can you try using UNTRUE on the "chunk"
https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2015/april/a-back-to-front-truecrypt-recovery-story-the-plaintext-is-the-ciphertext/

https://github.com/nccgroup/Untrue

jaclaz

That's exactly what I don't know/don't understand ( and that has my "common sense" tingling 😯 .

I mean, no offense whatever intended ) , but the probabilities that you found an "exact" chunk (exact at least for the beginning OR for the end, and thus for End-131072) are very low.

On the other hand, the probabilities that the *whatever* is loaded has Truecrypt "accept" the password (even if on third attempt) and provide even minimal and partial data (the 80 Gb size) are IMHO even lower.

Agreed, it seems very odd. I've always found TrueCrypt to be very repeatable, so something funny is going on.

Untrue should help verify the findings. It does assume that the headers will be in the right place but you can change that with the –volume_header_location option.

For pytruecrypt I've updated hunt now to deal with larger images. I'd seen that the ram usage exploded when trying to save the entropy data with pickle.

Essentially it now only stores high/low entropy and therefore needs only 1 bit per sector in the source where it used to store the full decimal number (could be around 24 bytes…)

So that means for an 8GB source file the entropy data is now 2mb v the 450mb before.

I've also added an option to output the full entropy information to a text file if that helps.

hunt <file> <passwords> --chain=256 --ent=entropy_log.txt
Would save a file called entropy_log.txt with all the entropy in it, which you could then graph in excel for example.

For pytruecrypt I've updated hunt now to deal with larger images. I'd seen that the ram usage exploded when trying to save the entropy data with pickle.

Essentially it now only stores high/low entropy and therefore needs only 1 bit per sector in the source where it used to store the full decimal number (could be around 24 bytes…)

So that means for an 8GB source file the entropy data is now 2mb v the 450mb before.

I've also added an option to output the full entropy information to a text file if that helps.

hunt <file> <passwords> --chain=256 --ent=entropy_log.txt
Would save a file called entropy_log.txt with all the entropy in it, which you could then graph in excel for example.

If the drive the container was in, is an ecrypted truecrypt volume that to the best of myr recollection wasn't formatted (unless truecrypt formats it by default probably), will chain method be of any use, will it see different entropies or just both high entropies?

Jaclaz, I'll run untrue tomorrow morning.

If the drive the container was in, is an ecrypted truecrypt volume that to the best of myr recollection wasn't formatted (unless truecrypt formats it by default probably), will chain method be of any use, will it see different entropies or just both high entropies?

It depends exactly what happened and how each volume was made. In short, if the volume is all high entropy data then there is nothing to give us hints as to where the header might be, and the chain method will not work.

Can you remind us of the full story, please? I had understood you had a 2TB disk which you had placed two large TrueCrypt containers onto. One around 80GB and the other around 50GB. I understand that the 2TB drive is not encrypted. If this is correct then the chain method should help, otherwise, you may be stuck with just brute force.

You said you got around 5GB/day with one core, assuming you have 4 cores you could get 20GB/day, so assuming you need to process the entire 2TB drive - you'll be done in just over 100 days. In practice, you will probably recover the headers faster than that. If you do intend to do this route let me update hunt so that it can at least be paused and resumed.

By default when a TrueCrypt volume is created it will consist of high entropy data inside and will slowly become less random as files are written to it and deleted. There are three possibilities that create low entropy on the outside of the container but still high on the inside. These are;
- creating a container and choosing the dynamic option
- creating a volume and choosing the 'Create encrypted volume and format it' option
- creating a system volume and choosing not to encrypt the empty space (my laptop uses GPT so I can't double check this at the moment)

So anyway, my apologies, I didn't read the instruction manual fully. The 80gb file supposedly discovered by this truecrypt backup header at the end of the drive was most probably encrypted using non-default aes-twofish and sha512, as that's what the 52gb chunk (or indeed any chunk linked to the end of the chunk) says when it opens raw in truecrypt. I modified the hunt.py like it says to do so in the manual. Now I can use the backup header to write a header into the beginning of the 52gb file, and hunt.py will indeed see it. Once it's written over of course, and by the backup. One thing hunt.py can't do is see the backup header though when it's alone and hunt hasn't seen the actual header in the same run, am I doing something wrong, or is the backup header just not available unless the header has manifested its presence in the correct place beforehand?

It depends exactly what happened and how each volume was made. In short, if the volume is all high entropy data then there is nothing to give us hints as to where the header might be, and the chain method will not work.

The volume has lots of space dedicated to virtualboxes of windows 7 that's full of books and of linux OSes with other data, but also some free space. Often the end of these files score as long successions of 000s (which I saw lot of because they're next to these free spaces I'm looking at) but of course they do have a lot of data. I'm not sure how big or small of an entropy that will show, and I didn't encrypt them, but I don't know how vdi images score on entropy levels.

Can you remind us of the full story, please? I had understood you had a 2TB disk which you had placed two large TrueCrypt containers onto. One around 80GB and the other around 50GB. I understand that the 2TB drive is not encrypted. If this is correct then the chain method should help, otherwise, you may be stuck with just brute force.

Well the 2tb is unencrypted in the sense that I have the password, but it is a 2tb encrypted truecrypt drive though, I'm not sure if I formatted the truecrypt space or not when I started. I'm not sure if that makes it low or high entropy, but an image of the drive is getting copied to hunt for it with chain now that I know the correct password crypto and hash search parameters of the first container (lord knows what the second container will be though).

You said you got around 5GB/day with one core, assuming you have 4 cores you could get 20GB/day, so assuming you need to process the entire 2TB drive - you'll be done in just over 100 days. In practice, you will probably recover the headers faster than that. If you do intend to do this route let me update hunt so that it can at least be paused and resumed.

That would be awesome, but first I should try the chain method tomorrow on the entire volume. It will probably fail though, so eventually I'll probably be using –brute. Unfortunately I can't remember the exact combination of crypto and hash that I used so like the -all function in the script, it could take me a very long time to find it. Do you know if there's a tool that can spot backup headers without having to see the actual header, just like truecrypt did on the third attempt?

Well I copied the hex values of the last part of the file into smaller and smaller sizes until I got a 4mb of the last data. Ran it through pytruecrypt and it didn't see anything. What do you think is happening?

That's exactly what I don't know/don't understand ( and that has my "common sense" tingling 😯 .

I mean, no offense whatever intended ) , but the probabilities that you found an "exact" chunk (exact at least for the beginning OR for the end, and thus for End-131072) are very low.

On the other hand, the probabilities that the *whatever* is loaded has Truecrypt "accept" the password (even if on third attempt) and provide even minimal and partial data (the 80 Gb size) are IMHO even lower.

What happens if you try the Truecrypt password on only the last part of the chunk (or only the initial part of it (or on *any* chunk that is a part of the original)?

Can you try using UNTRUE on the "chunk"
https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2015/april/a-back-to-front-truecrypt-recovery-story-the-plaintext-is-the-ciphertext/

https://github.com/nccgroup/Untrue

jaclaz

The explanation as you can see from my above post was much simpler xD, it was purely due to my noobish oversight in the encryption and hash. Indeed only the parts attached to the last area where the backup header is, react at the third attempt in truecrypt.

I don't know how to use untrue. I managed to use pytruecrypt because it was easy to mimic the steps with manuals and such, but with untrue am I supposed to be compiling the untrue c code into an exe file with something like visual studio? Is it a simple process that can be copied easily and if so could you point me to a guide? Of course I'm not sure if I need untrue now that I sort of know truecrypt was doing this because I got the crypto and the hash wrong, but if untrue decrypts half-recovered containers it could be of use as it's not looking good at all for the first container as there's not much space before the 52gb to account for the remaining 26gb of the 80gb.

So anyway, my apologies, I didn't read the instruction manual fully. The 80gb file supposedly discovered by this truecrypt backup header at the end of the drive was most probably encrypted using non-default aes-twofish and sha512, as that's what the 52gb chunk (or indeed any chunk linked to the end of the chunk) says when it opens raw in truecrypt. I modified the hunt.py like it says to do so in the manual. Now I can use the backup header to write a header into the beginning of the 52gb file, and hunt.py will indeed see it. Once it's written over of course, and by the backup. One thing hunt.py can't do is see the backup header though when it's alone and hunt hasn't seen the actual header in the same run, am I doing something wrong, or is the backup header just not available unless the header has manifested its presence in the correct place beforehand?

To be clear, have you seen hunt ever output a header? (In brute or chain mode) If you have then you have found the keys you need to decrypt the container. The whole process so far is so that we can get those keys. If you have this please post the outcome here (but not the keys or the password!).

If you think it is not AES/ripemd you'll need to use the –all option. This is super slow, but once we have the header we can make it a little faster.

The whole goal so far is to see something like this

Hunt will find a backup header without a normal header just fine, that is what example two is.

I suspect that what might be happening is that the volume header inside the container might be overwritten or in a different location, meaning that windows does not understand it. That's fine we'll just have to work harder to get the data out. What types of files are you trying to recover?

If TrueCrypt is opening the container and displaying it as raw then you should be good. In TrueCrypt select your mounted container and click volume properties. Post that screenshot as a reply.

Well the 2tb is unencrypted in the sense that I have the password, but it is a 2tb encrypted truecrypt drive though, I'm not sure if I formatted the truecrypt space or not when I started. I'm not sure if that makes it low or high entropy, but an image of the drive is getting copied to hunt for it with chain now that I know the correct password crypto and hash search parameters of the first container (lord knows what the second container will be though).

So to be clear, we are attempting to recover a TrueCrypt container that is inside another TrueCrypt container/volume?

Having the password is obviously useful but doesn't tell us about what the disk looks like. You've said there are lots of 00s which is a good thing. If you can remember when you encrypted the 2TB drive how long it took that will help us understand. If it took a long time the disk is likely filled with high entropy data (unless it has been overwritten by use), if it was very quick it will likely low entropy data.

That would be awesome, but first I should try the chain method tomorrow on the entire volume. It will probably fail though, so eventually I'll probably be using –brute. Unfortunately I can't remember the exact combination of crypto and hash that I used so like the -all function in the script, it could take me a very long time to find it. Do you know if there's a tool that can spot backup headers without having to see the actual header, just like truecrypt did on the third attempt?

For the time being, we should focus on the 52gb where we might have a header. I'll do this update at some point. I'll also make changes to brute so that it only tries to decrypt high entropy data and will therefore be faster.

@loonaloona
UNTRUE get the binary release here
https://github.com/nccgroup/Untrue/releases

jaclaz