±Forensic Focus Partners

Become an advertising partner

±Your Account


Username
Password

Forgotten password/username?

Site Members:

New Today: 0 Overall: 36209
New Yesterday: 3 Visitors: 173

±Follow Forensic Focus

Forensic Focus Facebook PageForensic Focus on TwitterForensic Focus LinkedIn GroupForensic Focus YouTube Channel

RSS feeds: News Forums Articles

±Latest Articles

±Latest Videos

±Latest Jobs

Anyone got a bot to find deleted truecrypt container header?

Computer forensics discussion. Please ensure that your post is not better suited to one of the forums below (if it is, please post it there instead!)
Reply to topicReply to topic Printer Friendly Page
Forum FAQSearchView unanswered posts
Page Previous  1, 2, 3, 4, 5, 6 
  

jaclaz
Senior Member
 

Re: Anyone got a bot to find deleted truecrypt container hea

Post Posted: Mar 20, 18 16:53

- loonaluna
I'm running UNTRUE by decrypting a clean test 5mb truecrypt container that I know isn't corrupted, and still daemon tools can't open it, saying it might be corrupt.

Well, Daemon tools is the LAST program you should use for that anyway
You should use a suitable hex editor (or dmde) to open the resulting file.
In any case it is entirely possible that for whatever reasons you are using the tool improperly (with command line tools even a minor typo might cause issues) or maybe it simly doesn't work properly in your OS/setup.

But anyway, this is not what I had suggested earlier, that was only a way to confirm that the header was valid.

- loonaluna

... And it's with this corrected file, that opens raw and unreadable in truecrypt by punching in the password just once (whereas in the uncorrected 52gb chunk you have to type it in 3 times and though it then opens, truecrypt advises to write the header in again), that I'm running UNTRUE.

And again this is not what I suggested.

Try going on your machine with the testcases provided with the tool.
This way you can confirm that in your setup the tool works as expected.
Then try using the same approach of testcases on the backup header and see what happens.

It is still not clear to me if you managed to find/isolate the backup header and if you successfully decrypted it, i.e. if you managed to obtain a 512 byte file that in a hex heditor looks similar to the example posted earlier :



Sector 28160: Fully valid header found
Hash Option: ripemd
Crypto Option: ['aes']
Password: password

Decrypted Header:
0000 54 52 55 45 00 05 07 00 2e c2 dd d8 00 00 00 00 TRUE............
0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0020 00 00 00 00 00 00 00 00 00 9c 00 00 00 00 00 00 ................
0030 00 02 00 00 00 00 00 00 00 9c 00 00 00 00 00 00 ................
0040 00 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00b0 00 00 00 00 00 00 00 00 00 00 00 00 b1 2d f8 8c .............-..
00c0 f3 00 d5 78 08 30 66 2f 17 99 08 27 28 17 c2 20 ...x.0f/...'(..
00d0 b7 2e 9a 14 79 da 01 77 63 98 37 af 75 da 95 41 ....y..wc.7.u..A
00e0 93 1f f6 7e 13 d3 b3 c5 de f2 2f cc 00 b5 98 b9 ...~....../.....
00f0 77 55 00 d4 5b b4 e4 7c 77 7e 5e 65 a3 ec 32 c3 wU..[..|w~^e..2.
0100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
01a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
01b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................



If you have the above, can you post it's contents?
The values in it may help to determine if the chunk size is correct.

Now, please follow me for a moment, just trying to explain the situation as I see it and (hopefully) save your time in attempts with various tools/programs that won't lead you anywhere.

Let us assume that you found the exact extents of the Truecrypt volume.
Since all that was found was the backup header and *somehow* you need to rebuild via Truecrypt the main header, at least a part (small or large) at the beginning of the volume is corrupted.
Now, even if the sectors inside the volumes are correctly decrypted (by truecrypt, hunt.py, untrue, *whatever*) the unencrypted output will be anyway a (seriously) corrupted disk or volume image.
There is no way any tool will properly access/read/mount the output properly.
The only tools that you can use is a hex/disk editor or a tool like dmde (i.e. a recovery tool).

jaclaz
_________________
- In theory there is no difference between theory and practice, but in practice there is. - 
 
  

loonaluna
Member
 

Re: Anyone got a bot to find deleted truecrypt container hea

Post Posted: Mar 20, 18 19:07

Thanks yet again for your remarks jaclaz. For some reason I'd totally forgotten about DMDE. I'll work through your suggestions.  
 
  

4144414D
Member
 

Re: Anyone got a bot to find deleted truecrypt container hea

Post Posted: Mar 20, 18 19:59

- loonaluna
so I did it on a backup of the 52gb container


The screen from TrueCrypt you posted shows that it thinks the volume is 85899083776 bytes (about 86GB) so we're missing a lot of this container to try and rebuild it. Parts of it will have been overwritten now by the other files.


- loonaluna
Also, I looked into what was happening with hunt.py, to make sure it doesn't see the backup header if it hasn't previously run through the actual header.


I am quite sure that hunt will find backup headers without seeing a normal header. That is quite literally what example two shows.

github.com/4144414D/py...-container

- loonaluna
The container is inside a truecrypt encrypted secondary file storage HDD. I have the password to this HDD

Unfortunately, the 00s aren't that frequent


This is the reason that hunt cannot find the headers so easily.


- jaclaz
Now, even if the sectors inside the volumes are correctly decrypted (by truecrypt, hunt.py, untrue, *whatever*) the unencrypted output will be anyway a (seriously) corrupted disk or volume image.

There is no way any tool will properly access/read/mount the output properly.
The only tools that you can use is a hex/disk editor or a tool like dmde (i.e. a recovery tool).


Yes, this is exactly the problem. We have the keys to open the container, but now we need to reconstruct the container in the right order. Then when it is decrypted, the resulting volume is likely to be damaged.


- loonaluna
Even a chunk that starts with a header, but doesn't have the backup header, finishes with this warning.


It looks like you have found a bug the utility that prints information to the screen when a possible header is found. It's not something I've ever seen before. I can't really debug it remotely, I would need you to post sector 3759 of the file you are running against, and the password you are trying. I appreciate you probably don't want to share that though.

I can make changes to the script to handle the error but I don't have the time to look at this at the moment. I see you've edited the script yourself for other things, so if you fix this before me you can make a pull request and I can merge your code.

EDIT: Also, given how overwritten this container looks to be it's going to get increasingly more complicated to explain. If you wanted to send me the 2TB image I will quite happily have a go at the recovery job myself.  
 
  

loonaluna
Member
 

Re: Anyone got a bot to find deleted truecrypt container hea

Post Posted: Mar 24, 18 04:22

Jaclaz and 4144414D,

This week instead of jumping at everything, I paused a little to think about what I was doing. I went through the examples and made sure everything was working properly on my setup. Turns out the 'string index out of range' issue refers to how I was willy-nilly copying parts near the end of the chunk. I should've been selecting an entire sector as the start of the block, but I was just using a random hex value, sometimes even dividing it into further smaller pieces using files that merely consisted of pages in winhex and no longer had any concept of what a sector was. So once I figured this out, selected a beginning of a block that was clearly determined as a full sector, I did indeed get the backup header. It's important to note this before I send 41D on a wild goose chase thinking this is a bug that thinks it's found a header. String index out of range is merely an issue that pops up when we do something as nonsensical as start a chunk in the middle of a sector xD. It happens both when there is and isn't a backup header actually sitting there.

Even with this backup header though, truecrypt claims it's an 80gb container (but when I click volume tools it says 86gb, so corruption, dynamic size expansion I may have forgotten about? etc...), and the chunk is 52gb, so I probably have to assume those 30gb or so are sitting in previous sectors, despite fragmentation, and in the worst case scenario, that they've been overwritten. With this in mind, how deadly is the slightest damage to a container? Does the file table data that upholds data file structure and gives filenames to the hex values, lie at the beginning of the container, the part most prone to getting overwritten if an HDD does indeed write from left to write like humans do? I tried dmde on the chunk but it doesn't see anything that makes sense, just a bunch of insanely huge png files. Whether they signify something more coherent or are they just garbage, I'm inclined to think the latter.


I'm open to suggestions, but my understanding right now is that --chain wouldn't help me much. Most of the other files on the HDD probably have a big entropy, stuff like a windows virtualbox covers half the drive in a fragmented manner. Sometimes there's a file bordering the free space that starts off with plenty of zeroes, like an iso of a videogame, but that's probably a rarity. Btw, I need bitarray to run the new improved hunt.py, and for that I'll download a different python as I'm on windows, so I haven't ran that yet. I tried to run a --chain of the old hunt on the 52gb extent as it sat on a decrypted formatted truecrypt container, but I got the usual memoryerror and eoferror. Only then did I realize that to get the backup header all I had to do was run a --brute of the last .5mb or so of the chunk, as long as the beginning of the .5mb consisted of an entire sector.

I think --brute could help me. Or even better, a program that combines the best of defraggler with the best of winhex. That way I could make extents out of free spaces that are easy to see on defraggler but whose specific sector number can only be found in winhex. Do you guys know of any such contraption? After all, I only know of half a dozen possible passwords and I can't remember the specific crypto hash combination I used, and trying to open extents of ends or beginnings of free space would probably be faster if done straight through truecrypt than in hunt, which goes very slow once we have to start doing different hash crypto combinations. This is bearing in mind that I have yet to find where the second container is sitting, and that despite fragmentation, the first container might be recoverable if spliced up together. Of course I could be wrong on the second count or both counts.


Thank you very much for your offer to do it yourself 41D. I really want to try it myself though and the data probably holds info that can't be shared. I'm not against in the future getting someone else to do it for me, but for now and quite some time, I'd prefer to try it myself.  
 
  

jaclaz
Senior Member
 

Re: Anyone got a bot to find deleted truecrypt container hea

Post Posted: Mar 24, 18 10:04

@loonaloona
Yes, what you report is "normal" the whole idea of a "block device" is that the accessible unit is a block, the disk exposes "sectors" so you normally go by 512 bytes and multiples.
I believe that a number of programs that allow read and writing parts of the sector are simply caching the whole sector or using a diffeernt mode of access.

About the volume, 30 GB sound a lot to be overwritten, but of course it is possible.

Until we were in the smaller 2-3 GB range I was more optimistic as normally on a NTFS volume the $MFT, i.e. the main indexing structure is generally (for volumes larger than around 5-6 GB) at cluster 0xC0000 i.e. 786432 which on a volume with 8 sectors per cluster means 3,221,225,472 bytes from start, so likely it could have been recovered, if not fully at least in large parts.

I have no idea if - before and outside the file recovery on the decrypted volume - the exact way the decryption happens, i.e. if - as it seems from the article about UNTRUE and from what I understood of the nice little scripts by 4144414D - the contents are decrypted correctly independently from their relative offfset. Question

I mean, if I got your report right Confused , you "forced" a Truecrypt header rercovery on a "chunk" that is smaller than the size of the original volume.
Is this right?
If now you have the exact size (thanks to the decryption of the backup header) you should try making the "chunk" the exact size, by either prepending to the 50 Gb or so "chunk" you have the sectors actually on disk before it or inserting at the beginning a suitable number of empty sectors.

DMDE will (if it can find no usable traces of the filesystems structures) attempt to do a "file signature" carving like Photorec would do (photorec being the alternative tool you may try on the data).

Of course it depends on the actual contents that were on the volume before, but it is unlikely that on such a large container no "valid" file can be found.

That may mean any of:
1) ALL files were of types not identifiable by signature or pattern
2) ALL files were fragmented
3) ALL files inside the encrypted container were themselves encrypted
4) the decryption was not successful

Right now I believe hypothesis #4 to be the more likely.

Though it is a lot of work, I would try making a "timeline" map of the disk (of course working on a copy).
I mean, let us assume that what happened was:
1) the container was 86 GB
2) the container was deleted
3) parts of the area(s) where the container originally was have been overwritten by new files

The #3 above may happen loosely in two ways:
a. a new file has been saved to the disk and happened to "land" on the area previously occupied by the container
b. an existing file has been enlarged to the point that part(s) of it were saved (fragmented) on that same area previously occupied by the container

So, each and every file with created and modified date before the date you deleted the container surely did not occupy the area where the container was.

Of course a number of files created and modified after the same date didn't happen to "land" on that area.

A number of files created before the date and modified later will NOT occupy the ex-container area IF they are NOT fragmented.

But, by making two (sorts of) "maps" before and after the date you might be able to approximately identify the boundaries of the 30 GB "missing" (of course assuming it was a unique 30 GB chunk) or maybe a couple 15 GB chunks.

There is not a "written law" about how a file is written on a NTFS, but it is unlikely that when the 86 GB container was created the algorithms made (say) 1475 fragments of the first 30 GB and then found and wrote some 50 GB in a contiguous set of unallocated space, it is more likley that this hypotherical fragmentation is anyway very low, like 1 30 GB chunk+1 50 GB chunk or 1 20 GB chunk + 1 10 GB chunk + 1 50 GB chunk.

jaclaz
_________________
- In theory there is no difference between theory and practice, but in practice there is. - 
 
  

loonaluna
Member
 

Re: Anyone got a bot to find deleted truecrypt container hea

Post Posted: Mar 25, 18 00:47

I have a vague idea that very little writing happened after the deletion date. I've gone over various times looking at the file dates and such. However I'm not sure of the depth of the file writing, it may be a lot, and if it writes in a fragmented way then I'm screwed. Most of the danger will come from a steam game update that happened and maybe an accidental install that I didn't turn off on time. I have a mental picture of this timestamp, but I can't go in perfect detail with winhex as it doesn't have an easier GUI to spot the free space, something like a graphical depiction like defraggler does. Something with the graphical free space identifier of a defraggler and the actual sector number identifier in winhex would be the perfect combination. Do you know of such a tool? or am I condemned to having to go through winhex every few thousand sectors spotting the free spaces in theiir specific sectors? If so I'm up for a very tedious affair what with there being a gazillion sectors. Winhex has a great feature named gather free space that is so close to being great but not quite, as I'm back in square one and all I have is a homogenous chunk that's terribly hard to deal with and it saves in the form of pages, not sectors, and it saves all the supposed free space, you can't go selecting parts of the free space to extract, and you can't select part of the homogenous chunk as you'll end up selecting in the middle of a sector and that'll give nothing but index out of range errors.

A couple things about what you say too, are you saying that if a file is created but already fragmented before the deletion, if it modifies after the deletion it will just skip willy nilly into the a huge portion of the free space? Or are you merely saying that if they're modified a ton and increase in size a lot, they'll fragment further and spill into the free space?

You got it right I did recover the backup header, and this 52gb does start to seem like the end of the first container. There's a fair amount of ''free space'' on winhex before it but it's in a fragmented manner, but in a large style, consisting of fairly large continuous extents each, in keeping in tone with the tendency of the drive to have lots of large files, like a massive virtual machine image that's all over the drive in a fragmented manner but in a large way. Most of the occupied filespace belongs to large files that were there before the deletion. Are you saying DMDE (or Photorec) will recover the filenames better if the container is the full size truecrypt claims it is, even if it's just empty data added before the 52gb chunk? And that if I just added empty data to fill up to truecrypt's advertised data size, DMDE would have a better chance of identifying the files successfully? Not that I want to reach that level yet, as I hope I can add actual free space on the drive instead, and if not, I want to have a go at the second container. Or maybe it has to be a perfect container, and the slightest of damage means the file reconstruction will always be jumbled up garbage? Maybe the container stores its MFT data, the ones that I understand would give shape or form to the file reconstruction, at a specific place of the container that must never be deleted, maybe they're stored at the beginning of the container?  
 
  

jaclaz
Senior Member
 

Re: Anyone got a bot to find deleted truecrypt container hea

Post Posted: Mar 25, 18 09:28

@loonaloona
Imagine that you have a file using exactly 8 sectors, where F is the file, 0 is empty (unallocated) space and S is Something_else, if the situation on disk is:
... 0000FFFFFFFF000000SSSSS ...
when you add a sector worth of data the result will be N being the new data added to F:
... 0000FFFFFFFFN00000SSSSS ...
the file remains contiguous, BUT IF the situation before was:
... 0000FFFFFFFFSSS000SSSSS ...
the result will be a non-contiguous file:
... 0000FFFFFFFFSSSN00SSSSS ...

Now, on how to proceed, I would do it analytically, not graphically.

As said anything that has both created and modified date before the deletion is "safe", in the sense that it cannot possibly be (unless of course a defrag has been run) occupying the space where the container was.

I would use a DIR command (or similar) to make a list of all files with their created and modified times.

Then I would make a list for each file of its extents on disk (now) as clusters and put everything in a spreadsheet.

Extents of files that:
1) were created and modified before the deletion can be removed form the map
2) were created before but modified after the deletion can be removed from the map (only if contiguous
3) belong to the NTFS structures can be removed from the map

What remains is either:
a. files created after deletion or modified after deletion AND fragmented
b. your deleted truecrypt container extents

DMDE can create a "cluster map" (Tools->Cluster map) but it is - while easier to navigate than the actual sector view, still IMHO not suitable as you will have a zillion files.

Making a map with cluster numbers in a spreadsheet will allow to better navigate it.

Now, the issue might be which specific tools to use for creating the base of data to insert in the spreadsheet.

Most probably the nice tools by Joachim Schicht can be of use:
github.com/jschicht?ta...positories

MFT2CSV:
github.com/jschicht/Mf...ki/Mft2Csv
And USNJRNL2CSV:
github.com/jschicht/UsnJrnl2Csv

being the first ones I would try.

Then I would try running via batch *something like* the myfragmenter.exe, that you can find here (part of Mydefrag 4.31):
www.softpedia.com/get/...frag.shtml

A sample batch (that I call myfragi.cmd) follows:
Code:
@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
Set InitialLBA=63
Set clusterSize=8
Set File=%~dpnx1
Set Timestamp=%~t1

IF NOT EXIST myfragi.log ECHO Ext:	Lcn:	LBAstart:	Sects:	Date:	File:>myfragi.log	
ECHO Ext:	Lcn:	LBAstart:	Sects:	File:
FOR /F "tokens=2,4,6,8,9 delims=:=, " %%A IN ('myfragmenter.exe -i "%File%" ^| FIND "="') DO (
CALL :convert_LBA %%A %%B %%C %%D

ECHO !Extent!	!Lcn!	!LBAstart!	!Sectors!	%Timestamp%	%File%
ECHO !Extent!	!Lcn!	!LBAstart!	!Sectors!	%Timestamp%	%File%>>myfragi.log
)

ECHO.>>myfragi.log
GOTO :EOF

:convert_LBA
SET /A Extent=%1
SET /A Lcn=%2
SET /A LBAstart=%InitialLBA%+%2*%clusterSize%
SET /A Sectors=%clusterSize%*(%4-%3)
CALL :set_length Extent 4
CALL :set_length Lcn 11
CALL :set_length LBAstart 12
CALL :set_length Sectors 7
GOTO :EOF

:set_length
SET %1=            !%1!
SET %1=!%1:~-%2,%2!
GOTO :EOF

jaclaz
_________________
- In theory there is no difference between theory and practice, but in practice there is. - 
 

Page 6 of 6
Page Previous  1, 2, 3, 4, 5, 6