Join Us!

e-zero - helps mana...
 
Notifications
Clear all

e-zero - helps manage e01 files  

  RSS
4144414D
(@4144414d)
Junior Member

Hello all,

I made a small tool to help automate FTK Imager CLI. This is mostly to save me the human time of moving forensic images, verifying a bunch of images, or reacquiring them to save space.

I tried to make it run in parallel as much as possible as possible, but only read or write once from a disk at a time. So if you have 2 source disks and 2 destinations it'll do 2 copies at a time, rather then 4 so that overall the copies go faster.

Hosted on GitHub so feel free to tell me how bad the code is!

https://4144414d.github.io/e-zero/

A quick preview

and it has a context menu for "Right Click Forensics"

Adam

Quote
Posted : 08/08/2015 5:47 pm
jaclaz
(@jaclaz)
Community Legend

Seems nice. ) (voted yes).

Cannot say how common it is the need to verify "in bulk" a whole SAN (or whatever large storage media) filled of .E01 images, but the consolidating is probably the most useful feature.

I am failing (at first sight) to understand the *need* to use the thingy to re-acquire an image (in the sense of improvement over re-acquiring it directly with FTK CLI) ? .

Three things (two small typos and a philosophical question)

  1. the link to DOCOPT is not working
  2. terrabytes seem like a rather large unit of measure wink
  3. at its core, isn't it a (nice) Python script?
  4. [/listo]

    If #3 is correct, than maybe you could provide also just the script, while the self-contained installer is a nice idea, I believe that most of the "intended audience" will have already a Python interpreter installed (or add a link to the project page)
    https://github.com/4144414D/e-zero

    jaclaz

ReplyQuote
Posted : 08/08/2015 11:20 pm
4144414D
(@4144414d)
Junior Member

Should have gone with TERRORbytes! I did laugh to my self when I saw the link to docopt… <a href="LINK TO DOCOPT">docopt</a> thanks for the heads up on that one!

It sure is just a python script, you can get the source by clicking the 'Download Source' button or you can follow the 'View on GitHub' link to get to the main project page, maybe I should make the icon bigger. (or just click here for the script). The other small benefit of the installer is that it sets up the context menu for verifying, but thats about it.

For me I do mostly use it for consolidation, connect a few drives and leave it over night and the two copies are ready in the morning.

The re-acquire is mostly because it's offered by the FTK Imager, so I thought why not, maybe one day someone will need to compress 10+ images at once. I think I've only used it once myself.

Adam

ReplyQuote
Posted : 09/08/2015 2:04 am
jaclaz
(@jaclaz)
Community Legend

The other small benefit of the installer is that it sets up the context menu for verifying, but thats about it.

Yep, which is something that can be seen as a feature ) or as an annoyance ( , being like beauty in the eye of the beholder.
Not a critic to your thingy, mind you, only if I had right-click menu provision for each program I have on my machine I would probably need a second monitor to allow all of them to show.

jaclaz

ReplyQuote
Posted : 09/08/2015 5:27 pm
4144414D
(@4144414d)
Junior Member

Yep, which is something that can be seen as a feature ) or as an annoyance ( , being like beauty in the eye of the beholder.
Not a critic to your thingy, mind you, only if I had right-click menu provision for each program I have on my machine I would probably need a second monitor to allow all of them to show.

jaclaz

Good point - I can probably make it optional during the installer. Then at least it gives the user the choice.

ReplyQuote
Posted : 09/08/2015 7:15 pm
4144414D
(@4144414d)
Junior Member

jaclaz - I've set up the context menu to be optional now. Anything else you think should be changed/added/improved?

Thanks again!

ReplyQuote
Posted : 10/08/2015 11:53 pm
hydrocloricacid
(@hydrocloricacid)
Junior Member

Works great. Will be very useful for the consolidation and mass verification of evidence.

FYI (from e-zero help page)

Note
FTKi CLI does not support the verification of ad1, L01, Lx01, or Ex01
images. As such e-zero is only able to copy these files and cannot
verify them. Please let me know if you are aware of a command line
tool that can verify these formats.

I know you can use ewfverify of the ewflib tools and it will try to verify a L01 , but as L01 files don't contain a hash to verify it just creates a hash of the content.
This could be useful to make sure you have a hash for your L01 files. (log the hash to a file when verifying)

AD1 seems a lot better than L01 being that it stores hashes that can be verified, pity Encase still doesn't support AD1 like most products do as it would make my job a lot easier. 😉

ReplyQuote
Posted : 02/09/2015 7:14 am
Bulldawg
(@bulldawg)
Active Member

I like this, and I plan to use it in my archiving routine once the kinks are worked out.

I'm having an issue with the verify routine. It works fine when the E01 files verify without error, but when I throw it at a set of segmented E01 files that I know have CRC errors and md5 hash mismatches I get the following

C\Users\XXX_Lab\e-zero-master\e-zero python>python e-zero.py verify F\2014-00-00_00-00-00\

Total images 1
Total sources 1
Total size 183GB

2015-09-03 211500 ftkimager.exe --verify "F\2014-00-00_00-00-00\XX-0000-XXXX.E01"
Traceback (most recent call last)
File "e-zero.py", line 443, in <module>
verify(arguments)
File "e-zero.py", line 372, in verify
dispatcher(False,True,[],files)
File "e-zero.py", line 350, in dispatcher
elif sha1_match.group(1) == 'Match'
AttributeError 'NoneType' object has no attribute 'group'

My installation is this
Python 2.7.10 running on Windows Server 2012 R2 x64. I have e-zero.py, docopt.py, and ftkimager CLI all located in the same directory. You can see I'm running it with the line "python e-zero.py verify "

Any ideas?

ReplyQuote
Posted : 04/09/2015 4:57 am
4144414D
(@4144414d)
Junior Member

Bulldawg - Thanks for that. I do love a good bug, thank you for letting me know. I think I know exactly what the problem is. Your image probably doesn't have an embedded md5 or a sha1 (i.e. it only has one) and my logic for dealing with that is wrong!

This is the line that is broken.
if md5_match or sha1_match Which basically means if we have either an valid MD5 or a valid SHA1 then we can continue and look for the matches. Which will give you an error if the MD5 works but the SHA1 doesn't, it looks for the match.group(1) which doesn't exist.

In this part
#Line 345
md5_match = md5_regex.search(result[3])
sha1_match = sha1_regex.search(result[3])
if md5_match or sha1_match
if md5_match.group(1) == 'Match'
verified_md5_only.append(result[2])
elif sha1_match.group(1) == 'Match'
verified_sha1_only.append(result[2])
else
failed_to_verify.append(result[2])

If you need a fix quickly I think chaining those lines to this should work. I haven't tested it yet properly enough, but I'll commit it to GitHub soon.


md5_match = md5_regex.search(result[3])
sha1_match = sha1_regex.search(result[3])
if md5_match
if md5_match.group(1) == 'Match'
verified_md5_only.append(result[2])
elif sha1_match
if sha1_match.group(1) == 'Match'
verified_sha1_only.append(result[2])
else
failed_to_verify.append(result[2])

If you could run ftkimager.exe outside of e-zero and let me know the output that will be very helpful as well, I can test the regex my end a bit more. So this
ftkimager.exe --verify "F\2014-00-00_00-00-00\XX-0000-XXXX.E01"

Otherwise if you go to the previous version of the script before I tried to implement this whole MD5 or SHA1 matching and it should work. https://github.com/4144414D/e-zero/blob/146f7d4bdd24ca989b111c675467db229b78b857/e-zero%20python/e-zero.py

ReplyQuote
Posted : 06/09/2015 4:34 pm
4144414D
(@4144414d)
Junior Member

Bulldawg - This should hopefully fix your issue, my 'temp' fix also didn't work, so please try this one instead. Seems to work with my test data but I'd be really keen to see if it works on your image.

https://github.com/4144414D/e-zero/blob/master/e-zero%20python/e-zero.py#L345-L357

ReplyQuote
Posted : 06/09/2015 7:01 pm
Bulldawg
(@bulldawg)
Active Member

Adam,
I'm following up to say thanks for this. It's been working well in my archiving and periodic verification processes from my Windows examination machines.

Out of curiosity, what would it take to make this python script work on Linux? Maybe even just the verify portion of the script. My NAS runs a Linux variant and has processor cores to spare. In the past I've used a simple batch script to verify using ftkimager, but your solution is so much more elegant than mine. If it's not too hard, I may finally decide to learn enough python to convert it to run on Linux as well.

ReplyQuote
Posted : 14/01/2016 6:31 pm
ccalderwood
(@ccalderwood)
New Member

Thanks for the script. Just saw it this morning, and it will be incredibly useful for our team as we often do consolidation from multiple drives, and re-acquire with best compression.

ReplyQuote
Posted : 19/01/2016 4:24 am
doppiamunnezza
(@doppiamunnezza)
New Member

Hi, i had this error with the exe file

Total images 1
Total sources 1
Total size 6GB

2016-02-18 130724 ftkimager.exe –verify "G\_\11192_14_21_SMCV\Acquisizioni\CTU01\WD-WMAMAA815069\ctu01_dell_11192_14
.E01"
Traceback (most recent call last)
File "C\Python27\lib\site-packages\cx_Freeze\initscripts\Console.py", line 27, in
File "C\Users\Adam\Documents\GitHub\e-zero\e-zero python\e-zero.py", line 443, in
# ch.setFormatter(ch_formatter)
File "C\Users\Adam\Documents\GitHub\e-zero\e-zero python\e-zero.py", line 372, in verify
logger = logging.getLogger('e-zero.verify')
File "C\Users\Adam\Documents\GitHub\e-zero\e-zero python\e-zero.py", line 350, in dispatcher
sha1_match = sha1_regex.search(result[3])
AttributeError 'NoneType' object has no attribute 'group'

So i read your post and tried the python script that gives me this other error

C\Users\Administrator>python c\cl\e-zero\e-zero.py verify G\_\11192_14_21_SMCV\Acquisizioni\CTU01\WD-WMAMAA815069
Traceback (most recent call last)
File "c\cl\e-zero\e-zero.py", line 34, in
from multiprocessing import Process, Lock, active_children, Queue
File "C\Python27\lib\multiprocessing\__init__.py", line 84, in
import _multiprocessing
ImportError DLL load failed %1 is not a valid Win32 application.

The environment is Windows 7 x64 and pythin 2.7.10
Any idea?
Thanks in advance

ReplyQuote
Posted : 18/02/2016 5:24 pm
4144414D
(@4144414d)
Junior Member

Hello guys,

Missed the replies so catching up now.

doppiamunnezza - what version exactly are you running? (e-zero –version) Out of interest what happens when you run ftkimager cli by yourself on the command line. The none type error means that the script isn't getting the correct information back from FTKi and I've not programmed it to deal with that bug - I'll add it to the list.

ftkimager.exe --verify "G\_\11192_14_21_SMCV\Acquisizioni\CTU01\WDWMAMAA815069\ctu01_dell_11192_14.E01"

ccalderwood - let me know if you have any issues.

Bulldawg - To work on Linux I'd need a tool similar to ftkimager that I can use to verify e01s. The obvious answer is ewftools and it just needs me to stop being lazy and add it in.

I'm not checking here often enough (hence the 4 month delay) so email me if you need quicker answers. adam at nucode co uk

Adam

ReplyQuote
Posted : 05/06/2016 4:30 pm
Share: