creating an md5 on a split dd image under linux

even using a loop, I'd want to know the number of files on the tape first.

I wish I had a tape drive to play with here…would make an interesting afternoon.

Hmm OK - I could manage with a solution that required knowing the number of files before I run the operation - mt will give me that, but I would normally have the answer from another source so would not even need mt.

Pity you are not a bit closer I could pop a drive over. But if you could devise a solution using a fixed number of image files on disk, then I would hopefully be able to do the rest of the work and post back here with a tested solution.

I'll be looking at tar this evening - but if memory serves I think this might be a dead end.

hmmm…I'm sure I've done this before using dd and blocksizes. Darned if I can remember.

Anyway, on using dd and piping to md5, if you know the number of files you can try doing a subshell with an incremental loop

(for ((i=0; i<n; i++)); do dd if=/dev/nst0; done) | md5sum

where n=the number of files, and this assumes the dd command will move to the next file as you described.

Basically the parentheses around the entire for loop makes the entire thing execute in a subshell. Since the dd command has no "of=" it will pass the output to stdout, which won't be sent to the pipe until the entire subshell exits (after the loop completes).

give it a shot. I'd like to know if this works.

Barry
www.linuxleo.com

Just given it a quick go, created a text file with the content as below to process an tape with 5 files

(for ((i=0; i<5; i++)); do dd if=/dev/nst0 bs=64k; done) | md5sum

made the file executable and ran it - nowt happens (

Tried it on the command line and the program does not return - and the tape stays idle.

Oops hang on - might have been a problem with a loose scsi lead

tape is whirring

Kewl - it works

Could come in handy on a job I am doing at the mo too, just need to sort out command line parameters to pass the number of tape files and a reference for the tape and pipe it all to a text file with a nice little header for each file and I'm away. Nice task for tomorrow 0

Cheers Barry.

Noice!

Glad it worked. My only concern is scalability (large numbers of files on the tape)…but that's more of a concern with arguments passed with a bash "glob". I'll be curious to see if this works in "production".

Anyway…g'night.

Nearly there - just one issue with the output from my script

echo Sanderson Forensics hashtape script
echo
date
echo
echo Creating hash on contents of tape identified as $1
echo command line - hashtape $1 $2
echo
echo rewinding tape
mt -f /dev/nst0 rewind
echo hashing $2 tape files
(for((i=0;i<$2;i++)); do dd if=/dev/nst0 bs=64k; done) | md5sum
echo
echo complete
date
the code above writes out the correct hash as below, but I would also like to see the output of dd in the log file as a sanity check, i.e. the 1+0 records in, x65536 records copied bit

current output is

Sanderson Forensics hashtape script

complete
Wed May 26 095244 BST 2010
so to summarise the above script works but the output from dd within the script is not redirected to the log

Hi,

It sounds like you ideally want to perform an incremental hash, on a per-part basis, so that you don't have to create/store the concatenated image at all? (noting that the method you have described above is really just getting the OS to concatenate the parts in memory and then perform the md5 on the full concatenated file, albeit stored in [actual/virtual]memory rather than on disk, which I think you wanted to avoid in your original request).

To perform a part-based incremental hash requires access/storage of the internal hash state information between part calculations (it sounds like you may be familiar with doing this on Windows?); In OpenSSL, the MD5 context structure used for this purpose (between MD5_Init,Update and Final functions) is fairly small (around 100bytes iirc); Those three functions could be wrapped with minimal code to create command line versions passing context in/out as a printable string, and then a simple script along the lines of

# Either define number of blocks (N) or use while loop with user query to complete
N = 5
ctx = `md5init`
for((loopi=0;loopi<N;loopi++))
do
# Possible user "any more tape parts?" query here, if while used instead of for
ctx=`dd if=/dev/nst0 bs=64k | md5update $ctx`
done
result=`md5final $ctx`

Of course the way you've already defined is fine provided you have sufficient memory/storage for the full concatenated image; just thought I'd suggest the alternative, since it solves the potential storage size issue, and as the openSSL wrapper required for turning the three required functions to command line form would be so minimal.

Phil.

Thaks Phil

I am just running the previous solution and it is proving to be slow on a larger tape.

I'll give yours a go and see if that is any faster