Best DAT tape softw...
 
Notifications
Clear all

Best DAT tape software?

16 Posts
5 Users
0 Likes
1,149 Views
Adam10541
(@adam10541)
Posts: 550
Honorable Member
Topic starter
 

I'm looking to expand my lab capabilities and was wondering about peoples experiences with DAT tapes.

I have previously used different types of software which relied on restoring the tape first but I had major problems finding compatible software for some tapes.

I've been made aware of Octane for Backup by Index Engines which is a pre-configured 2RU rack server with the software already set to go. You just provide the DAT drives and then you arrange your licensing with them. This initial set up costs seem to be around the 20k mark and then how much you pay depends on the licensing model you select.

From what I understand this software indexes most tape formats which allows you to then search and extract only what is relevant, instead of restoring the entire tape first. The attraction of this was that is supports just about every format out there and I wouldn't have to worry about different software for different tapes, however the set up cost is prohibitive, and the licensing models are not ideal.

Is anyone else aware of any software which can handle multiple DAT tape formats for extraction?

Thanks for any help )

 
Posted : 09/07/2012 8:58 am
jaclaz
(@jaclaz)
Posts: 5133
Illustrious Member
 

I've been made aware of Octane for Backup by Index Engines which is a pre-configured 2RU rack server with the software already set to go. You just provide the DAT drives and then you arrange your licensing with them. This initial set up costs seem to be around the 20k mark and then how much you pay depends on the licensing model you select.

From what I understand this software indexes most tape formats which allows you to then search and extract only what is relevant, instead of restoring the entire tape first. The attraction of this was that is supports just about every format out there and I wouldn't have to worry about different software for different tapes, however the set up cost is prohibitive, and the licensing models are not ideal.

But are you going to use the (rather expensive) thingy for "Forensics" or for "Data Recovery"?

If the first, you will have to face the issue of validating the whole stuff, if the second you risk providing the client with "sub-standard" services.

I mean, possibly that Octane stuff is the best thing since the invention of ice-cream, but should it miss even a single file due to it's (I presume proprietary and undisclosed) internal algorithms, you may face some issues.

In my personal (and very limited) experience, whenever I actually needed to get some data from a tape, it has been a nightmare, not because of the software, but because of the hardware, I would estimate (but I may have been particularly UNlucky) that 75% to 90% of tapes are "defective" or "corrupted" or *whatever*, and a lot of manual work was needed, besides finding the "right software" and trying on different tape readers and what not.

Of course, if you are going to undertake a case like the ones described in their whitepaper

  • Project one 2005 480 tapes US$ 8,000,000 18 months time
  • Project two 2009 800 tapes US$ 1,900,000 9 months time

the 20k bucks you talked about are to be considered like "pocket money" and will be less than what you spent over the period for pizza and coke delivery to the office…..

jaclaz

 
Posted : 09/07/2012 3:03 pm
(@mscotgrove)
Posts: 938
Prominent Member
 

With tapes you have two major problems

Drive compatibility and software compatibility

DAT drives started with DDS-1 and a few years ago were DDS-5. Most drives will typically read a few generations earlier.

DLT drives started years ago with TK50 format, before eventually ending up as SDLT.

Most tapes tend to deteriate over a few years, and/or suffer from stiction.

The software side is even more varied. Over approx 30 years there have been many hundreds of formats and variations. Again, typically software will read one or two previous versions, so even a current licenced version of software will not be fully compatible with 'version one'.

Speed, old tape drives are now very slow. LTOs will stream at very high speeds.

"From what I understand this software indexes most tape formats which allows you to then search and extract only what is relevant, instead of restoring the entire tape first. The attraction of this was that is supports just about every format out there and I wouldn't have to worry about different software for different tapes."

The only way to index tapes - I assume on a word basis is to read the data. The above statement is therefore slightly misleading as it still means the complete tape has to be read, just that files are not saved on your computer. Thus to index and extract the files, tapes will have to be processed twice.

Octane claim to read 'just above every tape format'. For an 8 year old company you need to check if you will processing tapes from the previous 20 plus years of tape history.

Modern tape structures are getting very complex with mutiple data streams multiplexed onto a single tape, or multiple tapes, and sometimes just incremental segements of data.

Why do you want to concentrate on DATs - most commercial use of tape is now LTO and possibly SDLT.

 
Posted : 09/07/2012 3:32 pm
Adam10541
(@adam10541)
Posts: 550
Honorable Member
Topic starter
 

I'll try to clarify somewhat, the reason for this whole exercise is that a client we are trying to win asked me about my DAT tape capability and experience a little while back an hinted that something was coming up where they would need DAT tapes dealt with.

I've had limited exposure to backup tapes, I had to process around 100 a few years back when I was still with the Police. I was lucky enough to have contacts at a local university where we were studying and they had a good supply of old hardware, so I got the right drive I needed to read them. I managed to find some software, I can't recall the vendor but it had the word Power in it. In any case it read the tapes and I was able to restore data of quite a few of the tapes, which I could then view on Windows or with a tool of choice.

The only reason I mentioned Index Engine and Octane is that a former colleague of mine who is also out in the corporate world is using this very tool and speaks quite highly of it. I've seen the set up in his office but not seen it working. His company does quite a lot of work with DAT tapes which is why I was curious about it.

As for why DAT tapes, it would appear that DAT tapes were the most commonly used in Perth or at least my contacts in the IT consulting industry tell me that is was they most commonly see so I thought I'd start with that as odds are better they will be used. I can purchase more on an 'as needs' basis if I win work where I require a different format.

Given that my lab is quite small and my staff limited I don't imagine we would win or even quote on jobs the size of those mentioned in the whitepapers, my clients are more likely to have small amounts of tapes.

I've had a quote from Index Engine and it's only $12.5k US for the hardware to purchase, you supply your own tape drives, then the software can index for free from there, but if you want to search and extract you pay per drive. Not sure exactly how they make this all work but the processing fees are quite small per tape, a few hundred dollars.

With regards to the indexing my understanding is the tape is read once entirely and contents indexed, the software creates an index file with all the file details known. You can then search this index file based on keywords, dates etc and one you select the files you are interested in the software then goes to the relevant section on the tape and extracts only that data. So in effect the tape is read completely only once. And allegedly as they are only indexing not extracting the data this process is much faster than restoring the tape.

Anyway, that is all claims I guess I need to see it in action to really assess it's effectiveness before I lay out that kind of money. It was very attractive though because if it works as claimed then it virtually negates the software issues which you have rightly pointed out one of the biggest hurdles to overcome when accessing tapes.

I think my approach for now will be to sources a few DAT tape drives (4mm and 8mm) which will hopefully cover the majority of what I'll encounter in sleepy Perth, grab some blank tapes and dump data on them and start testing software until I find something that works.

 
Posted : 10/07/2012 8:23 am
jaclaz
(@jaclaz)
Posts: 5133
Illustrious Member
 

I've had limited exposure to backup tapes, I had to process around 100 a few years back when I was still with the Police. I was lucky enough to have contacts at a local university where we were studying and they had a good supply of old hardware, so I got the right drive I needed to read them. I managed to find some software, I can't recall the vendor but it had the word Power in it. In any case it read the tapes and I was able to restore data of quite a few of the tapes, which I could then view on Windows or with a tool of choice.

Out of curiosity, did the 100 tapes job had a "cost" of US$ 8,000,000/480*100=1,666,667 or US$ 1,900,000/800*100=237,500 or slightly less?

I think my approach for now will be to sources a few DAT tape drives (4mm and 8mm) which will hopefully cover the majority of what I'll encounter in sleepy Perth, grab some blank tapes and dump data on them and start testing software until I find something that works.

With all due respect ) there is a flaw in this approach, Murphy's Law is there exactly for these cases, if you test 19 tape formats and you are happy with the result, your next job will be with a 20th, unknown format AND needing a particular drive that you don't have. (

jaclaz

 
Posted : 10/07/2012 1:51 pm
Adam10541
(@adam10541)
Posts: 550
Honorable Member
Topic starter
 

Hi Jaclaz, I'm not sure how things work in Italy but in Western Australia all the forensic work is conducted by sworn Police officers (which I was up until about a year ago) and as such we didn't charge for our time. So the job with 100 odd DAT tapes cost the client (ie the Police) exactly $0 above my normal hourly wage.

And while I respect your forensic knowledge I would have to take a different path in regards to my business development.

Yes I have other options, I could do nothing and not offer this service to my clients.

I could not buy the hardware and software and not test it in advance, wait until my client has a need and find out exactly what type of tapes they need accessed. This would mean that the whole process would be delayed by weeks as I try to source hardware and software, then delayed again as I test the software and hardware, and would also necessitate lying to my client and saying I had a capability that I did not. Very little of this hardware is available for sale in Australia at a reasonable price, mostly it would be coming from the US or UK.

These type of delays and lack of planning or testing would result in me delivering bad service to my clients which would result in no repeat business and a unprofessional reputation for me.

Or I could do the smart thing, buy some entry level hardware, find some software and start testing so that I can honestly tell my client I have backup tape capabilities and experience. Then if the tape they have is a different format I will have had some experience in sourcing hardware / software and will be in a position to quote accurate time frames based on experience……but that's just me )

 
Posted : 11/07/2012 5:38 am
jhup
 jhup
(@jhup)
Posts: 1442
Noble Member
 

Or, do a research (as in ask) on the most popular tape, drive and software combinations of your target market and get coverage of the top 80%… -)

 
Posted : 11/07/2012 9:24 am
Adam10541
(@adam10541)
Posts: 550
Honorable Member
Topic starter
 

Has anyone experience using MMPC or NovaXchange?

And please, while I respect everyone's opinion and have no problem when they differ to mine, the purpose of this thread was to find out about software that people may have had hands on experience with D

 
Posted : 11/07/2012 1:24 pm
jaclaz
(@jaclaz)
Posts: 5133
Illustrious Member
 

Hi Jaclaz, I'm not sure how things work in Italy but in Western Australia all the forensic work is conducted by sworn Police officers (which I was up until about a year ago) and as such we didn't charge for our time. So the job with 100 odd DAT tapes cost the client (ie the Police) exactly $0 above my normal hourly wage.

And while I respect your forensic knowledge I would have to take a different path in regards to my business development.

I guess you misunderstood me ( , how you manage your business is of course your exclusive area of interest and competence ) , the veiled sarcastic comment on the "Project 1" ad "Project 2" the good guys at Octane present as an example, was about the fact that in 2005 getting data from 480 tapes in 18 months, would have meant, assuming that each tape was at the most 36 Gb, that
480*36=17,280 Gb
But the whitepaper talks of extracting "only" 4 Tb of data, later reduced to 1.5 Tb.
This happened ove a period that is once defined as "almost two years" and elsewhere as "18 months".
Let's say it is 20 months, 20*22*8= 3,520 worked hours
17,280/3,520=5.45 Gb/working hour
or
4,000/3,520=1.14 Gb/working hour
or
1,500/3,520=0.43 GB/working hour

and
8,000,000/(18*22*8)=2,525 US$/working hour (to be divided by the number of people involved), but anyway
2,525/5,45=500 US$ per Gigabyte of analyzed data

My question was - indirectly - at the time you did the work on the tape - how much time did it took?
Which was the cost per tape or per Gb? (the cost exists also if you were a Police Officer, the cost of your wages and those of your colleagues on the case).
I would guess that the cost was far less than US$ 500/Gb of analyzed data. 😯

Yes I have other options, I could do nothing and not offer this service to my clients.

I could not buy the hardware and software and not test it in advance, wait until my client has a need and find out exactly what type of tapes they need accessed. This would mean that the whole process would be delayed by weeks as I try to source hardware and software, then delayed again as I test the software and hardware, and would also necessitate lying to my client and saying I had a capability that I did not. Very little of this hardware is available for sale in Australia at a reasonable price, mostly it would be coming from the US or UK.

These type of delays and lack of planning or testing would result in me delivering bad service to my clients which would result in no repeat business and a unprofessional reputation for me.

Or I could do the smart thing, buy some entry level hardware, find some software and start testing so that I can honestly tell my client I have backup tape capabilities and experience. Then if the tape they have is a different format I will have had some experience in sourcing hardware / software and will be in a position to quote accurate time frames based on experience……but that's just me )

I did not want in any way to somehow "put you down" or something like that, nor hint that you should not try your best to provide an additional service to your customers, actually I like and appreciate your plan of getting familiar with the thingies little by little and - from a strictly forensics point of view I suspect that the Octane thingy carries with it - like any "triage-like" tool - the risk of providing incomplete results.

The note was only because of my (as said very little) experience with tapes, which, believe me, has been a nightmare (and BTW had far less "relevance" as it was about a "mere" data recovery).
Call me pessimist as much as you want, but I believe not that a "one-size-fits-all" thingy exixts and not even that the nice Octane thingy is the "magic wand" they describe.

So, IMHO your approach is the "right one" to take ) , but - as I see it - it is like genius

His genius he was quite content in one brief sentence to define; Of inspiration one percent, of perspiration, ninety nine.

Should you by any chance happen to get a "big" job to do on tapes you can always ask your ex-colleague to help you or outsorce some of the work to his company, right?

jaclaz

 
Posted : 11/07/2012 3:41 pm
Adam10541
(@adam10541)
Posts: 550
Honorable Member
Topic starter
 

ohhhh okay, sorry I misunderstood oops I am a delicate little flower sometimes lol

I don't think I'd outsource to my friend as he's a direct competitor P

As an aside I've now had a quote for MMPC and for their software solution they are asking 10,500 GBP with yearly renewal being 15% of the relevant price at the time.

Index Engines Octane solution includes some very decent hardware at $12,500 US…..or NovaXchange is around $1200 US, there doesn't seem to be anything in between unless I look at TapeCat which is freeware but I think that only indexes the data on the tape not extract or restore.

My choices are not looking great really..

 
Posted : 12/07/2012 5:16 am
Page 1 / 2
Share: