Join Us!

Lotus Notes Collect...
 
Notifications
Clear all

Lotus Notes Collection  

Page 1 / 2
  RSS
isth
 isth
(@isth)
Member

Hi All,

We have a potential collection job coming up which involves the collection of multiple (~12) custodian hard drives along with each users' mail. The client has sparse details as of yet, but one of the things they mentioned is that their primary mail system is Lotus Notes. I have experience collecting from Exchange, either using Exmerge or copying the entire EDB, but I have never worked directly with Lotus Notes - besides using the trial version to view NSFs. Would someone be able to point me in the direction of methodology for collecting notes email? Is it just a dump of NSFs in a predetermined folder or something that requires an exmerge-esc utility?

Any insight would be appreciated.

Thanks!

Quote
Posted : 04/11/2010 1:05 am
jonstewart
(@jonstewart)
Junior Member

The nice thing about Lotus Notes is that it's NSFs on the client and it's NSFs on the server. Contrast that with Exchange/Outlook, where it's PSTs on the client and EDBs on the server. You don't have to worry about using an exmerge-like utility.

That's the only thing nice about Lotus Notes. It's otherwise hellacious to deal with. Lotus Notes itself is buggy, the file format is complex, the types of data it stores is very flexible–it's not just email–which means you have to figure out whether an organization is using custom forms and how best to produce that data, and tool support for Notes is generally not as good as for Outlook.

Oh, and encryption? Yeah, there's encryption. Notes has "ID" files, and you need those to decrypt the NSFs. I cannot remember at the moment whether there's a master escrow ID file (i.e., an administrator ID file). You're dead in the water if you don't collect these.

I'm not a Notes expert, so I don't want to comment beyond my expertise, but… do your homework and run through some trials before going onsite. It is not a forgiving, learn-as-you-go technology.

Jon

ReplyQuote
Posted : 04/11/2010 1:47 am
roncufley
(@roncufley)
Active Member

Jon is correct in that if you have the .nsf files you have all the data, the views, the forms……. everything. You can always be sure that you can carry out the extraction and analysis later. There should be at least one .nsf for each custodian which may be on the server or the workstation or both and may be replicated onto other servers. His point about the id files may or may not matter, if the .nsf files are encrypted then you need both the id files and the users' passwords; depending upon how the system is set up these may be available from the admins (or, indeed, they may not).

There is a further possible wrinkle in that it is permissible to have a single email repository like Exchange Server but this is very rarely used, if it does exist it will be encrypted as a virtual certainty.

Any questions - just ask.

Good luck
Ron

PS By the way, Lotus Notes is a wonderful system, don't listen to the detractors.

ReplyQuote
Posted : 04/11/2010 6:08 pm
isth
 isth
(@isth)
Member

Thanks so much for the replies, gents! We're scheduled to have a call with the client to obtain more details on the exact configuration in the near future. It does seem like this will be a fairly straight forward task though.

Thanks again for the feedback!

ReplyQuote
Posted : 04/11/2010 7:13 pm
gblack
(@gblack)
Junior Member

if you have the .nsf files you have all the data, the views, the forms……. everything

That's not 100% correct. A copy of the NSF can be made by an administrator in which you don't get design elements, only documents. Make sure this doesn't happen. Often the Notes admin can make a physical copy of the NSF directly from the server and get everything. NSFs from a Notes server are typically not encrypted, and once you have a local copy permissions are ignored. If you get one of these, you MUST double check and make sure it opens after you receive the copy. I have seen NSFs get corrupted from physical copies off of Notes servers, especially when the files are active mailboxes in use.

If you're collecting from the desktop or home/group shares, this is where you have to worry about the ID files and passwords.

PS By the way, Lotus Notes is a wonderful system, don't listen to the detractors.

Bah, humbug! The Notes dev API is as screwed up as they come. Someone put very little forethought into the design of it and developers pay the price. Notes is the devil! )

ReplyQuote
Posted : 04/11/2010 9:35 pm
roncufley
(@roncufley)
Active Member

if you have the .nsf files you have all the data, the views, the forms……. everything

That's not 100% correct. A copy of the NSF can be made by an administrator in which you don't get design elements, only documents.

I think that one can say that it is axiomatic that if a copy is made that intentionally leaves things out then those things that are left out will not be in the copy, I didn't realise that I had to specify that. (We are talking forensics here aren't we?)

<…..> and once you have a local copy permissions are ignored.

Perhaps or should I say often?

PS By the way, Lotus Notes is a wonderful system, don't listen to the detractors.

Bah, humbug! The Notes dev API is as screwed up as they come. Someone put very little forethought into the design of it and developers pay the price. Notes is the devil! )

Workmen and tools?

ReplyQuote
Posted : 04/11/2010 10:31 pm
gblack
(@gblack)
Junior Member

I think that one can say that it is axiomatic that if a copy is made that intentionally leaves things out then those things that are left out will not be in the copy, I didn't realise that I had to specify that. (We are talking forensics here aren't we?)

I don't think we are, actually. This sounds like an eDiscovery collection to me. There's more than one way to get a "copy" of a Notes mailbox. Since the OP is obviously unfamiliar, I'd rather give more information than less.

ReplyQuote
Posted : 04/11/2010 11:09 pm
isth
 isth
(@isth)
Member

To add, the collection is for the purposes of eDiscovery, yes. We typically make dd images of custodian desktop drives (which would encompass any NSFs that may exist on the users machine) AND we collect all mail from the mail servers for the applicable users. This approach is typically highly duplicative but it ensures we have the most complete dataset, since e-mail is often of the most interest and it's really easy for someone to tamper with email on their own machine.

Appreciate the additional info.

ReplyQuote
Posted : 05/11/2010 12:20 am
roncufley
(@roncufley)
Active Member

(We are talking forensics here aren't we?)

I don't think we are, actually. This sounds like an eDiscovery collection to me.

This raises an interesting question, is this not a distinction without a difference? Forensics is producing evidence to place before the Court and eDiscovery is producing evidence to place before the Court. I appreciate the the tools and techniques might not be the same but do we not have to exercise the same care and attention to detail? Can we really afford to say, "Oh those records are probably missing because I might not have copied the whole file," just because it is "only" eDiscovery? I think not, what do others think?

ReplyQuote
Posted : 05/11/2010 2:39 am
mbarnes86
(@mbarnes86)
New Member

Hi

About a Year ago my employer went over to Outlook & Exchange Server from Lotus Notes and Lotus Domino Server so my recollection maybe a bit hazy
There were several thousand users with many Notes servers

The domino servers (Win 2003) had 1 nsf file per user,.
The users U (users private area on login server) had some data and identity files
and the Local PC had some files and data copied from the U when the user first used Notes on a PC these were updated while the Notes Client was in use.
The server notes files were compacted by a process which ran each night
to remove deleted messages

smaller systems may be less complex

regards
Mike Barnes

ReplyQuote
Posted : 05/11/2010 3:19 am
gblack
(@gblack)
Junior Member

This raises an interesting question, is this not a distinction without a difference? Forensics is producing evidence to place before the Court and eDiscovery is producing evidence to place before the Court.

I'm not all that familiar with UK law concerning eDiscovery, so I really can't speak to any difference on burden you might experience. In the US, courts routinely weigh cost, burden, and reasonableness in what will be collected and reviewed for any given case, forensic or eDiscovery.

I appreciate the the tools and techniques might not be the same but do we not have to exercise the same care and attention to detail?

We absolutely must exercise attention to detail - it's just as important in eDiscovery as it is in forensics, just in different areas of specificity.

Can we really afford to say, "Oh those records are probably missing because I might not have copied the whole file," just because it is "only" eDiscovery? I think not, what do others think?

I think you've completely misinterpreted my statements. My original post was intended to give the OP more information on how to avoid missing data that will be useful later. You can obtain a copy of all messages in an NSF and miss design elements. This causes problems for eDiscovery Review vendors if there are documents present with custom forms. That being said, forensics and eDiscovery are necessarily different in their requirements and connotations.

ReplyQuote
Posted : 05/11/2010 7:39 pm
Buster
(@buster)
Junior Member

isth

AND we collect all mail from the mail servers for the applicable users

That's a good plan with Lotus/Domino systems otherwise you may just end up with the headers and no message bodies due to Domino's "Single Copy Object Store" (SCOS) which allows servers to store a single copy of messages received by multiple recipients in a special central database, or object store.

There are some useful Domino notes either side of this link.

Stu

ReplyQuote
Posted : 08/11/2010 1:16 pm
isth
 isth
(@isth)
Member

There are some useful Domino notes either side of this link.

Stu

Thanks, Stu this is good to know! Quick question on this… if SCOS is enabled the notes mention that the unique copies are stored in a separate "shared mail database." Is this separate database just another series of NSF files or is it in a different file format that would require special handling to collect?

An FYI to anyone who comes across this thread and is looking for more info on Notes, here's a snip-bit on that same site regarding Mail Journaling Link

ReplyQuote
Posted : 08/11/2010 8:36 pm
Buster
(@buster)
Junior Member

isth

Is this separate database just another series of NSF files or is it in a different file format

From memory, they will be in NSF format although around v7, IBM added the ability to use DB2 as an alternative (NSFDB2) although I believe this is now deprecated and no longer supported.

I also seem to remember a set of files called "MAIL.BOX" which were relevant (on the server side) but I can' t recall why off the top of my head. I'll try and dig out some old notes (if I still have them) and see if they will be of any use.

It is also worth knowing that any attachments will be compressed using LZ1.

Stu

ReplyQuote
Posted : 08/11/2010 9:47 pm
joachimm
(@joachimm)
Active Member

I cannot recall IBM terminating NSFDB2 support
http//www-01.ibm.com/support/docview.wss?uid=swg21384421

According to this link they are supporting it until 2017.

Note that I did some work on the NSF file format.
http//sourceforge.net/projects/libnsfdb/

This work is focussed on the NSF file format not on the Notus/Domino product.
It is far from complete because NSF is a complex and versatile file format.
However the references could provide you some background information.

Buster, note that in light of this project, I'm also interested in your old notes; BTW was this an intended pun?

ReplyQuote
Posted : 08/11/2010 10:57 pm
Page 1 / 2
Share: