I need help making sense of ntuser.dat file internet history

Just a bit of background about myself. I'm not a professional with this stuff by any means, I was hoping to go to school/get some training with forensics in year or two. So please guys, I know this is probably frusterating trying to explain this to me (repeatedly), but I honestly am extreamly appreciative of all the help from everyone. If you could bare with me for a bit longer, I would be very greatful! 😉

Well everyone had to start somewhere, no problem ) , but you will also need some patience.
Let us (for the moment) completely FORGET about your specific question about Internet artifacts in the Registry, it is more important that you understand the basic concepts underlying,

Anyway, I'm going to try and see if I understand this. Let me know if I'm on the right track.
1.The ntuser.dat isn't just a repeat of everything I can see from Regedit.

2. Reading the ntuser.dat file raw (which is potentially very bad) probably will yield more data such as fragmented data that will be difficult to associate with a specific application, but if I back it up as a "self-standing file"(which just means it isn't live and recording data?) and then mount it the way that link describes , it should help me figure out where it came from.

Yes and no (mostly no ( )
1. Viceversa parts of the Registry are "a repeat" of some of what you can find in NTUSER.DAT (the NTUSER.DAT or other backing file contains MORE information than what you can see in Regedit BUT in Regedit you can also see data in structures that are not in the backing file)
2. It is not "bad" to read the backing file as RAW, it is additional, as there may be data in it which is not shown in non-RAW viewers (such as Regedit), what is important is to access the backing file(s) OFFLINE.

Let's see if I can explain.

The Registry DOES NOT exist if not during the time the OS is booted, i.e. it is in itself "volatile".
When Windows NT boots the Registry is created on the fly by combining a few "backing" files (actually "existing") as per the MS article I posted a link to earlier.
As jhup cleared, changes to the Registry are committed to the backing files (but - as a side example - in a PE the files remain untouched, i.e. the Registry is "entirely volatile") and since in a "normal" Windows these files need to be written to by the OS, they are thus "in use" or "locked" to prevent possible alterations when ONLINE.

There is nothing "bad" in reading one of the backing files, BUT since the contents of the file is continuously updated by the OS this creates three potential issues
1) data may have been changed BEFORE you can look at it (by the mere acting of booting the OS)
2) data may change WHILE you are looking at it (due to a concurrent operation of the OS)
3) data may change AFTER you looked at it (due to the *whatever* operations the OS carries on when shutting down)
On the other hand one of the main ideas about digital forensics is to "take a snapshot" of a given status at a given time and do so in such a way that a later taken snapshot will povide exactly the same data.

So the main thing is that you normally look at a file while "offline" and NOT on a booted system.

The way the Registry Editor works is another issue, let's for the moment imagine that it is only a viewer, it shows you a "view" of the Registry, organized according to a given graphical convention and showing only a part of the data actually available in the backing file (as an example Regedit won't show you the last modification date of a record).

And the Registry itself is in itself an "abstraction" of the contents of the actual files, I will provide you with a common example about the way parts of the Registry are "created on the fly".

If you open Regedit you will see in HKEY_LOCAL_MACHINE\SYSTEM
ControlSet001
ControlSet002
Controlsetxyz <- it depends on the specific machine, normally only 001 and 002 exist, but machines with 25 or more ControlSets have been observed
CurrentControlSet

You can click on them and you cannot find/observe any difference (in the behaviour of the tool or in the contents you observe) when exploring ControlSet001 vs. CurrentControlSet.

If you check the key
HKEY_LOCAL_MACHINE\SYSTEM\Select
you will (very likely) find in it a value of "Current" set to REGDWORD 0x00000001 (1).

What happens is that the whole key CurrentControlSet does NOT actually exist on the backing file and the Registry (when the OS is booted) is creating an "alias" of ControlSet001 (actually ControlSet<value of HKEY_LOCAL_MACHINE\SYSTEM\Select>) and when you access CurrentControlSet you are in reality accessing ControlSet001 "by another name".

Finally there is (yet another) possible issue, the Registry can be seen as said either as a "database" or as a "filesystem", and behaves like them in case of deleting an item, the item is not really deleted, it is simply removed from the database index (or from the filesystem file allocation table), so the contents are still there BUT it is not listed in Regedit (or in any other tool rendering the object according to specifications, to them the item is "deleted" or "invisible" or "hidden").

Imagine that in a library the librarian index card for a given book is lost or destroyed, the book cannot be found, it seems like it never was there, still it is in the library where it has always been and can be found by looking for it on the shelves.

So, for a valid forensic search it is VITAL to examine the "backing files" (and not the Registry) when OFFLINE and for a deeper research it is strongly suggested to examine additionally the backing files RAW data.

Hope the above is clear, if you have question/doubts, feel free to ask.

jaclaz

Just a bit of background about myself. I'm not a professional with this stuff by any means, I was hoping to go to school/get some training with forensics in year or two. So please guys, I know this is probably frusterating trying to explain this to me (repeatedly), but I honestly am extreamly appreciative of all the help from everyone. If you could bare with me for a bit longer, I would be very greatful! 😉

Well everyone had to start somewhere, no problem ) , but you will also need some patience.
Let us (for the moment) completely FORGET about your specific question about Internet artifacts in the Registry, it is more important that you understand the basic concepts underlying,

Anyway, I'm going to try and see if I understand this. Let me know if I'm on the right track.
1.The ntuser.dat isn't just a repeat of everything I can see from Regedit.

2. Reading the ntuser.dat file raw (which is potentially very bad) probably will yield more data such as fragmented data that will be difficult to associate with a specific application, but if I back it up as a "self-standing file"(which just means it isn't live and recording data?) and then mount it the way that link describes , it should help me figure out where it came from.

Yes and no (mostly no ( )
1. Viceversa parts of the Registry are "a repeat" of some of what you can find in NTUSER.DAT (the NTUSER.DAT or other backing file contains MORE information than what you can see in Regedit BUT in Regedit you can also see data in structures that are not in the backing file)
2. It is not "bad" to read the backing file as RAW, it is additional, as there may be data in it which is not shown in non-RAW viewers (such as Regedit), what is important is to access the backing file(s) OFFLINE.

Let's see if I can explain.

The Registry DOES NOT exist if not during the time the OS is booted, i.e. it is in itself "volatile".
When Windows NT boots the Registry is created on the fly by combining a few "backing" files (actually "existing") as per the MS article I posted a link to earlier.
As jhup cleared, changes to the Registry are committed to the backing files (but - as a side example - in a PE the files remain untouched, i.e. the Registry is "entirely volatile") and since in a "normal" Windows these files need to be written to by the OS, they are thus "in use" or "locked" to prevent possible alterations when ONLINE.

There is nothing "bad" in reading one of the backing files, BUT since the contents of the file is continuously updated by the OS this creates three potential issues
1) data may have been changed BEFORE you can look at it (by the mere acting of booting the OS)
2) data may change WHILE you are looking at it (due to a concurrent operation of the OS)
3) data may change AFTER you looked at it (due to the *whatever* operations the OS carries on when shutting down)
On the other hand one of the main ideas about digital forensics is to "take a snapshot" of a given status at a given time and do so in such a way that a later taken snapshot will povide exactly the same data.

So the main thing is that you normally look at a file while "offline" and NOT on a booted system.

The way the Registry Editor works is another issue, let's for the moment imagine that it is only a viewer, it shows you a "view" of the Registry, organized according to a given graphical convention and showing only a part of the data actually available in the backing file (as an example Regedit won't show you the last modification date of a record).

And the Registry itself is in itself an "abstraction" of the contents of the actual files, I will provide you with a common example about the way parts of the Registry are "created on the fly".

If you open Regedit you will see in HKEY_LOCAL_MACHINE\SYSTEM
ControlSet001
ControlSet002
Controlsetxyz &lt;- it depends on the specific machine, normally only 001 and 002 exist, but machines with 25 or more ControlSets have been observed
CurrentControlSet

You can click on them and you cannot find/observe any difference (in the behaviour of the tool or in the contents you observe) when exploring ControlSet001 vs. CurrentControlSet.

If you check the key
HKEY_LOCAL_MACHINE\SYSTEM\Select
you will (very likely) find in it a value of "Current" set to REGDWORD 0x00000001 (1).

What happens is that the whole key CurrentControlSet does NOT actually exist on the backing file and the Registry (when the OS is booted) is creating an "alias" of ControlSet001 (actually ControlSet&lt;value of HKEY_LOCAL_MACHINE\SYSTEM\Select&gt;) and when you access CurrentControlSet you are in reality accessing ControlSet001 "by another name".

Finally there is (yet another) possible issue, the Registry can be seen as said either as a "database" or as a "filesystem", and behaves like them in case of deleting an item, the item is not really deleted, it is simply removed from the database index (or from the filesystem file allocation table), so the contents are still there BUT it is not listed in Regedit (or in any other tool rendering the object according to specifications, to them the item is "deleted" or "invisible" or "hidden").

Imagine that in a library the librarian index card for a given book is lost or destroyed, the book cannot be found, it seems like it never was there, still it is in the library where it has always been and can be found by looking for it on the shelves.

So, for a valid forensic search it is VITAL to examine the "backing files" (and not the Registry) when OFFLINE and for a deeper research it is strongly suggested to examine additionally the backing files RAW data.

Hope the above is clear, if you have question/doubts, feel free to ask.

jaclaz

I think I'm getting it..

1. So basically, the registry just pulls together data from different backing files. The current user hive (Which is specific for each user) is what displays some of the info from the ntuser.dat and ntuser.dat.log files upon starting up the OS.

So my first question about the other backing files.. Since the Hkey Current User hive is the hive that is user specific and displays info from the ntuser.dat files, does that mean other backing files for the other hives wouldn't contain user specific things like internet history/searches etc.? That would be my understanding of it at the moment.

2. Things can be "deleted" from registry but aren't actually deleted, they are still in the backing files, just now not being displayed in regedit? This would explain "typed URLS" in the ntuser.dat file that are not in regedit.

3. The way I've been viewing the raw file is to log off the account I want to examine the ntuser.dat, go on a different admin account and view it on that. I figured it was safe enough. But it would probably be better to just mae a backup and view it somewhere else entirely.

kurt2121

I good method to answer your own questions is to

1) Install Windows fresh on a computer

2) Use a tool such as FTK Imager to create a forensic image of the freshly installed Windows computer. The goal is to create a forensic image of a computer with no user activity whatsoever and thus a pristine registry.

3) Perform specific tasks on the Windows computer such as installing Firefox and Chrome and then visiting a few websites.

4) Create a forensic image of the Windows computer that now contains evidence of known specific activities.

5) Use a program such as OSForensics to analyze and compare the original forensic image and the forensic image of the current-state computer. OSForensics has a nice step-by-step tutorial on how to do exactly this http//www.osforensics.com/faqs-and-tutorials/identifying_uninstalled_software.html

Basically, the idea is to perform a "controlled" experiment to identify exactly what traces, in the registry for example, specific user activities such as browsing to a website using Firefox leave.

BTW full credit goes to the good folks at Passmark, not me, for suggesting and publishing this scientific testing methodology.

Okay, definitely a good idea! To be honest with you, I feel like I wouldn't be able to do get far with it considering my limited knowledge.

Would it be absolutely necessary to use all those programs? Could I just make searches on firefox and chrome and then search the raw files and see if they are there?

@kurt2121
#1. Yes ) and yes, BUT there is no "law" stating that only the "current user" is to be used by programs, while a well written, logical program should write to the profile of the current logged in user, it may well decide that something is better written to other areas of the Registry, the only thing that you can be pretty much sure about (excluding malware or however specifically written software) is that the ntuser.dat that is mounted under HKCU is the only profile mounted, so that ntuser.dat of other users won't be affected (as they are not auto-mounted in the Registry).
#2.Yes and no, deleted (de-indexed) or "hidden" items in a hive are not visible not only in Regedit but also in any other tool that parses the file contents according to specifications/using the Windows API, they are visible using dedicated tools or carving the RAW file, I posted earlier a link to a regidtry viewer that can show "null name" keys, now you might find of interest yaru (Yet Another Registry Utility)
https://tzworks.net/prototype_page.php?proto_id=3
More tools/parsers here
http//www.forensicswiki.org/wiki/Windows_Registry
#3.If you have not a "preservation" need (the file will change the next time you boot as the connected user), viewing another user ntuser.dat is fine, as the file is anyway "offline", but sure the "normal" way is to operate on a copy/backup of the file.

jaclaz

@kurt2121
#1. Yes ) and yes, BUT there is no "law" stating that only the "current user" is to be used by programs, while a well written, logical program should write to the profile of the current logged in user, it may well decide that something is better written to other areas of the Registry, the only thing that you can be pretty much sure about (excluding malware or however specifically written software) is that the ntuser.dat that is mounted under HKCU is the only profile mounted, so that ntuser.dat of other users won't be affected (as they are not auto-mounted in the Registry).
#2.Yes and no, deleted (de-indexed) or "hidden" items in a hive are not visible not only in Regedit but also in any other tool that parses the file contents according to specifications/using the Windows API, they are visible using dedicated tools or carving the RAW file, I posted earlier a link to a regidtry viewer that can show "null name" keys, now you might find of interest yaru (Yet Another Registry Utility)
https://tzworks.net/prototype_page.php?proto_id=3
More tools/parsers here
http//www.forensicswiki.org/wiki/Windows_Registry
#3.If you have not a "preservation" need (the file will change the next time you boot as the connected user), viewing another user ntuser.dat is fine, as the file is anyway "offline", but sure the "normal" way is to operate on a copy/backup of the file.

jaclaz

Thanks jaclaz!

I just saved the ntuser.dat files to a USB drive, just incase. I'm going to try these things out (eventually) once I feel a bit more comfortable with what I'm exactly doing.

So, it's not certain that personal information couldn't be written into the other backing files, depending on what program it is.

jaclaz, the help I've received from you (and everybody here for that matter) has been incredible. Honestly, I didn't think a message board could be as helpful as this. You guys are showing me HOW to do things, being patient with a new guy etc. I feel like I learned way more than before I signed up.

Jumping back to the original question, jaclaz, I would LOVE to hear your own personal thoughts(as well as anybody else reading this) as to whether or not firefox/chrome history/searches ect. would be found in any of these backing files including ntuser.dat files. I asked on other message boards before this one, and they all scoff at the notion, claiming firefox/chromes usage of the registry is minimal. What do you think?

Jumping back to the original question, jaclaz, I would LOVE to hear your own personal thoughts(as well as anybody else reading this) as to whether or not firefox/chrome history/searches ect. would be found in any of these backing files including ntuser.dat files. I asked on other message boards before this one, and they all scoff at the notion, claiming firefox/chromes usage of the registry is minimal. What do you think?

Basically the concept that firefox/chrome and *any* other browser but Internet Explorer (or possibly nowadays the stupid Edge) is essentially correct.

The main difference is that for several releases of the Windows NT operating systems (since Windows 2000) Internet Explorer is (stupidly) "integrated" in the OS and is used for several other chores (besides "pure" Internet browsing).

All "other" browsers are "normal" programs and they do not make the same level of (ab)use of the Registry, a laege (if not all) data coming from the use of such browser goes on dedicated files, outside of the Registry.

As a general rule of thumb anything by Microsoft makes an extensive (and often improper) use of the Registry while every other non-MS program limits its use of it.

This article
http//link.springer.com/content/pdf/10.1186%2F1687-417X-2013-6.pdf
might be of interest to you, even if it is aimed to a slightly different topic, as well as this "cheatsheet"
https://uk.sans.org/posters/windows_artifact_analysis.pdf

jaclaz

Well, that answers that. Even that article you linked me to says explorer history can be found in ntuser.dat but doesn't mention it with firefox/chrome.

I'll definitely test this for myself. And I'll definitely come back here for help/advice! Thanks guys!

Bump

I was looking through an offline ntuser.dat file when I came across some random search terms lumped together. I have absolutely no idea how to determine what application was used with these searches, so I figured I'd just copy and paste it and see if anybody here would be able to look at it and determine what wrote it there.

I bolded the search terms, and included the text a few lines above and below it. I have no idea how much of it is relavent to the search terms. If anybody can tell me what, or if it is even possible to determine what was used here, I'd be very greatful.

a , m 7 5 , m 1 5 , m p 2 , m p m , m p v , m p a ¸ÿÿÿm p e g , m p g , m 1 s , m 1 a , m p 2 , m p m , m p a , m 2 a Øÿÿÿvk
0 audio/3gpp pç Øÿÿÿvk 8 video/3gpp2 ¸é àÿÿÿp i c t , p i c , p c t ¸ÿÿÿm p e g , m p g , m 1 s , m 1 a , m p 2 , m p m , m p a , m 2 a Øÿÿÿvk = audio/3gpp2 pç Øÿÿÿvk ÈÁ video/sd-video Ðÿÿÿvk hÊ application/x-mpeg på Øÿÿÿvk ` video/mp4ç Hç Øÿÿÿvk Z audio/mp4ê (ê Øÿÿÿvk h audio/x-m4a €í Øÿÿÿvk X% audio/x-m4p pç Øÿÿÿvk À= audio/x-m4b ¸é Øÿÿÿvk í image/x-pictpç Øÿÿÿvk €f image/pngê ¸é Øÿÿÿvk 0r image/x-png Èì Øÿÿÿvk W image/x-sgi hï Øÿÿÿvk °V video/x-m4v (î Øÿÿÿvk
àé image/pict pç Øÿÿÿvk
ÈW image/x-targaç Øÿÿÿvk øs image/jp2ê ¸é Ðÿÿÿvk À‡ image/x-jpeg2000-image Øÿÿÿvk
€s-À Time StampltÿÿÿÿØÿÿÿvk
¸q ExtensionsÿÿÿÿÿÿØÿÿÿW i n d o w s B i t m a p s èÿÿÿè¹ ð¼ ÀÒ 8Ñ 0308Øÿÿÿvk ¸| image/jpeg2000 Ðÿÿÿvk ¨€ image/jpeg2000-image æ 8ÿÿÿpå ¸å æ 0æ Xæ ˆæ Øæ ç pç ˜ç Hç Pè ðç Øè xè é (é ê ¸é xê (ê Pê ê Èê ðê ë ¸ë àë Èì ðì €í ¨í Ðí î (î Pî xî î hï ï Èî ðî ï @ï ¸ï àï Èð ðð ð pþÿÿw h e r e i s b i n l a d e n ^ e j e r u s e l i m ^ e a c r e m i d d l e e a s t ^ e a c r e ^ e s c o t l a n d l o c h n e s s ^ e s c o t l a n d ^ e l o c h n e s s s c o t l a n d ^ e l o c h n e s s ^ e n h l ^ e a n n e f r a n k h o u s e ^ e n l a k e ^ e p l e a s e n t p o i n t w v ^ e ^ e p l e a s e n t p o i n t ^ e t y a r d ^ e ÿÿÿ ÿØÿÿÿvk
€Ÿ>À I Time Stampo c a Øÿÿÿvk
È H Extensionsy \ H Øÿÿÿvk €€ M Compression TypeØÿÿÿvk
èo 0 Black Is 0 Ðÿÿÿvk à ShowProgressDialog vk Øÿÿÿvk € ptDescription vk àÿÿÿvk xk epBiLevel àÿÿÿG o o g l e C h r o m e À nk ÜK·“®Ñ K ÿÿÿÿÿÿÿÿ 3 ÿÿÿÿÿÿÿÿ . + Servicesh vk € ProcessIDnable€ @ h t t p / / s o u t h p a r k z o n e . c o m / r k . c o m / e . a s p ? M T = y o u t u b e . + o m & s r c h = 3 & p r o v = g o g l & u t f 8 ÿ h t t p / / l u c a s a r t s . c o m / e t w o r k . c o m / e . a s p ? M T = y o u t u b e . + o m & s r c h = 3 & p r o v = g o g l & u t f 8 ÿØÿÿÿvk €€ I Compression TypeØÿÿÿvk
Ø× I Black Is 0o c a Ðÿÿÿvk h
H ShowProgressDialogs t o Øÿÿÿvk € s Description 8 0 Øÿÿÿvk @÷ \ GrayScale vk àÿÿÿ1 3 6 , 1 3 1 , 0 ÿ¨ÿÿÿnk Ø(v¯Ñ ( ÿÿÿÿÿÿÿÿ pp 8× ÿÿÿÿ ·ðy Shell¶Ä é]c»éœ¶ÒNÍó¿ê¡V%YÐÿÿÿvk €ÿÿÿÿ ZØMinPos1024x768(1).y®Šl>}Ðÿÿÿvk €ÿÿÿÿ ƒMaxPos1024x768(1).x¦†CÐÿÿÿvk €ÿÿÿÿ %
MaxPos1024x768(1).yR‚vŠ‚àÿÿÿvk €ÿÿÿÿ 4Buttonsɨÿÿÿnk œKڐ¯Ñ Ȩ1 ÷ ÿÿÿÿ ˆô 8× ÿÿÿÿ D 1tem Dat List

I was looking through an offline ntuser.dat file when I came across some random search terms lumped together. I have absolutely no idea how to determine what application was used with these searches, […]

Then you better learn. Get a decent book that explains what the registry is. Myself, I recommend Jerry Honeycutt's book, but there are other possibilities. (That recommendation assumes you do want to learn.)

… so I figured I'd just copy and paste it and see if anybody here would be able to look at it and determine what wrote it there.

Yes, you do need to learn about registry.

The registry is a navigational database of information related to the configuration of the system and its applications (past and present). It contains namevalue tuples, organized in a key tree, somewhat similar to a file tree in a disk volume. The interpretation of a particular namevalue tuple depends to great extent on the path where it is located. There are reasonably few givens here an application can create its own paths for the data it needs to keep around. But it needn't document them, or register them or anything like that. Nor does it need to retain the same meaning of a configuration. That may mean that registry analysis depends very much on the versions of the software that created and used the keys and namevalues that are of interest.

Exaggerating a bit, your question is somewhat similar to 'I've found these terms somewhere in an image of a hard disk. Can anyone tell me what program wrote them?'

In fact, the similarities extend to the notion of unallocated blocks. The registry files contain deleted data, just as a disk image contains deleted data. Analysis will be just as difficult as tracing an unallocated block to the file it once was part of.

In this particular case, it sounds as if the RegFileExport tool from Nirsoft might be useful. You'll get a text file of all active contents in the registry file, and you can search that for the terms you have found. The path at which they are found may give you additional information – at the very least, you will be able to ask if anyone is familiar with that particular path, what application(s) that use it, and to what purpose. (Other tools may give you similar means of doing the same thing.)

However, tools won't help you understand. Again, get a decent book on Windows registry analysis.

The registry is a navigational database of information related to the configuration of the system and its applications (past and present). It contains namevalue tuples, organized in a key tree, somewhat similar to a file tree in a disk volume.

Yep ) , I have to repeat myself

Additionally, a hive file is (it depends on how you see it) either a database or a file system, so it is entirely possible (unless it has been - again it depends whether you see it as a database or as a file system - "compacted" or "defragged") that there is something that you can find analyzing the RAW file that wouldn't come up in the mentioned tools.

A hive is a database and at the same time it is a filesystem (with some starking similarities to NTFS, like permissions and softlinks or reparse points).
As a matter of fact there is (was) a filesystem driver for the Registry under Windows NT systems and there is a Linux project, hivexsh
http//libguestfs.org/hivexsh.1.html
that uses commands like cd, ls, and lsval to navigate the "filesystem tree".

jaclaz