Every now and then, I'm asked to erase various files and directories from Windows PCs. Usually, I just use a commercial file wiper and don't think anything of it. Then I decided to audit the wiping since this was the first NTFS file system that I had to wipe. To audit the wiping, I ran an off-the-shelf file recovery program. I was surprised to see that the files were renamed but completely recoverable. The program said that the files belonged to a "Lost Directory" and were lost files since Windows no longer could recognize them.
I reran the wiping with numerous commercial wipers followed by the recovery software. Files were still there.
The OS is Windows XP and the file system is NTFS. Is there something about NTFS that current wiping software can't handle?
It might help if you stated the name of the wiping product(s) you used. I believe that there was once an analysis performed on various wiping applications, sometime within the last 2 yrs or so, and those products might be on the list.
Also, what recovery program did you use?
H
I was using SecureClean and Eraser2003, both are fine products. I noticed the problem when I ran RecoverMyFiles with a physical scan. The report showed recoverable files linked to a "lost files" directory.
After thinking about it, the most wiping products would only walk the NTFS known structure. My understanding is that the "lost files" are really "lost sectors/clusters" that are no longer linked to NTFS. On the other hand, RecoverMyFiles with the physical device scan works like forensic software in that it looks at each header on the sector and notes the file type, size and etc.
The lost sectors/clusters are caused by having to power off Windows because either the app hung, Windows hung, or Windows wouldn't shut down. Thinking that its really a Windows problem, I looked at various commands that would relink the lost sectors to either a recovered directory or to free space. I haven't found the right command yet. "Chkdsk doesn't do it. There is another command, "ckntfs" I believe, but I haven't tried it. If all else fails, I could Ghost the NTFS structure to a work disk, initialize the whole drive, then Ghost the image back. Probably would work.
It would take a very complex wiper product to walk the NTFS and gather known file locations for a index and then consider everything else as free space. I don't know if that type of product exists or not.
Also, does anybody know what Windows commands should be used to determine if lost clusters/sectors exist? And, has anybody else used a Windows command to relink the sectors back into the freespace chain?
I often use the ZAP function of SecureClean, because it is easy to use. However, I have never tried to validate it.
For a general drive wiping I use X-Ways Security. My best recollection is that I have often validated that visually in X-Ways Forensics and WinHex. I say that that is my best recollection, because I can't be certain that I visually examined the entire drive, sector by sector (I use Hex 00, so it is easy).
It sounds like the files that are being missed are orphan files. You might want to examine the drive in a Hex Editor to see what is going on.
Yes they are "orphan" files. I've checked with some Windows guys and nobody seemed to be real familiar with the internals of NTFS and how to relink the files, so that they could be deleted.
I like the Zap function, too. I like the graphics.
I don't think that you necessarily need to link them, if your goal is to make them go away. You seem to be "married" to SecureClean. Ditch SecureClean and overwrite the entire physical drive with something else. WipeDrive is a DOS program made by White Canyon, the same guys who make SecureClean. You may want to consider that. I have found White Canyon to be helpful on the phone. As I say, I'm comfortable using X-Ways Security to overwite the physical media, but I have never tried to validate it.
Another quick experiment you might want to conduct involves the use of the FTK Imager. If you don't use FTK you can get it as a free download of FTK Imager from the Access Data web site. The sign says that you have to pay, but you really don't! The people at Access Data seem to know that and let it go for nothing (A kind of "Puppy Selling" I suspect.) In any case, the FTK Imager with allow you to to look at the Hex in a matter of seconds. It will break it down into 3 main folders, "Root", "Unallocated Space" and "orphan files". You can then inspect each folder visually to see what is going on.
FTK Imager is a good idea to help see what is going on.
The reason I don't use wipe the complete drive is that I don't want to destroy the rest of the drive's data. I just need to remove the orphaned files.
It's kind of bothersome to think that if a person abnormally ends an app or powers off the PC, that orphaned files are created. Or, for that matter, if a parent directory is deleted, they are created.
You seem to be "married" to SecureClean. Ditch SecureClean and overwrite the entire physical drive with something else.
I have used several file wiping utils, and spent time to validate some.
The most trustworthy I have found is Accessdata's SecureClean (Whitecanyon), but it all depends on if you want to sanitise an existing O.S., or the entire physical drive. I have found most other file wipers lacking in some respect, one or two others quite dangerous in that they simply don't do what they're advertised as doing.
Another satisfactory util I have found is "DirectorySnoop" (briggsoft.com).
This will wipe freespace (unallocated), and will purge deleted MFT entries, therefore leaving no trace of "empty temporary files" created by other file wiping utils…including itself. But this is only really any use when you know what it is you're deleting, and are happy knowing where the data was on an existing O.S. Just my tuppence worth D
In the not so old days Unix SA's had to periodically backup filesystems, rebuild the filesystem and restore the filesystem from the backup. As this was the only really quick and efficent way to compress and defragment filesystems. I've spent many a wee morning hour performing these tasks on Unix, NT and Win2K, and even XP(lately). No filesystem is completely free from these anomalies. Unix comes the closest. If your system has experienced numerous power failures or aborted shutdowns you will have these missing clusters.
Most recovery tools perform some level of intelligent guessing when recovering files. This is because after a file or directory has been deleted parts of the previous structure can be reallocated, leaving the appearance of orphaned components. If a directory tree has been deleted and part of the upper most structure has been reallocated. The identified lower structure components to not have a place to be reconnected. These typically are referred to a lost - in my experiences.
I will probably get some flames about this comment – I have never seen any commerical product live up to it's marketing claims. Yes, some come closer than others. I have a philosophy of such things - I will not bore you with them.
I prefer to write programs/scripts or procedures to accomplish rather compensate for things of this nature.
In this particular case - I would backup the filesystem (file based backup), wipe the partition, format the partition and restore the filesystem from backup.
I recommend "File System Forensic Analysis" by Brian Carrier and "Forensic Discovery" by Farmer & Venema as good reference to understand how things work – both books are extremely deep and dense.
At least in the unixworld there are a lot of "secure remove" tools which first overwrite the file and after that, remove them. I wrote one myself (perl) and tested it on some small files. After recovering the deleted files with sleuthkit/autopsy browser, they appeared to to contain only the random data i put in myself using my srm (secure remove). I haven't tested it on big files nor on other filesystems as (v)fat and ext2/3.
http//
Maybe it could run in activeperl and works on Windows too.
Just an idea.