…snip…
Using a copy of local cache is also an interesting idea but for an expansive website it could be quite cumbersme to make sure you access every page linked on the site.
…snip…
Thanks for the ideas and responses!
That is what httrack (from my new limited experice with it), wget -r, and the web proxy paros, do.
They follow every single link on the page (spider it if you will).
No, if there are pages on the server, but not linked….well then you would have the same problem if you were following every link manualy.
I hear what your saying, it isn't easy, anyway you look at it (except the real orgional static site).
Good luck man,
Skip
this can be done in a couple of ways.
1) you can you adobe acrobat professional to to PDF an intaire site. it will include all the pages from the links and can contion working links.
2) use Microsoft visio to open the site and it will copy the site and return you a flow chart of the site.
3) and probaly the most extensive would be to use Black Widow to copy the entire site and serach for anyting you want.
Hope this helps
Here is some more info on Black Widow
it is made by Softbyte Labs and is available at http//
here is the info from there site.
Internet scanning and downloading tool for the expert and the novice.
Download pictures or any kind of files, a whole web site, or just build the structure of a site in an explorer view.
Network Spy Monitor your internet activity, listing all the links you have visited, including images and downloaded files.
Site Structure Reproduce the web site directory structure, as if you were to look at the site using Windows Explorer.
Web Browser Equipped with a browser so that you can navigate to the site or page you want to scan.
Page Source View the web page source codes, even those in frames or blocked.
Errors Find broken links, mistyped links, server errors such as configuration errors etc. Great for Webmasters.
Download Download only specific files, using wildcards, regular expressions and file size.
Expandable BlackWidow can be customized in any which way you like by using Plugins.
Multi-Language You can run BlackWidow in a multi-language interface.