Main Area and Open Discussion > General Software Discussion

How to make a local copy of an ancient Web forum?

(1/6) > >>

David.P:
Hi Forum,

there is this stone-age Web forum by the L&H company that has ceased to exist already like a decade ago:

http://support.lhsl.com/databases/dragon/webdisc.nsf/

It is still operative but it is to be feared that the current site owner is simply going to shut the forum down pretty soon as he has done with other similar forums.

Therefore it would be great to find a way how to make a local copy of all that forum's threads in some way.

I have already tried Adobe Acrobat  :-[ and WebsitePacker (this is great, makes *.CHM files out of an entire site).

However both take ages and download literally Gigabytes of stuff because these tools are too stupid not to click on each and every link (especially the "Collapse" and "Expand" links next to every thread :mad:). Therefore, every single posting sort of gets downloaded like a dozen times instead only once, and it would take days, create hundreds of thousands of files and use up dozens of Gigabytes to download everything.

Any more ideas how I could actually rip that entire forum to one compact file or directory structure?

Thanks heaps already,
David.P

mouser:
This is actually a great question -- i look forward to hearing the replies about what to do.
Do i understand correctly that you do not have access to the backend database? ie you just have normal forum member rights to view posts?

David.P:
Do i understand correctly that you do not have access to the backend database? ie you just have normal forum member rights to view posts?
-mouser (November 19, 2008, 11:38 AM)
--- End quote ---

I have the same rights to that L&H forum e.g. as you, or as everyone else reading the present thread.

Probably it would need some sort of "intelligent" crawler that only downloads links up to a certain depth (like about depth "2" or something) while neglecting other links (like especially those "Collapse" and "Expand" links).

Cheers David.P

mouser:
yeah for a forum, what you might be better off with is using a web spider thing but NOT in the mode that crawl a website, but rather in a mode that grabs all pages of the form:
https://www.donationcoder.com/forum/index.php?topic=1
https://www.donationcoder.com/forum/index.php?topic=2
https://www.donationcoder.com/forum/index.php?topic=3
etc.

The only tricky thing is for topics that are multiple pages long.

city_zen:
Teleport Pro is supposedly the best of its class, but it's a bit expensive. It's probably worth taking a look at it, though, to see if it has the features you need for this job.

Navigation

[0] Message Index

[#] Next page