Main Area and Open Discussion > General Software Discussion
How to make a local copy of an ancient Web forum?
mouser:
i still prefer Offline Explorer (Pro?) over teleport (last time i checked at least), but it is also expensive.
David.P:
Thanks for the tips -- however it seems that basically, each of Httrack and BlackWidow 4.4 (both free) basically can do most of what those commercial website rippers offer.
Otoh -- Httrack, BlackWidow and also WebsitePacker all seem to load tens of thousands of links from that forum where actually there are only are a few thousand useful entries. It seems as if that forum's software generates unique URL's every time something is accessed. This is why every single posting of that forum gets checked and downloaded like a dozen times instead of only once.
So far, I think that BlackWidow is the best software since it can build an explorer-like tree representation of the web site without actually downloading everything first. Then in the end, you only pick the folder containing the stuff you want and download that folder.
It however still seems to take days for BlackWidow to build that folder tree (if it ever gets there at all), for the same "unique URL" reason above... >:(
David.P
Paul Keith:
I haven't tried these before. Can anyone provide an estimate for how much space it will take to download an entire forum/website on average? I'm wondering whether as a casual user, I'm better off with the Scrapbook Firefox extension though to be honest, I don't know much about capturing to a certain depth.
J-Mac:
I haven't tried these before. Can anyone provide an estimate for how much space it will take to download an entire forum/website on average? I'm wondering whether as a casual user, I'm better off with the Scrapbook Firefox extension though to be honest, I don't know much about capturing to a certain depth.
-Paul Keith (November 21, 2008, 08:27 AM)
--- End quote ---
That depends, Paul. What do you consider to be an average forum/website? "Average" is relative to each user, I've found.
However even a relatively small forum has tens of thousands of links and thousands of separate page views. I would guess that for most forums it would be more tha most users would want to store on their home computers. I'm talking here about the front-end views, of course. If you had access to a forum's back-end you could store just the database, which would be considerably smaller.
Jim
Paul Keith:
Oh that's too bad. I would consider DonationCoder to be an average sized forum.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version