Main Area and Open Discussion > Living Room

For DC Fans: Bring DC Forum to your Desktop for searching/browsing/etc.

(1/7) > >>

Wordzilla:
I recently wrote a DC forum crawler app that reads topics/posts on the DonationCoder.com forum, and saves them as individual html files.

After 10 hours of continuous torturing mouser's server :mrgreen:, we now have 8277 individual html files in a compressed RAR package (divided into 4 vols), which is available from:

http://www.mrcody.com/crawl/032007/

Download the four RAR files (67.9MB in total) and uncompress (458MB) to your desired directory.


This package presumably archives all publicly available (anonymous-level) topics/posts:

From topic #1 (restricted) to topic #7825 (Which programs have you found that do not work with Vista?) as at 03:06:11 AM, March 20, 2007 (Central Time)

Wordzilla:
You will perhaps find it much more useful than it appears (useless and redundant) if you are:

1. A DC hardcore  8)
2. Using one or more desktop search engines/utilities like Yahoo! X1, Google Desktop Search, Copernic and Windows Desktop Search

From my experience, my currently installed desktop search tools (all of the above) yield more and also more accurate results than Google does, in most cases (given the same search query), and of course much faster.

Another advantage of this approach is that search results are often previewable.


We have a member by the username of sable here and she has a huge champion cat which WoWs me, now I need to find the pic of her cat to show you.  ;)


Query: sable cat


My X1 Desktop Search - took it 5 mins to completely index these html files




Google Search - Google is your friend!




DC Forum Search - SMF search sucks  :P

jgpaiva:
One has to love the amazing oh-so-useful forum search results..  ;D ;D
This actually is a very good idea. Next time i search DC and can't find what i'm looking for, i'll know where to go  :Thmbsup:

urlwolf:
This is very clever and a real timesaver.
Which of the  desktop search engines you recommend?
Thanks!

Wordzilla:
Thx!  :)

Which of the  desktop search engines you recommend?

--- End quote ---

If you would like to get the most out of the DC forum crawl, perhaps it's a good idea to dedicate one search engine to indexing & searching these files, like what I did.

In this case, go for Yahoo's X1 Enterprise Client (free). Fast indexing & full-text search with highlighting.

For DC Fans: Bring DC Forum to your Desktop for searching/browsing/etc.



Go to Tools -> Option and set up the search engine like shown below (just my suggestions):

No Delay Indexing

For DC Fans: Bring DC Forum to your Desktop for searching/browsing/etc.

Index exclusively DC Forum html pages

For DC Fans: Bring DC Forum to your Desktop for searching/browsing/etc.

For DC Fans: Bring DC Forum to your Desktop for searching/browsing/etc.

Build one-time index (takes about 5 mins)

For DC Fans: Bring DC Forum to your Desktop for searching/browsing/etc.


note: beware that the X1 client may ask u to install yahoo toolbar etc. during installation, remember to deselect them.

Navigation

[0] Message Index

[#] Next page