Welcome Guest.   Make a donation to an author on the site November 23, 2014, 01:46:13 PM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
View the new Member Awards and Badges page.
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1] 2 3 4 5 Next   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: ZillaRank: An improved search for smf forum  (Read 148197 times)
mouser
First Author
Administrator
*****
Posts: 33,692



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« on: July 04, 2007, 01:48:22 PM »

As many of you know, we often joke that the SMF search function on the forum sucks, and seems to miss returning search results that we know are there.

Well, our own Wordzilla has been immersing himself in the SMF code, and has begun totally revamping the search code used by the forum.

He's just gotten started, but we need your help to:
  • test it and see if you can find any posts that should be returned during a search but aren't -- this is most important request -- try to find searches that are missing results that should have been found
  • make other suggestions on what you'd like to see improved (one thing already planned by wordzilla is to be able to sort results easily by date, author, relevancy).

We look forward to your help in bughunting!


NOTE: SEE ALSO THIS THREAD: http://www.donationcoder....um/index.php?topic=9134.0
« Last Edit: August 01, 2007, 10:24:36 AM by mouser » Logged
jgpaiva
Global Moderator
*****
Posts: 4,711



Artificial Idiocy

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #1 on: July 04, 2007, 02:17:26 PM »

I'm glad to know this is working in the forum!!!

First suggestion:
Ignore the "MOVED: " threads, since those are a repetition of the real thread.
Logged

nudone
Cody's Creator
Columnist
***
Posts: 4,116



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #2 on: July 04, 2007, 02:22:20 PM »

fantastic.

i'm sure wordy will make a brilliant job of it.
Logged
Wordzilla
Forum Search Daemon
Charter Member
***
Posts: 470



Two there should be; no more, no less.

View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #3 on: July 04, 2007, 02:27:50 PM »

Good idea Joao, maybe I should give all "MOVED: " threads extra low relevance score.

I'm hoping that this first update will be able to address these 2 issues:

1. Missing results

2. Excessive noise/irrelevant info in results


The next update to forum search will (hopefully) feature a better relevance ranking system. We dont need having 10-pages of results - IMO just a dozen most relevant topics on topmost would suffice, for most people.  smiley
Logged

Lashiec
Member
**
Posts: 2,374


see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #4 on: July 04, 2007, 05:12:56 PM »

So this is the the SMF search you were going to put your hands on Wink

Excellent! Now all the other forums (especially those using phpBB) would be envious of the improved search function here
Logged
f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #5 on: July 04, 2007, 06:09:10 PM »

If this turns out well, are you going to contribute this back to the SMF people?
Logged

- carpe noctem
mouser
First Author
Administrator
*****
Posts: 33,692



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #6 on: July 04, 2007, 06:21:18 PM »

Quote
If this turns out well, are you going to contribute this back to the SMF people?

Of course.  What a ridiculous question! we're not savages here!  tongue
Logged
lanux128
Global Moderator
*****
Posts: 6,132



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #7 on: July 04, 2007, 09:13:14 PM »

so how do i get involved? can i just start typing the queries now? smiley
Logged

mouser
First Author
Administrator
*****
Posts: 33,692



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #8 on: July 04, 2007, 09:23:45 PM »

yep, just start searching for all those posts you couldnt find before smiley
Logged
mouser
First Author
Administrator
*****
Posts: 33,692



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #9 on: July 04, 2007, 09:41:07 PM »

Here's a really good idea come up with on the irc channel with hamradio:

Right now if you search using the editbox at top of the forum, while you are in a forum section, it will ONLY search in that forum section.

This is very disorienting and most people don't realize it.

What if ZillaRank changed this behavior and always searched the whole board, BUT (here is the clever part), gave a higher score bonus to results from the section you are in at the time of search.
Logged
lanux128
Global Moderator
*****
Posts: 6,132



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #10 on: July 04, 2007, 10:52:45 PM »

What if ZillaRank changed this behavior and always searched the whole board, BUT (here is the clever part), gave a higher score bonus to results from the section you are in at the time of search.
that would be cool! i fully agree.. Thmbsup
Logged

f0dder
Charter Honorary Member
***
Posts: 8,774



[Well, THAT escalated quickly!]

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #11 on: July 05, 2007, 04:09:28 AM »

Quote
If this turns out well, are you going to contribute this back to the SMF people?

Of course.  What a ridiculous question! we're not savages here!  tongue
Well, I meant more in the sense "and do you reckon they might fix up their default behaviour?". Doing the contribution itself is mostly a no-brainer, I even think their license requiresa it? smiley

Right now if you search using the editbox at top of the forum, while you are in a forum section, it will ONLY search in that forum section.

This is very disorienting and most people don't realize it.

What if ZillaRank changed this behavior and always searched the whole board, BUT (here is the clever part), gave a higher score bonus to results from the section you are in at the time of search.
It's very disorienting indeed!

I think a drop-down solution might be nice, with "prefer hits from this section", "only search this section" and "just search entire board, dammit!" choices...
Logged

- carpe noctem
Wordzilla
Forum Search Daemon
Charter Member
***
Posts: 470



Two there should be; no more, no less.

View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #12 on: July 05, 2007, 05:28:49 AM »

Quote
I think a drop-down solution might be nice, with "prefer hits from this section", "only search this section" and "just search entire board, dammit!" choices...

Yup I believe this is more practical. An update to the script will be ready soon. By default it searches the entire forum unless user specifizes board/topic.
Logged

Wordzilla
Forum Search Daemon
Charter Member
***
Posts: 470



Two there should be; no more, no less.

View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #13 on: July 05, 2007, 12:50:41 PM »

Update: v1.2.1 is now live on DC forum

So what's new?

1. Searches entire forum by default, but is also capable of search within board/topic

Quote
Right now if you search using the editbox at top of the forum, while you are in a forum section, it will ONLY search in that forum section.

Now you no longer have to go back to forum index to search entire site, with new our DC smf search.

Quote
I think a drop-down solution might be nice, with "prefer hits from this section", "only search this section" and "just search entire board, dammit!" choices...

It's fairly easy for us to modify the current search boxes to better support searching within board/topic, if desired.


2. Active board ranking adjustment (thanks to mouser)

Quote
What if ZillaRank changed this behavior and always searched the whole board, BUT (here is the clever part), gave a higher score bonus to results from the section you are in at the time of search.

The script now raises the ranking of all matching topics within the forum you are browsing, while displaying all matching topics in the entire forum.

By applying a multiplier to existing smf relevance score, it distorts the ranking just a bit.

It only taking a boolean value change to turn off this feature.


Unlike standard smf search, DC search does not ignore words with 3 letters or less. So any keyword counts!  smiley

feedback
Logged

mouser
First Author
Administrator
*****
Posts: 33,692



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #14 on: July 05, 2007, 12:54:44 PM »

the new active board scoring heuristic is great.
(note for testers, if you do the same exact search twice in a row you wont see the effect because of smf caching, so make sure if you test this you conduct a search for some other term in between tests).
Logged
Wordzilla
Forum Search Daemon
Charter Member
***
Posts: 470



Two there should be; no more, no less.

View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #15 on: July 05, 2007, 01:24:44 PM »

Also numbers are now acceptable as keywords, try:

20% discount, smf 2.0, etc.
Logged

Wordzilla
Forum Search Daemon
Charter Member
***
Posts: 470



Two there should be; no more, no less.

View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #16 on: July 05, 2007, 03:53:25 PM »

Update: v1.2.2 is live now

thanks to our testers (esp. cranioscopical and Lashiec), a bug involving fulltext search engine and '-' dash handling has been fixed. 

Thanks!! Thmbsup
Logged

Wordzilla
Forum Search Daemon
Charter Member
***
Posts: 470



Two there should be; no more, no less.

View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #17 on: July 06, 2007, 01:33:12 PM »

Update: v1.4.0 is live now

A few minor bugs have been fixed.

And much improved search performance for:

1. Phrases

2. Words that are prefixed and/or suffixed


We are looking forward to releasing this script soon. Join us in bughunting and plz do lemme know if you run into anything search-related that confuses/bothers you!

feedback
« Last Edit: July 06, 2007, 02:00:09 PM by Wordzilla » Logged

nudone
Cody's Creator
Columnist
***
Posts: 4,116



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #18 on: July 06, 2007, 03:10:17 PM »

fantastic!
Logged
mouser
First Author
Administrator
*****
Posts: 33,692



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #19 on: July 06, 2007, 11:04:04 PM »

I hearby offer a prize to anyone who can now find a search on this forum that misses a result it should return.
Logged
lanux128
Global Moderator
*****
Posts: 6,132



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #20 on: July 07, 2007, 12:57:51 AM »

I hearby offer a prize to anyone who can now find a search on this forum that misses a result it should return.
which begets the question: what is the prize? smiley btw, good job, WordZilla.. now to undo the habit of searching from Fx's search plugins..
Logged

mouser
First Author
Administrator
*****
Posts: 33,692



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #21 on: July 07, 2007, 01:07:01 AM »

Quote
what is the prize?
that depends on if i have to give it out.  if not, it's a million dollars.  if so, it's a piece of gum and a kind word.
Logged
taichimaster
Charter Member
***
Posts: 159



View Profile WWW Give some DonationCredits to this forum member
« Reply #22 on: July 07, 2007, 01:21:08 AM »

Quote
what is the prize?
that depends on if i have to give it out.  if not, it's a million dollars.  if so, it's a piece of gum and a kind word.

 Grin Grin Grin Grin
Logged

cranioscopical
Friend of the Site
Supporting Member
**
Posts: 4,195



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #23 on: July 07, 2007, 08:22:07 PM »

Quote
hearby offer a prize to anyone who can now find a search on this forum that misses a result it should return.

I searched for the message telling me that I'd won all of the prizes in this month's draw, but it couldn't be found!
Logged

Chris
Wordzilla
Forum Search Daemon
Charter Member
***
Posts: 470



Two there should be; no more, no less.

View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #24 on: July 08, 2007, 04:27:05 AM »

DC SMF Search Mod is out!

http://www.donationcoder....ic=9134.msg67388#msg67388


Quote
Improvements to existing SMF code

  • Significant reduction of missing results (now very rare)
  • Less noise and therefore reasonably better relevance
  • Enhanced phrase search
  • Intelligent handling for ‘ apostrophes and – dashes
  • Supports short keywords (3 characters or less)
  • Supports stopwords (clichés)
  • Selectively expanding the search by finding keyword inflections


Features unique to this mod

Active board ranking heuristic (based on mouser’s suggestion)


Now u can even search phrases with ' apostrophe(s) in them, e.g. "it's worth it", "mouser's"

At last I no longer have to send my little bot to crawl DC forum twice-monthly! Grin

I hope other SMFers will find it useful as well.
Logged

Pages: [1] 2 3 4 5 Next   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.051s | Server load: 0.07 ]