Anyone have any tips for how to create a great custom search engine?

[Paul Keith]

Man, I just wasted half a day trying to make BuildaSearch.com and Google's custom search engine work.

I can't explain why both of those apps work and not work as soon as I change something.

BuildaSearch was working fine until I tried editing the search engines and it just put all the search engines in one line but it's a beta.

Google's initial bulk engine won't give me a preview even when I changed the engines to using wildcards that I had to go to each individual engine and set them back to their default state and click on using the entire url even though I did that previously with the bulk site loader and it didn't work earlier on.

Worse, Google can have more sites than BuildaSearch but the site results seem to be worse. I also don't know how to manipulate the indexing methods and I ended up getting a more customizable search engine that didn't search for anything because BuildaSearch won't detect the screwed up lists and Google ends up being so sporadic, I don't really know how to tweak it.

A search for to do for example gets on to the 4th search engine before it has any relevance to a "to do list" even though I have added Lifehacker as one of the search engines. Also spiders who spider sites like popurl and alltop seem to be unable to yield the links within them.

[tinjaw]

Have you tried Rollyo? I haven't played with it for years, but I remember it being pretty good.

[Paul Keith]

Thanks tinjaw. That worked surprisingly well.

I'm still not a fan of the search results but I guess I really have to narrow down the search engines.

[Lonestar75]

Paul,

 I one of the developer of BuildaSearch, what is your "buildasearch.com/yoursearch" search called?  I can look up the list and let you know what is wrong with it. 
What is the URL where you have BuildaSearch installed?

Diego

Man, I just wasted half a day trying to make BuildaSearch.com and Google's custom search engine work.

I can't explain why both of those apps work and not work as soon as I change something.

BuildaSearch was working fine until I tried editing the search engines and it just put all the search engines in one line but it's a beta.

Google's initial bulk engine won't give me a preview even when I changed the engines to using wildcards that I had to go to each individual engine and set them back to their default state and click on using the entire url even though I did that previously with the bulk site loader and it didn't work earlier on.

Worse, Google can have more sites than BuildaSearch but the site results seem to be worse. I also don't know how to manipulate the indexing methods and I ended up getting a more customizable search engine that didn't search for anything because BuildaSearch won't detect the screwed up lists and Google ends up being so sporadic, I don't really know how to tweak it.

A search for to do for example gets on to the 4th search engine before it has any relevance to a "to do list" even though I have added Lifehacker as one of the search engines. Also spiders who spider sites like popurl and alltop seem to be unable to yield the links within them.

-Paul Keith (January 28, 2009, 06:20 AM)

[Paul Keith]

Hi Lonestar. Sorry, if I didn't contact you guys. I understood that the service is new and rough around the edges so I didn't bother. (For ex. the site seems to retain your old customization but when you're back in the customize look screen, it drops to the default settings even if it retained your settings)

The problem pretty much is this:

hubpages.com__gather.com__clusty.com__piratebay.org__mininova.org__demonoid.com__donationcoder.com__lifehacker.com__mixx.com__reddit.com__slashdot.com__friendfeed.com__stumbleupon.com__twitter.com__diigo.com

Not only did it get this way when I tried re-editing the engines but this is my original list and not the re-edited version. (I've tried both going with a single and multiple search change but it retained this error)

This is the url of the search: http://www.buildasea...ch.com/scrapemantics

[Lonestar75]

Paul,

Not sure what caused the underscore problem, I have fixed the list and it should be working now.  Let me know if you have any other questions or feedback.  If you need to edit the "url" list, it is recommended to use the "multiple" sites option. BTW we are releasing our new release in a few weeks, do you want us to notify you of the release?

Diego

[Paul Keith]

No, I'm good. I was just testing your service out because I happened to be reading past articles of killerstartups.

I probably need more time to collect my list of search engines anyway.

[Paul Keith]

Sorry Lonestar. Error happened again as soon as I tried switching a search engine with zudos.com

[Lonestar75]

Hummm.... What browser are you using?  You are making the URL updates via the "multiple" url textbox is that correct?  It should work once again, looking into the issue.

[Paul Keith]

I'm using Firefox 3.0.5 (though in general I tend to switch between Chrome, Opera, Safari and IE)

All my usage with BuildaSearch so far has been with Firefox though. I've also enabled NoScript on it but outside of that, I can't think of any peculiar firefox extension I might have that would be interfering with a text box.

[Lonestar75]

The underscore issue only occurs when you try to edit is that right?  What happens if you copy and paste the list below and click "save changes" 

====LIST BELOW====
hubpages.com
gather.com
clusty.com
piratebay.org
mininova.org
demonoid.com
donationcoder.com
lifehacker.com
mixx.com
reddit.com
slashdot.com
friendfeed.com
stumbleupon.com
twitter.com
diigo.com

[Paul Keith]

It still doesn't work. I've tried it before but I thought I'd try it again and it gave the same result.

[Lonestar75]

It is very odd... have you tried using any other browser except IE?

[Paul Keith]

That's weird. I just did. It fixed the error when I used the site on Opera. Usually Firefox is the one that works fine.

Btw just a heads-up, Rollyo isn't as good as I thought it was as far as setting up search engines. I tried adding Ask Slashdot and it wouldn't register it.

[Lonestar75]

Will add the underscore issue to our bug queue. BaS has been tested on every browser except (IE), and we have not encountered such a problem. Anyways we have rebuilt about 90% of BuildaSearch for the next release, with tons of new features.  Are there any features you think we should add to BaS? We may be able to add them before the beta 2 release. 

[Lonestar75]

Pretty sure the underscore issue has been fixed with Firefox.  Let me know if it is working correctly.

[Paul Keith]

Well, I don't really know what goes into the algorithm of a search engine but it seems you guys have a better grasp at the search results than both Google Custom Search and Rollyo so I think getting up to par with their features are the stuff I would suggest but again, I'm only saying this not because it's what I need but because they are the differences I noticed.

What Google Custom Search has when I tried it:
=Embed to another site like a blog
=Wildcards (though I'm not satisfied with the end result as a search engine but it does allow for more sites to be chained together)
=Google Marker which allows you to add any site you're on automatically to your custom search.
=Multiple Search Engines

What Rollyo has when I tried it:
=Multiple Search Engines both public and private
=Ability to add a search engine someone else made to your collection
=Still more search sites allowed
=Shows the current engines on the right side for when you want to narrow it down.
=Shows what engine the searches are using on a hover pop-up area

Personally I think you have something here in that you're actually providing the Google of custom search sites. Very simple. Very fast. Easily understandable.

I think speed and stability would be my suggestion-list if I were speaking from a non-marketing point of view. Right now IMO you have enough features that can satisfy a user if the service was just a bit more stable.

Then it'd be easier to suggest features because the core service is working. Also technology hasn't progressed far enough right now it seems to add stuff like categories and other stuff to completely mimic Google so it's really how much these services mature. Right now though at least a stable service would probably attract loyal users of the service.

Edit: Yup, the bug seems to have been fixed. Thanks for the quick feedback and fix.

[Lonestar75]

Beta 2 will bring much more stablity to BuildaSearch.  The new search backbone is already in place, but not accessible to the public as of yet. Custom search engines will be allowed greater customization. Our main focus is to allow anyone to build a custom search in minutes instead of hours and without having to write a single line of code, all while providing a great search service.  You can embed BuildaSearch to a blog or any site.  example here: http://www.buyandselljeans.com

I greatly appreciate your feedback!