Welcome Guest.   Make a donation to an author on the site April 24, 2014, 01:46:11 PM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
The N.A.N.Y. Challenge 2013! Download dozens of custom programs!
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1] 2 3 Next   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: NANY 2010 Final Release: FarrWebMetaSearch  (Read 25261 times)
phitsc
Honorary Member
**
Posts: 918



see users location on a map View Profile Give some DonationCredits to this forum member
« on: December 11, 2009, 09:51:25 AM »

NANY 2010 Entry Information

Application Name FarrWebMetaSearch
Version 0.9.1
Short Description A FARR plugin used to query results from a web page and display them in list form.
Supported OSes MS Windows
Web Page http://farrwms.objecttechnology.com
Download Link download
System Requirements
Version History
  • 0.9.1 - preliminary - use FARR's launching functionality to open URLs
  • 0.9.0 - preliminary - first release
Author phitsc
Credits Thanks to hamradio for designing the plugin icon!
Screencast http://donationcoder.com/...search/webmetasearch.html
YouTube: http://www.youtube.com/watch?v=pxKcvQ-uHjQ


Description
FarrWebMetaSearch is another generic web retrieval FARR plugin. Unlike ewemoas Webgrab this plugin will show results in list mode. It is therefore targeted at web sites delivering list-like results.

Configuration and usage of the plugin will be as follows:
  • searches are configured via a text configuration file, one for each 'search'. the file will specify search url and a regular expression to capture the result.
  • search terms will be specified through the FARR input box

See implementation notes further down for further clarification.

Features
It just does what it says in the description.

Planned Features
  • Fix icon copyright problem
  • Fix launching problem
  • Add proper help file
  • Add a way for users to contribute / get searches
  • Add logging/notification in case a configuration files is invalid
  • Improve usage for news feeds
  • Make original search string available as a variable
  • Add many more searches cheesy
  • Either make plugin asynchronious or implement some way to delay searching
  • Add a description for each search to xml files and FARR searches list
  • Implement filtering of searches
  • Implement auto-completion of searches with Enter key (well, maybe)
  • Add a way to specify search options (for web sites that support it)
  • Make parsing of searches files more robust

Screencast
<a href="http://www.youtube.com/watch?v=pxKcvQ-uHjQ" target="_blank">http://www.youtube.com/watch?v=pxKcvQ-uHjQ</a>

Screenshots

FarrWebMetaSearch Web Site
MSDN
CDDB

and a teaser screencast.

Implementation Notes

FarrWebMetaSearch reads all configuration files in its searches subdirectory when it's being loaded. When the plugin alias (currently wms) is entered, it will list all configured searches. To use a search one would then enter, e.g.:

wms msdn PathAppend

to find MSDN entries for the PathAppend API function.

Search files currently look like this (this is the cddb.conf file):

description=Microsoft Developer Network
searchUrl=http://social.msdn.microsoft.com/Search/en-GB?query=
resultPattern=<a onmousedown=\".*\" href=\".*\">(.*)</a>\r\n<br/>\r\n<div class=\"ResultDescription\">(.*)</div>\r\n<div class=\"ResultUrl\">(.*)</div>
farrCaption=$1
farrGroup=$2
farrPath=$3

The regex sub-expressions can (should, actually) be used to form the results.
« Last Edit: January 08, 2010, 05:21:52 PM by phitsc » Logged

mouser
First Author
Administrator
*****
Posts: 32,700



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #1 on: December 11, 2009, 10:50:13 AM »

Sounds wonderful!
Reminds me of this FARR plugin as well which deserves more attention and users:
http://www.donationcoder....m/index.php?topic=16006.0
Logged
phitsc
Honorary Member
**
Posts: 918



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #2 on: December 13, 2009, 03:45:43 PM »

I've added some screen shots.
Logged

mouser
First Author
Administrator
*****
Posts: 32,700



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #3 on: December 13, 2009, 03:51:44 PM »

whoa now this is looking good.
Logged
phitsc
Honorary Member
**
Posts: 918



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #4 on: December 13, 2009, 04:52:18 PM »

updated implementation notes and added a todo list
Logged

mouser
First Author
Administrator
*****
Posts: 32,700



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #5 on: December 13, 2009, 07:06:44 PM »

it doesnt show results live as we type does it?
Logged
phitsc
Honorary Member
**
Posts: 918



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #6 on: December 14, 2009, 02:42:54 AM »

it doesnt show results live as we type does it?
It's meant to. But fetching something from the web obviously takes time. So currently the plugin is best suited to touch typists Wink
Logged

mouser
First Author
Administrator
*****
Posts: 32,700



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #7 on: December 14, 2009, 02:56:18 AM »

NICE  thumbs up thumbs up thumbs up
Logged
CWuestefeld
Supporting Member
**
Posts: 911



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #8 on: December 15, 2009, 11:30:28 AM »

Cool.

Request: put in a search file for IMDB.
Logged



phitsc
Honorary Member
**
Posts: 918



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #9 on: December 15, 2009, 03:27:21 PM »

I've implemented the most important features now, especially asynchronious searching. I'm going to try and add some more searches now with focus on stability. I'm going to start with IMDB then.
Logged

mouser
First Author
Administrator
*****
Posts: 32,700



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #10 on: December 15, 2009, 04:06:56 PM »

imdb live search is already provided in google plus addon by the way, not that it's a bad idea to add it.
now adding a live search of amazon and amazon books.. DROOOOOOL...... Kiss Kiss Kiss Kiss Kiss i'd give my left arm for that.
Logged
TucknDar
Charter Member
***
Posts: 1,093


Advanced coder of Nowt

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #11 on: December 15, 2009, 04:14:58 PM »

Really looking forward to this one, looks incredibly useful! Thmbsup
Amazon, play.com, eBay, yellow pages...
Logged
mouser
First Author
Administrator
*****
Posts: 32,700



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #12 on: December 15, 2009, 05:02:16 PM »

ebay and amazon, that's like the holy grail of instant search smiley
Logged
phitsc
Honorary Member
**
Posts: 918



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #13 on: December 16, 2009, 02:49:33 AM »

Amazon will be next on my list. I've ordered myself a Kindle for Christmas cheesy
Logged

phitsc
Honorary Member
**
Posts: 918



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #14 on: December 17, 2009, 02:10:51 PM »

I've changed the configuration files from xml to ini-style (see implementation notes in first post). Makes it much easier to handle the (already complex) regular expressions, as you don't have to encode < and > etc.. And then xml would be overkill for this anyway.

Capturing IMDB's output turned out to be a hard nut to crack. I'm afraid I can't offer anything satisfactory.
Logged

mouser
First Author
Administrator
*****
Posts: 32,700



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #15 on: December 17, 2009, 03:07:19 PM »

live search of some of the dictionary website would be awesome as well, while we are building a wishlist smiley

(since imdb is already supported as live search in google plus addon for farr it's not high on my priority list)
Logged
Perry Mowbray
N.A.N.Y. Organizer
Moderator
*****
Posts: 1,795



Thoughtful Scribbles

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #16 on: December 17, 2009, 07:02:09 PM »

Dictonary sites would have to be my most used  cheesy
Logged

TucknDar
Charter Member
***
Posts: 1,093


Advanced coder of Nowt

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #17 on: December 18, 2009, 01:51:22 AM »

Dictonary sites would have to be my most used  cheesy
Actually, you missed an 'i' there, it's 'Dictionary'




 Grin
Logged
Perry Mowbray
N.A.N.Y. Organizer
Moderator
*****
Posts: 1,795



Thoughtful Scribbles

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #18 on: December 18, 2009, 01:59:47 AM »

Dictonary sites would have to be my most used  cheesy
Actually, you missed an 'i' there, it's 'Dictionary'

 Grin

Cool: you know I staired at it for ages but I couldn't see it!! I often don't have an eye for detal...
Logged

TucknDar
Charter Member
***
Posts: 1,093


Advanced coder of Nowt

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #19 on: December 18, 2009, 02:02:57 AM »

Dictonary sites would have to be my most used  cheesy
Actually, you missed an 'i' there, it's 'Dictionary'

 Grin

Cool: you know I staired at it for ages but I couldn't see it!! I often don't have an eye for detal...
tongue Wink
Logged
cranioscopical
Friend of the Site
Supporting Member
**
Posts: 4,158



see users location on a map View Profile Read user's biography. Give some DonationCredits to this forum member
« Reply #20 on: December 18, 2009, 10:35:32 AM »

I staired at it for ages but I couldn't see it!!
Tread softly or someone will try to take a riser out of you  ohmy
Logged

Chris
phitsc
Honorary Member
**
Posts: 918



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #21 on: December 21, 2009, 02:14:28 PM »

I've recorded a short teaser screen cast.
Logged

TucknDar
Charter Member
***
Posts: 1,093


Advanced coder of Nowt

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #22 on: December 21, 2009, 02:26:38 PM »

I've recorded a short teaser screen cast.
greenclp
Logged
mouser
First Author
Administrator
*****
Posts: 32,700



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #23 on: December 21, 2009, 02:46:55 PM »

nice!  Thmbsup Thmbsup Thmbsup
Logged
phitsc
Honorary Member
**
Posts: 918



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #24 on: December 23, 2009, 08:19:15 AM »

just realised FarrWebMetaSearch could be (mis)used to read news feeds. because they are xml they are actually simpler to split up with regular expressions than 'normal' web pages.

here's an example:


description=Tages Anzeiger Front
searchUrl=http://www.tagesanzeiger.ch/rss.html?
resultPattern=<item>\s*<id>\d*</id>\s*<title>(.*)</title>\s*<description><!\[CDATA\[(.*)\]\]></description>\s*<category>.*</category>\s*<link>(.*)</link>\s*<guid.*</guid>\s*<pubDate>(.*)</pubDate>\s*</item>
farrCaption=$1 ($4)
farrGroup=$2
farrPath=$3


will give you this:

Logged

Pages: [1] 2 3 Next   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.056s | Server load: 0.02 ]