topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Monday December 2, 2024, 7:53 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Last post Author Topic: FARR plugin: FarrWebMetaSearch 1.6.0  (Read 88028 times)

phitsc

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 1,198
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.4.2
« Reply #50 on: June 24, 2010, 09:16 AM »
I'd like to be able to specify local files for searchUrl (e.g. for a local copy of the autohotkey script listing page).

Is it possible to do this already and if so, is there a way to do it without specifying absolute paths?

You mean instead of searching in a file downloaded from the web, you want to search in a local file?
Any chance you might be motivated to work on this?

I'd like to use it for things like:

  http://www.stoicjoke...ain.php?Target=WinXP

This alone shouldn't be to hard to add to the current version. I'll check it out.

ewemoa

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 2,922
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.4.2
« Reply #51 on: June 24, 2010, 09:51 AM »
Thanks for your consideration -- looking forward to good news :Thmbsup:

phitsc

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 1,198
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.4.2
« Reply #52 on: July 05, 2010, 10:18 AM »
I'd like to be able to specify local files for searchUrl (e.g. for a local copy of the autohotkey script listing page).

Is it possible to do this already and if so, is there a way to do it without specifying absolute paths?

You mean instead of searching in a file downloaded from the web, you want to search in a local file?
Any chance you might be motivated to work on this?

I'd like to use it for things like:

  http://www.stoicjoke...ain.php?Target=WinXP

Turns out this is already possible with the current version. I did the following:

* Open the above mentioned URL in my Web Browser
* View page source and copy it into an .html file (Save Page As... made some nasty line breaks into the file which would have made the regexing a bit difficult)
* Created a new FarrWMS .conf file that looks like this:

description=Windows XP Keyboard Shortcuts
searchUrl=C:\Temp\Xp Keyboard Shortcuts.htm
resultPattern=<tr>\s*<td [^>]*>([^<]*)</td>\s*<td [^>^]*>([^<]*)</td>
farrCaption=$2
farrGroup=$1
farrPath=$1
isFeed=true

and it works!

Now the question is: does this already solve your problem, or did you have something else / more in mind?

ewemoa

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 2,922
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.4.2
« Reply #53 on: July 05, 2010, 04:27 PM »
Thanks for looking into this :)

I tried to follow the steps you outlined, but for whatever reason (perhaps not being able to follow the "made some nasty line breaks" one appropriately?) didn't succeed.

I did have luck with the following .conf file for accessing the content over the network.  Strangely, after I save it locally and modify the conf file to point to it, it doesn't seem to work here (no results appear):

description=Windows XP Keyboard Shortcuts
searchUrl=http://www.stoicjoker.com/HotKeys/Main.php?Target=WinXP
resultPattern=<tr>[^<]*<td class='black10b' bgcolor='#ededed' align='center' width='150'>([^<]*)</td>[^<]*<td class='black10' bgcolor='#FFFFFF'>&nbsp;([^<]*)</td>[^<]*</tr>
farrCaption=$2
farrGroup=$1
farrPath=$1
isFeed=true

Now the question is: does this already solve your problem, or did you have something else / more in mind?
I didn't succeed in carrying the procedure out for local access so I didn't test whether this procedure would work when specifying a searchUrl which is a non-absolute file path.  Does that work? 

I much prefer not to place absolute file paths into my configuration files -- so for example, if I change the location of my FARR installation I wouldn't have to modify my configuration files to change file path information recorded within.

phitsc

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 1,198
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.4.2
« Reply #54 on: July 06, 2010, 09:46 AM »
I much prefer not to place absolute file paths into my configuration files -- so for example, if I change the location of my FARR installation I wouldn't have to modify my configuration files to change file path information recorded within.
Ah yes, you mentioned that but I forgot. I think you said using FARR user variables would be ok?

ewemoa

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 2,922
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.4.2
« Reply #55 on: July 06, 2010, 12:03 PM »
Ah yes, you mentioned that but I forgot. I think you said using FARR user variables would be ok?
Yes.  For reference, the ideas were:

One idea is to be able to specify a path using FARR's "virtual folder" functionality as mentioned in the "Variable Replacements, Virtual Folder Names, Env Paths" section of the help file.  For example, being able to use %APPDRIVE%, %PLUGINDIR%, and the like as part of the path specification.  For reference, I believe "resolve" described in the "Values Getable by Plugins" section of the FARR help file can help with this.

Another is to be able to apply user variables.  For example, I have something like the following defined in the [Paths] section of my user variables:

  par=%APPDRIVE%\apps

So using:

  %uservar.Paths.par%

as part of a path.

The last idea is for non-absolute paths to be resolved relative the FarrWebMetaSearch plugin (or perhaps searches) directory.

Looking back on the ideas, it seems to me that the FARR user variables idea is the least convenient for users -- I assume this because I don't see much mention of it being used.  The easiest in terms of writing .conf files seems to me to be the "resolve relative to" idea.  Do you have an opinion about this?

Thanks for your continued attention and consideration regarding this, btw :)

phitsc

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 1,198
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.4.2
« Reply #56 on: July 09, 2010, 03:49 AM »
Thanks for your continued attention and consideration regarding this, btw :)
I just take you as a shining example :)

I've boldly uploaded version 1.6.0 so you can use DCUpdater to update to the current version (if DCUpdater works with your setup).

Searching local files should now work with both relative paths (relative to where FindAndRunRobot.exe is) and FARR virtual folder variables. I've not implemented user variables.

from the help file:

searchUrl
This can be either a local file or a web page. For web pages, start with http://, for local files, either use an absolute path (e.g. c:\temp\somefile.txt), a relative path (e.g. ..\..\temp\somefile.txt) which will be resolved relative to where FindAndRunRobot.exe is, or use any of the FARR virtual folder variables (.e.g %MYDOCUMENTS%\somefile.txt).

Let me know if it works or not. If it doesn't, enable the log file with wms !Enable Logging. Then check the log file with wms !Open log file.

ewemoa

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 2,922
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.4.2
« Reply #57 on: July 09, 2010, 04:08 AM »
I've boldly uploaded version 1.6.0 so you can use DCUpdater to update to the current version (if DCUpdater works with your setup).
...and I've boldly updated :)

Searching local files should now work with both relative paths (relative to where FindAndRunRobot.exe is) and FARR virtual folder variables. I've not implemented user variables.
Both implemented options seem to work here  :Thmbsup:

Out of curiosity, why did you decide to make the relative path relative to where the .exe is -- as compared with where the FarrWebMetaSearch plugin lives?

Let me know if it works or not. If it doesn't, enable the log file with wms !Enable Logging. Then check the log file with wms !Open log file.
Cool features  :up:

Thanks for the local file feature!

phitsc

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 1,198
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.4.2
« Reply #58 on: July 09, 2010, 04:42 AM »
Out of curiosity, why did you decide to make the relative path relative to where the .exe is -- as compared with where the FarrWebMetaSearch plugin lives?
Öhhhm, ähhhm, well ... to be honest, I didn't even think about that. Just took whatever was easiest. The difference just being passing NULL or some variable to a Win32 API function. If relative from where the plugin lives would be more convenient to you, I can certainly change that :)

Thanks for the local file feature!
You are very welcome!

ewemoa

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 2,922
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #59 on: July 09, 2010, 06:25 AM »
If relative from where the plugin lives would be more convenient to you, I can certainly change that :)
My current thoughts about this are that a common case would be to store local files to be parsed in Plugins\FarrWebMetaSearch\searches so that those files could live alongside their corresponding .conf files.  Consequently, having the "base" directory from which to resolve relative references be the Plugins\FarrWebMetaSearch\searches directory seems convenient to me.

BTW, I don't know if I mis-tested or I'm doing something wrong but I'm not finding the following sort of thing to be working now:

searchUrl=%PLUGINDIR%\searches\ahkscriptlisting.html

I get a popup informing me about not finding a file at: <driveletter>:\searches\ahkscriptlisting.html

For reference, the following seems to work:

searchUrl=%FARRDIR%\Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

and the following too:

Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

May be I've messed up somewhere...

lanux128

  • Global Moderator
  • Joined in 2005
  • *****
  • Posts: 6,277
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #60 on: July 15, 2010, 09:57 PM »
i made a .conf to search sites like Steam, D2D and Gamersgate. they are video games retailers who specialize in digital distribution. you may refer to the screenshot below to get an idea of how it works.

Usage info:
Spoiler
• Download and install the FarrWebMetaSearch plugin if you don't have it already. (link)
• Then unzip steam_dd.zip into the searches sub-folder within FarrWebMetaSearch folder (may look like this C:\Program Files\FindAndRunRobot\Plugins\FarrWebMetaSearch\searches\)
• Restart FARR.

To use it, type:
wms steam [search]
wms d2d [search]
wms gamersgate [search]

Hint: you can type the 1st word then press 'Tab' to complete.



FindAndRunRobot.exe_12_07_2010_000.pngFARR plugin: FarrWebMetaSearch 1.6.0


btw, i'd like to thank [user]Perry Mowbray[/user] for his help in making these .conf and also to [user]phitsc[/user] for this awesome plugin..



edit: fixed the regex for games that are on sale.
« Last Edit: July 22, 2010, 12:10 AM by lanux128 »

phitsc

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 1,198
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #61 on: July 16, 2010, 06:36 AM »
And thank you for your contribution!  :Thmbsup:

phitsc

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 1,198
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #62 on: July 16, 2010, 06:49 AM »
BTW, I don't know if I mis-tested or I'm doing something wrong but I'm not finding the following sort of thing to be working now:

searchUrl=%PLUGINDIR%\searches\ahkscriptlisting.html

I get a popup informing me about not finding a file at: <driveletter>:\searches\ahkscriptlisting.html

For reference, the following seems to work:

searchUrl=%FARRDIR%\Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

and the following too:

Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

May be I've messed up somewhere...

Yes, I confirm this. It seems like FARR does not replace the %PLUGINDIR% variable correctly.

Lashiec

  • Member
  • Joined in 2006
  • **
  • Posts: 2,374
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #63 on: July 16, 2010, 10:25 AM »
How come I managed to miss this plugin until now? :-[

Thanks to lanux for bringing it to my attention, and for the Steam alias, and to phitsc for all his hard work

* Lashiec pokes mouser to update DcUpdater plugin list

ewemoa

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 2,922
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #64 on: July 16, 2010, 11:40 PM »
Yes, I confirm this. It seems like FARR does not replace the %PLUGINDIR% variable correctly.
It is strange to me because IIRC in some of my plugins I use "resolve:%PLUGINDIR%" via callback_get_strvalue successfully.  I wonder what's going on...

phitsc

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 1,198
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #65 on: July 17, 2010, 04:23 PM »
It is strange to me because IIRC in some of my plugins I use "resolve:%PLUGINDIR%" via callback_get_strvalue successfully.  I wonder what's going on...

I'm using "resolvefile:". Will check if it makes a difference.

mouser

  • First Author
  • Administrator
  • Joined in 2005
  • *****
  • Posts: 40,913
    • View Profile
    • Mouser's Software Zone on DonationCoder.com
    • Read more about this member.
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #66 on: July 17, 2010, 04:44 PM »
i should be able to fix this if i understand where it is failing to resolve %PLUGINDIR%

Hastur

  • Participant
  • Joined in 2007
  • *
  • default avatar
  • Posts: 2
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #67 on: July 20, 2010, 11:44 AM »
There's a minor error in the regex for the timeanddate searches.

<td class="r"> should be <td class=r> in resultPattern.


Also, if you enter "wms timeanddate +sun http:", FARR will crash with a MSVC++ runtime library runtime error.  :(

phitsc

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 1,198
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #68 on: July 20, 2010, 02:07 PM »
Thanks Hastur for the report and a warm welcome to the donationcoder forums :)

What you found is one of FarrWMS's weaknesses. Unfortunately, when a web site changes the format of its pages, regexes can get "out of date". The same happened to the MSDN search a while back (and I'm still working on fixing it). TimeAndDate is one of the searches Perry made. I'll ask him if he can check it out.

I'll look into the runtime error, if I can reproduce it (will try tomorrow).

putnum

  • Participant
  • Joined in 2010
  • *
  • default avatar
  • Posts: 68
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #69 on: October 19, 2010, 12:24 PM »
I love this plugin but I am having an issue with onelook. When I am using it it gets stuck on "plugins are searching" and it stays that way for prob a half a minute. Does anybody know why?

Alien282

  • Participant
  • Joined in 2010
  • *
  • default avatar
  • Posts: 6
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #70 on: October 22, 2010, 03:58 PM »
Hello,

What you found is one of FarrWMS's weaknesses. Unfortunately, when a web site changes the format of its pages, regexes can get "out of date".

Absolutely. I see that the Google thingy is not working anymore, at least for the calculator. :) Here is an update:

Code: Javascript [Select]
  1. resultPattern=(?:<h3 class="?r"?>\s*<a href="(.+?)".*?>(.*?)</a>.*?<div.*?>\s*(?:<div.*?/div>|(<span class="?f"?>.*?</span>.*?)<br>)?(.*?)<br>|<div id="?topstuff"?>.*?<img.*?calculator.*?>.*?<h2.*?>(.*?)</h2>)
  2. farrCaption=$5$2 $3
  3. farrGroup=$5$4
  4. farrPath=$5$1

It should be a bit faster too and shows PDF file results. Enjoy. :)

phitsc

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 1,198
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #71 on: October 22, 2010, 04:31 PM »
Thanks guys for the feedback. I should maybe go over all the available searches and check if others are not working properly any more either. I'll definitely integrate your suggested changes so thanks for that.

putnum

  • Participant
  • Joined in 2010
  • *
  • default avatar
  • Posts: 68
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #72 on: October 24, 2010, 01:51 PM »
Hi phitsc any update on this by chance?

phitsc

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 1,198
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #73 on: October 27, 2010, 03:16 AM »
Not yet I'm afraid. With <tt>wms !Enable logging</tt> you can enable logging. I don't think it will help with tracking down timing problems though. I'll see if I can improve that.

ewemoa

  • Honorary Member
  • Joined in 2008
  • **
  • Posts: 2,922
    • View Profile
    • Donate to Member
Re: FARR plugin: FarrWebMetaSearch 1.6.0
« Reply #74 on: November 03, 2011, 08:25 PM »
I have come across a site that appears to return a JSON array as a result (with mime type plain/text).  Is this format of data something FarrWebMetaSearch can handle?