Author Topic: FARR plugin: FarrWebMetaSearch 1.6.0 (Read 88187 times)

phitsc · « **Reply #50 on:** June 24, 2010, 09:16 AM »

I'd like to be able to specify local files for searchUrl (e.g. for a local copy of the autohotkey script listing page).

Is it possible to do this already and if so, is there a way to do it without specifying absolute paths?

-ewemoa (February 09, 2010, 05:29 AM)
You mean instead of searching in a file downloaded from the web, you want to search in a local file?
-phitsc (February 09, 2010, 05:37 AM)
Any chance you might be motivated to work on this?

I'd like to use it for things like:

http://www.stoicjoke...ain.php?Target=WinXP

-ewemoa (June 23, 2010, 03:48 PM)

This alone shouldn't be to hard to add to the current version. I'll check it out.

ewemoa · « **Reply #51 on:** June 24, 2010, 09:51 AM »

Thanks for your consideration -- looking forward to good news

phitsc · « **Reply #52 on:** July 05, 2010, 10:18 AM »

I'd like to be able to specify local files for searchUrl (e.g. for a local copy of the autohotkey script listing page).

Is it possible to do this already and if so, is there a way to do it without specifying absolute paths?

-ewemoa (February 09, 2010, 05:29 AM)
You mean instead of searching in a file downloaded from the web, you want to search in a local file?
-phitsc (February 09, 2010, 05:37 AM)
Any chance you might be motivated to work on this?

I'd like to use it for things like:

http://www.stoicjoke...ain.php?Target=WinXP

-ewemoa (June 23, 2010, 03:48 PM)

Turns out this is already possible with the current version. I did the following:

* Open the above mentioned URL in my Web Browser
* View page source and copy it into an .html file (Save Page As... made some nasty line breaks into the file which would have made the regexing a bit difficult)
* Created a new FarrWMS .conf file that looks like this:

[Select]

description=Windows XP Keyboard Shortcuts
searchUrl=C:\Temp\Xp Keyboard Shortcuts.htm
resultPattern=<tr>\s*<td [^>]*>([^<]*)</td>\s*<td [^>^]*>([^<]*)</td>
farrCaption=$2
farrGroup=$1
farrPath=$1
isFeed=true

and it works!

Now the question is: does this already solve your problem, or did you have something else / more in mind?

ewemoa · « **Reply #53 on:** July 05, 2010, 04:27 PM »

Thanks for looking into this

I tried to follow the steps you outlined, but for whatever reason (perhaps not being able to follow the "made some nasty line breaks" one appropriately?) didn't succeed.

I did have luck with the following .conf file for accessing the content over the network. Strangely, after I save it locally and modify the conf file to point to it, it doesn't seem to work here (no results appear):

[Select]

description=Windows XP Keyboard Shortcuts
searchUrl=http://www.stoicjoker.com/HotKeys/Main.php?Target=WinXP
resultPattern=<tr>[^<]*<td class='black10b' bgcolor='#ededed' align='center' width='150'>([^<]*)</td>[^<]*<td class='black10' bgcolor='#FFFFFF'> ([^<]*)</td>[^<]*</tr>
farrCaption=$2
farrGroup=$1
farrPath=$1
isFeed=true

Now the question is: does this already solve your problem, or did you have something else / more in mind?

I didn't succeed in carrying the procedure out for local access so I didn't test whether this procedure would work when specifying a searchUrl which is a non-absolute file path. Does that work?

I much prefer not to place absolute file paths into my configuration files -- so for example, if I change the location of my FARR installation I wouldn't have to modify my configuration files to change file path information recorded within.

phitsc · « **Reply #54 on:** July 06, 2010, 09:46 AM »

I much prefer not to place absolute file paths into my configuration files -- so for example, if I change the location of my FARR installation I wouldn't have to modify my configuration files to change file path information recorded within.
-ewemoa (July 05, 2010, 04:27 PM)

Ah yes, you mentioned that but I forgot. I think you said using FARR user variables would be ok?

ewemoa · « **Reply #55 on:** July 06, 2010, 12:03 PM »

Ah yes, you mentioned that but I forgot. I think you said using FARR user variables would be ok?
-phitsc (July 06, 2010, 09:46 AM)

Yes. For reference, the ideas were:

One idea is to be able to specify a path using FARR's "virtual folder" functionality as mentioned in the "Variable Replacements, Virtual Folder Names, Env Paths" section of the help file. For example, being able to use %APPDRIVE%, %PLUGINDIR%, and the like as part of the path specification. For reference, I believe "resolve" described in the "Values Getable by Plugins" section of the FARR help file can help with this.

Another is to be able to apply user variables. For example, I have something like the following defined in the [Paths] section of my user variables:

par=%APPDRIVE%\apps

So using:

%uservar.Paths.par%

as part of a path.

The last idea is for non-absolute paths to be resolved relative the FarrWebMetaSearch plugin (or perhaps searches) directory.

Looking back on the ideas, it seems to me that the FARR user variables idea is the least convenient for users -- I assume this because I don't see much mention of it being used. The easiest in terms of writing .conf files seems to me to be the "resolve relative to" idea. Do you have an opinion about this?

Thanks for your continued attention and consideration regarding this, btw

phitsc · « **Reply #56 on:** July 09, 2010, 03:49 AM »

Thanks for your continued attention and consideration regarding this, btw
-ewemoa (July 06, 2010, 12:03 PM)

I just take you as a shining example

I've boldly uploaded version 1.6.0 so you can use DCUpdater to update to the current version (if DCUpdater works with your setup).

Searching local files should now work with both relative paths (relative to where FindAndRunRobot.exe is) and FARR virtual folder variables. I've not implemented user variables.

from the help file:

searchUrl
This can be either a local file or a web page. For web pages, start with http://, for local files, either use an absolute path (e.g. c:\temp\somefile.txt), a relative path (e.g. ..\..\temp\somefile.txt) which will be resolved relative to where FindAndRunRobot.exe is, or use any of the FARR virtual folder variables (.e.g %MYDOCUMENTS%\somefile.txt).

Let me know if it works or not. If it doesn't, enable the log file with wms !Enable Logging. Then check the log file with wms !Open log file.

ewemoa · « **Reply #57 on:** July 09, 2010, 04:08 AM »

I've boldly uploaded version 1.6.0 so you can use DCUpdater to update to the current version (if DCUpdater works with your setup).
-ewemoa (July 06, 2010, 12:03 PM)
-phitsc (July 09, 2010, 03:49 AM)

...and I've boldly updated

Searching local files should now work with both relative paths (relative to where FindAndRunRobot.exe is) and FARR virtual folder variables. I've not implemented user variables.

Both implemented options seem to work here

Out of curiosity, why did you decide to make the relative path relative to where the .exe is -- as compared with where the FarrWebMetaSearch plugin lives?

Let me know if it works or not. If it doesn't, enable the log file with wms !Enable Logging. Then check the log file with wms !Open log file.

Cool features

Thanks for the local file feature!

phitsc · « **Reply #58 on:** July 09, 2010, 04:42 AM »

Out of curiosity, why did you decide to make the relative path relative to where the .exe is -- as compared with where the FarrWebMetaSearch plugin lives?
-ewemoa (July 09, 2010, 04:08 AM)

Öhhhm, ähhhm, well ... to be honest, I didn't even think about that. Just took whatever was easiest. The difference just being passing NULL or some variable to a Win32 API function. If relative from where the plugin lives would be more convenient to you, I can certainly change that

Thanks for the local file feature!
-ewemoa (July 09, 2010, 04:08 AM)

You are very welcome!

ewemoa · « **Reply #59 on:** July 09, 2010, 06:25 AM »

If relative from where the plugin lives would be more convenient to you, I can certainly change that
-phitsc (July 09, 2010, 04:42 AM)

My current thoughts about this are that a common case would be to store local files to be parsed in Plugins\FarrWebMetaSearch\searches so that those files could live alongside their corresponding .conf files. Consequently, having the "base" directory from which to resolve relative references be the Plugins\FarrWebMetaSearch\searches directory seems convenient to me.

BTW, I don't know if I mis-tested or I'm doing something wrong but I'm not finding the following sort of thing to be working now:

searchUrl=%PLUGINDIR%\searches\ahkscriptlisting.html

I get a popup informing me about not finding a file at: <driveletter>:\searches\ahkscriptlisting.html

For reference, the following seems to work:

searchUrl=%FARRDIR%\Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

and the following too:

Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

May be I've messed up somewhere...

lanux128 · « **Reply #60 on:** July 15, 2010, 09:57 PM »

i made a .conf to search sites like Steam, D2D and Gamersgate. they are video games retailers who specialize in digital distribution. you may refer to the screenshot below to get an idea of how it works.

Usage info:

Spoiler

• Download and install the FarrWebMetaSearch plugin if you don't have it already. (link)
• Then unzip steam_dd.zip into the searches sub-folder within FarrWebMetaSearch folder (may look like this C:\Program Files\FindAndRunRobot\Plugins\FarrWebMetaSearch\searches\)
• Restart FARR.

To use it, type:
wms steam [search]
wms d2d [search]
wms gamersgate [search]

Hint: you can type the 1st word then press 'Tab' to complete.

FARR plugin: FarrWebMetaSearch 1.6.0

btw, i'd like to thank [user]Perry Mowbray[/user] for his help in making these .conf and also to [user]phitsc[/user] for this awesome plugin..

edit: fixed the regex for games that are on sale.

phitsc · « **Reply #61 on:** July 16, 2010, 06:36 AM »

And thank you for your contribution!

phitsc · « **Reply #62 on:** July 16, 2010, 06:49 AM »

BTW, I don't know if I mis-tested or I'm doing something wrong but I'm not finding the following sort of thing to be working now:

searchUrl=%PLUGINDIR%\searches\ahkscriptlisting.html

I get a popup informing me about not finding a file at: <driveletter>:\searches\ahkscriptlisting.html

For reference, the following seems to work:

searchUrl=%FARRDIR%\Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

and the following too:

Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

May be I've messed up somewhere...
-ewemoa (July 09, 2010, 06:25 AM)

Yes, I confirm this. It seems like FARR does not replace the %PLUGINDIR% variable correctly.

Lashiec · « **Reply #63 on:** July 16, 2010, 10:25 AM »

How come I managed to miss this plugin until now?

Thanks to lanux for bringing it to my attention, and for the Steam alias, and to phitsc for all his hard work

* Lashiec pokes mouser to update DcUpdater plugin list

ewemoa · « **Reply #64 on:** July 16, 2010, 11:40 PM »

Yes, I confirm this. It seems like FARR does not replace the %PLUGINDIR% variable correctly.
-phitsc (July 16, 2010, 06:49 AM)

It is strange to me because IIRC in some of my plugins I use "resolve:%PLUGINDIR%" via callback_get_strvalue successfully. I wonder what's going on...

phitsc · « **Reply #65 on:** July 17, 2010, 04:23 PM »

It is strange to me because IIRC in some of my plugins I use "resolve:%PLUGINDIR%" via callback_get_strvalue successfully. I wonder what's going on...
-ewemoa (July 16, 2010, 11:40 PM)

I'm using "resolvefile:". Will check if it makes a difference.

mouser · « **Reply #66 on:** July 17, 2010, 04:44 PM »

i should be able to fix this if i understand where it is failing to resolve %PLUGINDIR%

Hastur · « **Reply #67 on:** July 20, 2010, 11:44 AM »

There's a minor error in the regex for the timeanddate searches.

<td class="r"> should be <td class=r> in resultPattern.

Also, if you enter "wms timeanddate +sun http:", FARR will crash with a MSVC++ runtime library runtime error.

phitsc · « **Reply #68 on:** July 20, 2010, 02:07 PM »

Thanks Hastur for the report and a warm welcome to the donationcoder forums

What you found is one of FarrWMS's weaknesses. Unfortunately, when a web site changes the format of its pages, regexes can get "out of date". The same happened to the MSDN search a while back (and I'm still working on fixing it). TimeAndDate is one of the searches Perry made. I'll ask him if he can check it out.

I'll look into the runtime error, if I can reproduce it (will try tomorrow).

putnum · « **Reply #69 on:** October 19, 2010, 12:24 PM »

I love this plugin but I am having an issue with onelook. When I am using it it gets stuck on "plugins are searching" and it stays that way for prob a half a minute. Does anybody know why?

Alien282 · « **Reply #70 on:** October 22, 2010, 03:58 PM »

Hello,

What you found is one of FarrWMS's weaknesses. Unfortunately, when a web site changes the format of its pages, regexes can get "out of date".
-phitsc (July 20, 2010, 02:07 PM)

Absolutely. I see that the Google thingy is not working anymore, at least for the calculator.

Here is an update:

Code: Javascript [Select]

resultPattern=(?:<h3 class="?r"?>\s*<a href="(.+?)".*?>(.*?)</a>.*?<div.*?>\s*(?:<div.*?/div>|(<span class="?f"?>.*?</span>.*?)<br>)?(.*?)<br>|<div id="?topstuff"?>.*?<img.*?calculator.*?>.*?<h2.*?>(.*?)</h2>)
farrCaption=$5$2 $3
farrGroup=$5$4
farrPath=$5$1

It should be a bit faster too and shows PDF file results. Enjoy.

phitsc · « **Reply #71 on:** October 22, 2010, 04:31 PM »

Thanks guys for the feedback. I should maybe go over all the available searches and check if others are not working properly any more either. I'll definitely integrate your suggested changes so thanks for that.

putnum · « **Reply #72 on:** October 24, 2010, 01:51 PM »

Hi phitsc any update on this by chance?

phitsc · « **Reply #73 on:** October 27, 2010, 03:16 AM »

Not yet I'm afraid. With <tt>wms !Enable logging</tt> you can enable logging. I don't think it will help with tracking down timing problems though. I'll see if I can improve that.

ewemoa · « **Reply #74 on:** November 03, 2011, 08:25 PM »

I have come across a site that appears to return a JSON array as a result (with mime type plain/text). Is this format of data something FarrWebMetaSearch can handle?