Welcome Guest.   Make a donation to an author on the site September 02, 2014, 11:48:52 AM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
Learn about the DonationCoder.com microdonation system (DonationCredits).
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: Prev 1 2 [3] 4 Next   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: FARR plugin: FarrWebMetaSearch 1.6.0  (Read 26916 times)
phitsc
Honorary Member
**
Posts: 989



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #50 on: June 24, 2010, 09:16:26 AM »

I'd like to be able to specify local files for searchUrl (e.g. for a local copy of the autohotkey script listing page).

Is it possible to do this already and if so, is there a way to do it without specifying absolute paths?

You mean instead of searching in a file downloaded from the web, you want to search in a local file?
Any chance you might be motivated to work on this?

I'd like to use it for things like:

  http://www.stoicjoker.com...eys/Main.php?Target=WinXP

This alone shouldn't be to hard to add to the current version. I'll check it out.
Logged

ewemoa
Honorary Member
**
Posts: 2,415



View Profile Give some DonationCredits to this forum member
« Reply #51 on: June 24, 2010, 09:51:04 AM »

Thanks for your consideration -- looking forward to good news Thmbsup
Logged
phitsc
Honorary Member
**
Posts: 989



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #52 on: July 05, 2010, 10:18:46 AM »

I'd like to be able to specify local files for searchUrl (e.g. for a local copy of the autohotkey script listing page).

Is it possible to do this already and if so, is there a way to do it without specifying absolute paths?

You mean instead of searching in a file downloaded from the web, you want to search in a local file?
Any chance you might be motivated to work on this?

I'd like to use it for things like:

  http://www.stoicjoker.com...eys/Main.php?Target=WinXP

Turns out this is already possible with the current version. I did the following:

* Open the above mentioned URL in my Web Browser
* View page source and copy it into an .html file (Save Page As... made some nasty line breaks into the file which would have made the regexing a bit difficult)
* Created a new FarrWMS .conf file that looks like this:

[copy or print]
description=Windows XP Keyboard Shortcuts
searchUrl=C:\Temp\Xp Keyboard Shortcuts.htm
resultPattern=<tr>\s*<td [^>]*>([^<]*)</td>\s*<td [^>^]*>([^<]*)</td>
farrCaption=$2
farrGroup=$1
farrPath=$1
isFeed=true

and it works!

Now the question is: does this already solve your problem, or did you have something else / more in mind?
Logged

ewemoa
Honorary Member
**
Posts: 2,415



View Profile Give some DonationCredits to this forum member
« Reply #53 on: July 05, 2010, 04:27:43 PM »

Thanks for looking into this smiley

I tried to follow the steps you outlined, but for whatever reason (perhaps not being able to follow the "made some nasty line breaks" one appropriately?) didn't succeed.

I did have luck with the following .conf file for accessing the content over the network.  Strangely, after I save it locally and modify the conf file to point to it, it doesn't seem to work here (no results appear):

[copy or print]
description=Windows XP Keyboard Shortcuts
searchUrl=http://www.stoicjoker.com/HotKeys/Main.php?Target=WinXP
resultPattern=<tr>[^<]*<td class='black10b' bgcolor='#ededed' align='center' width='150'>([^<]*)</td>[^<]*<td class='black10' bgcolor='#FFFFFF'>&nbsp;([^<]*)</td>[^<]*</tr>
farrCaption=$2
farrGroup=$1
farrPath=$1
isFeed=true

Quote
Now the question is: does this already solve your problem, or did you have something else / more in mind?
I didn't succeed in carrying the procedure out for local access so I didn't test whether this procedure would work when specifying a searchUrl which is a non-absolute file path.  Does that work? 

I much prefer not to place absolute file paths into my configuration files -- so for example, if I change the location of my FARR installation I wouldn't have to modify my configuration files to change file path information recorded within.
Logged
phitsc
Honorary Member
**
Posts: 989



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #54 on: July 06, 2010, 09:46:35 AM »

I much prefer not to place absolute file paths into my configuration files -- so for example, if I change the location of my FARR installation I wouldn't have to modify my configuration files to change file path information recorded within.
Ah yes, you mentioned that but I forgot. I think you said using FARR user variables would be ok?
Logged

ewemoa
Honorary Member
**
Posts: 2,415



View Profile Give some DonationCredits to this forum member
« Reply #55 on: July 06, 2010, 12:03:24 PM »

Ah yes, you mentioned that but I forgot. I think you said using FARR user variables would be ok?
Yes.  For reference, the ideas were:

Quote
One idea is to be able to specify a path using FARR's "virtual folder" functionality as mentioned in the "Variable Replacements, Virtual Folder Names, Env Paths" section of the help file.  For example, being able to use %APPDRIVE%, %PLUGINDIR%, and the like as part of the path specification.  For reference, I believe "resolve" described in the "Values Getable by Plugins" section of the FARR help file can help with this.

Quote
Another is to be able to apply user variables.  For example, I have something like the following defined in the [Paths] section of my user variables:

  par=%APPDRIVE%\apps

So using:

  %uservar.Paths.par%

as part of a path.

Quote
The last idea is for non-absolute paths to be resolved relative the FarrWebMetaSearch plugin (or perhaps searches) directory.

Looking back on the ideas, it seems to me that the FARR user variables idea is the least convenient for users -- I assume this because I don't see much mention of it being used.  The easiest in terms of writing .conf files seems to me to be the "resolve relative to" idea.  Do you have an opinion about this?

Thanks for your continued attention and consideration regarding this, btw smiley
Logged
phitsc
Honorary Member
**
Posts: 989



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #56 on: July 09, 2010, 03:49:10 AM »

Thanks for your continued attention and consideration regarding this, btw smiley
I just take you as a shining example smiley

I've boldly uploaded version 1.6.0 so you can use DCUpdater to update to the current version (if DCUpdater works with your setup).

Searching local files should now work with both relative paths (relative to where FindAndRunRobot.exe is) and FARR virtual folder variables. I've not implemented user variables.

from the help file:

Quote
searchUrl
This can be either a local file or a web page. For web pages, start with http://, for local files, either use an absolute path (e.g. c:\temp\somefile.txt), a relative path (e.g. ..\..\temp\somefile.txt) which will be resolved relative to where FindAndRunRobot.exe is, or use any of the FARR virtual folder variables (.e.g %MYDOCUMENTS%\somefile.txt).

Let me know if it works or not. If it doesn't, enable the log file with wms !Enable Logging. Then check the log file with wms !Open log file.
Logged

ewemoa
Honorary Member
**
Posts: 2,415



View Profile Give some DonationCredits to this forum member
« Reply #57 on: July 09, 2010, 04:08:21 AM »

I've boldly uploaded version 1.6.0 so you can use DCUpdater to update to the current version (if DCUpdater works with your setup).
...and I've boldly updated smiley

Quote
Searching local files should now work with both relative paths (relative to where FindAndRunRobot.exe is) and FARR virtual folder variables. I've not implemented user variables.
Both implemented options seem to work here  Thmbsup

Out of curiosity, why did you decide to make the relative path relative to where the .exe is -- as compared with where the FarrWebMetaSearch plugin lives?

Quote
Let me know if it works or not. If it doesn't, enable the log file with wms !Enable Logging. Then check the log file with wms !Open log file.
Cool features  thumbs up

Thanks for the local file feature!
Logged
phitsc
Honorary Member
**
Posts: 989



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #58 on: July 09, 2010, 04:42:27 AM »

Out of curiosity, why did you decide to make the relative path relative to where the .exe is -- as compared with where the FarrWebMetaSearch plugin lives?
Öhhhm, ähhhm, well ... to be honest, I didn't even think about that. Just took whatever was easiest. The difference just being passing NULL or some variable to a Win32 API function. If relative from where the plugin lives would be more convenient to you, I can certainly change that smiley

Thanks for the local file feature!
You are very welcome!
Logged

ewemoa
Honorary Member
**
Posts: 2,415



View Profile Give some DonationCredits to this forum member
« Reply #59 on: July 09, 2010, 06:25:09 AM »

If relative from where the plugin lives would be more convenient to you, I can certainly change that smiley
My current thoughts about this are that a common case would be to store local files to be parsed in Plugins\FarrWebMetaSearch\searches so that those files could live alongside their corresponding .conf files.  Consequently, having the "base" directory from which to resolve relative references be the Plugins\FarrWebMetaSearch\searches directory seems convenient to me.

BTW, I don't know if I mis-tested or I'm doing something wrong but I'm not finding the following sort of thing to be working now:

Quote
searchUrl=%PLUGINDIR%\searches\ahkscriptlisting.html

I get a popup informing me about not finding a file at: <driveletter>:\searches\ahkscriptlisting.html

For reference, the following seems to work:

Quote
searchUrl=%FARRDIR%\Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

and the following too:

Quote
Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

May be I've messed up somewhere...
Logged
lanux128
Global Moderator
*****
Posts: 6,088



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #60 on: July 15, 2010, 09:57:07 PM »

i made a .conf to search sites like Steam, D2D and Gamersgate. they are video games retailers who specialize in digital distribution. you may refer to the screenshot below to get an idea of how it works.

Usage info:





btw, i'd like to thank Perry Mowbray for his help in making these .conf and also to phitsc for this awesome plugin..



edit: fixed the regex for games that are on sale.

* steam_dd.zip (4.81 KB - downloaded 77 times.)
« Last Edit: July 22, 2010, 12:10:00 AM by lanux128 » Logged

phitsc
Honorary Member
**
Posts: 989



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #61 on: July 16, 2010, 06:36:07 AM »

And thank you for your contribution!  Thmbsup
Logged

phitsc
Honorary Member
**
Posts: 989



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #62 on: July 16, 2010, 06:49:31 AM »

BTW, I don't know if I mis-tested or I'm doing something wrong but I'm not finding the following sort of thing to be working now:

Quote
searchUrl=%PLUGINDIR%\searches\ahkscriptlisting.html

I get a popup informing me about not finding a file at: <driveletter>:\searches\ahkscriptlisting.html

For reference, the following seems to work:

Quote
searchUrl=%FARRDIR%\Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

and the following too:

Quote
Plugins\FarrWebMetaSearch\searches\ahkscriptlisting.html

May be I've messed up somewhere...

Yes, I confirm this. It seems like FARR does not replace the %PLUGINDIR% variable correctly.
Logged

Lashiec
Member
**
Posts: 2,374


see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #63 on: July 16, 2010, 10:25:58 AM »

How come I managed to miss this plugin until now? embarassed

Thanks to lanux for bringing it to my attention, and for the Steam alias, and to phitsc for all his hard work

* Lashiec pokes mouser to update DcUpdater plugin list
Logged
ewemoa
Honorary Member
**
Posts: 2,415



View Profile Give some DonationCredits to this forum member
« Reply #64 on: July 16, 2010, 11:40:08 PM »

Yes, I confirm this. It seems like FARR does not replace the %PLUGINDIR% variable correctly.
It is strange to me because IIRC in some of my plugins I use "resolve:%PLUGINDIR%" via callback_get_strvalue successfully.  I wonder what's going on...
Logged
phitsc
Honorary Member
**
Posts: 989



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #65 on: July 17, 2010, 04:23:43 PM »

It is strange to me because IIRC in some of my plugins I use "resolve:%PLUGINDIR%" via callback_get_strvalue successfully.  I wonder what's going on...

I'm using "resolvefile:". Will check if it makes a difference.
Logged

mouser
First Author
Administrator
*****
Posts: 33,294



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #66 on: July 17, 2010, 04:44:12 PM »

i should be able to fix this if i understand where it is failing to resolve %PLUGINDIR%
Logged
Hastur
Participant
*
Posts: 2

View Profile Give some DonationCredits to this forum member
« Reply #67 on: July 20, 2010, 11:44:08 AM »

There's a minor error in the regex for the timeanddate searches.

<td class="r"> should be <td class=r> in resultPattern.


Also, if you enter "wms timeanddate +sun http:", FARR will crash with a MSVC++ runtime library runtime error.  Sad
Logged
phitsc
Honorary Member
**
Posts: 989



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #68 on: July 20, 2010, 02:07:09 PM »

Thanks Hastur for the report and a warm welcome to the donationcoder forums smiley

What you found is one of FarrWMS's weaknesses. Unfortunately, when a web site changes the format of its pages, regexes can get "out of date". The same happened to the MSDN search a while back (and I'm still working on fixing it). TimeAndDate is one of the searches Perry made. I'll ask him if he can check it out.

I'll look into the runtime error, if I can reproduce it (will try tomorrow).
Logged

putnum
Participant
*
Posts: 68

View Profile Give some DonationCredits to this forum member
« Reply #69 on: October 19, 2010, 12:24:16 PM »

I love this plugin but I am having an issue with onelook. When I am using it it gets stuck on "plugins are searching" and it stays that way for prob a half a minute. Does anybody know why?
Logged
Alien282
Participant
*
Posts: 6

View Profile Give some DonationCredits to this forum member
« Reply #70 on: October 22, 2010, 03:58:17 PM »

Hello,

What you found is one of FarrWMS's weaknesses. Unfortunately, when a web site changes the format of its pages, regexes can get "out of date".

Absolutely. I see that the Google thingy is not working anymore, at least for the calculator. smiley Here is an update:

Formatted for Javascript with the GeSHI Syntax Highlighter [copy or print]
  1. resultPattern=(?:<h3 class="?r"?>\s*<a href="(.+?)".*?>(.*?)</a>.*?<div.*?>\s*(?:<div.*?/div>|(<span class="?f"?>.*?</span>.*?)<br>)?(.*?)<br>|<div id="?topstuff"?>.*?<img.*?calculator.*?>.*?<h2.*?>(.*?)</h2>)
  2. farrCaption=$5$2 $3
  3. farrGroup=$5$4
  4. farrPath=$5$1

It should be a bit faster too and shows PDF file results. Enjoy. smiley
Logged
phitsc
Honorary Member
**
Posts: 989



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #71 on: October 22, 2010, 04:31:20 PM »

Thanks guys for the feedback. I should maybe go over all the available searches and check if others are not working properly any more either. I'll definitely integrate your suggested changes so thanks for that.
Logged

putnum
Participant
*
Posts: 68

View Profile Give some DonationCredits to this forum member
« Reply #72 on: October 24, 2010, 01:51:51 PM »

Hi phitsc any update on this by chance?
Logged
phitsc
Honorary Member
**
Posts: 989



see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #73 on: October 27, 2010, 03:16:37 AM »

Not yet I'm afraid. With wms !Enable logging you can enable logging. I don't think it will help with tracking down timing problems though. I'll see if I can improve that.
Logged

ewemoa
Honorary Member
**
Posts: 2,415



View Profile Give some DonationCredits to this forum member
« Reply #74 on: November 03, 2011, 08:25:38 PM »

I have come across a site that appears to return a JSON array as a result (with mime type plain/text).  Is this format of data something FarrWebMetaSearch can handle?
Logged
Pages: Prev 1 2 [3] 4 Next   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.065s | Server load: 0.01 ]