Farr Plugin Idea Again: Generic web page retrieve with regex show

[mouser]

This request goes out to czechboy and ecaradec who have been doing javascript plugins lately.

I don't know if ecaradec's plugin supports this yet, but one thing you can do with plugins that people haven't been using yet is you can let an alias call a plugin directly.  This isn't really much different than calling it when someone types something in but if you read my suggestion you'll see why i mention it.

I think it would be really useful to have a generic plugin that was meant to be called from alieses.

Here's what the plugin would do, it would be invoked like this:

webgrab URL "regexgrab" "outputformat"

This would cause plugin to
1. grab the page at URL
2. Match text regexgrab with pattern group matching
3. Display memo result of outputformat doing replacement of matched groups

This webgrab function is generic function that anyone could write aliases for.

So for example i might write an alias like this:
Alias: wtemp (.*)
Invokes: webgrab.dll webgrab http://www.weather.com/weather/local/$$1 "Feels like (.*)F" "Weather in your town is %1 degrees"

using this alias would cause the webgrab plugin to:

1. grab the page at http://www.weather.c.../weather/local/61820
2. Match text Feels like (.*)F
3. Display memo result of Weather in your town is %1 degrees (where %1 is the regex pattern match in step 2)

Hope its not too confusing to see the dual use of regular expressions and replacement..
Basically the alias has its own regex match which grabs the users zipcode that they type,
And then the second use is when telling the webgrab plugin how to match and display results.


The idea is that this one plugin could be used by lots of people to make and share different aliases that could grab all kinds of info from web pages.

[jgpaiva]

I see this idea, looks really cool!

I have another suggestion: how about if it accepted several trios webpage regex1 display and displayed each on its own line?
This way, we could have a bunch of predifined cities when people wrote "weather", and the first one would be the one people typed in.

[nitrix-ud]

Very good idea ! 

concerning webpages, i'm still waiting for the showmemo command to be able to display webpages, that would be great too 

[mouser]

dc member ewemoa has a working version of this plugin that i have been testing.  it's EXTREMELY cool. 

[ewemoa]

A preliminary version should be available for interested parties

[ewemoa]

A new version (1.0.0.5) should be available now (see previous message above for download location).

This version provides an alias "wgt" which launches an IE window from which one should be able to test various arguments to feed to Webgrab.  One can do testing via this window and once a satisfactory set of arguments is obtained, manually translate them [1] into an alias for later use.


[1] I'm hoping to provide some functionality eventually to "compute" an appropriate alias string so the translation doesn't have to be done manually.  For the moment, one can probably get some idea of what to do to translate based on the pre-installed alias "wgwz".

[mouser]

[1] I'm hoping to provide some functionality eventually to "compute" an appropriate alias string so the translation doesn't have to be done manually.

this plugin has such incredible potential.. i think once people see what it can be used to do it will open up a whole slew of stuff.

but what do you meant about avoiding manual translation?

[ewemoa]

The testing via IE is done by filling in forms in a field -- so there is one field for the URL to fetch, one field for the regular expression to use for matching, and one field for the template to use to build the results. 

In the context of the FARR window (or specifying an alias), the arguments to "webgrab" (i.e. URL, regex string, and template string) are not all exactly in the same format as for the form.  In particular, the latter two must be surrounded by double quotes and consequently (I believe) be escaped appropriately (e.g. if there was a double quote in a regular expression tested in the form context, it would need to be escaped before being used as an argument to the "webgrab" alias [1]).

In general then, if you start w/ the form, come up w/ a set of arguments that works in that context, and then want to save them for later use, say as an alias, you may be faced w/ the task of "translating" them as a straight copy-and-paste will not work (at a minimum you'll need to wrap the regex string and template string in double quotes and depending on the precise content of the two arguments may be faced w/ the task of escaping certain characters).

Is this any clearer?

On a side note, I've also been wondering about potential mismatches between FARR's regular expression handling and that of JS -- I think ecaradec alluded to this in another thread at some point too -- AFAIU, they are not exactly the same.


[1] But in the current implementation, I don't think Webgrab handles double quotes in regex strings or templates