Welcome Guest.   Make a donation to an author on the site October 31, 2014, 09:04:41 PM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
Read the full one-year retrospective report on DonationCoder.com.
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: [1] 2 Next   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: Latest Web Link Captor Release - v1.11.01 - Dec 17, 2013  (Read 15153 times)
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« on: December 11, 2011, 04:20:49 AM »

Web Link Captor is my new application, which was created as part of the NANY 2012 event at DonationCoder.com

Web Link Captor is a program that takes a list of plaintext items, and builds an output document containing a list of url-linked results based on web searches for the items. It's a way to quickly build a clickable list of items from just their names, suitable for posting on your blog or a forum, etc.

Download latest version from Official Web Page: HERE (download now available to the public).

Watch videos of the program in action: Part 1 and Part 2.

Features:
  • Friendly GUI lets you build flexible sets of processing steps to parse plaintext input lists and perform search and output functions.
  • You can save and load projects and script configurations for easy re-use.
  • Uses a set of standalone utility scripts that can be run without the windows front end GUI, and can be easily modified and expanded.
  • The included backend scripts are in Python (but other languages can be used)
  • Uses caching to avoid repeating web searches unnecessarily.
  • Comes with output formatters for HTML, BBCode, Markdown/Wikipedia
  • Comes with searching engines for google, and bing/opensearch (which can handle amazon, wikipedia, and many others)
  • Flexible scoring system lets you interactively or programmatically tweak your results to get the best list of links.





Version History

v1.11.01 - December 17, 2013
  • [Feature] Scripts now create different temporary filenames in temp directory to make it much easier to debug where a search or manipulation script went wrong.
  • [Documentation] Added help information for how to narrow a search to a specific site in google and how to add text to input.
  • [BugFix] Excerpts in result grid that contained newlines were messing up the result grid (and potentially the results).
  • [Feature] Option to check for updates at startup.

v1.10.01 - April 17, 2012
  • [BugFix] Double-quotes in search strings were not being handled properly and resulting in 0 results

v1.08.01 - December 24, 2011
  • [Feature] Now packaging with a minimal python addon included

v1.07.01 - December 23, 2011
  • [Feature] New script combobox is hierarchical.
  • [Feature] Script Tool files now specified hierarchical path and unique identifier which will make it easier to change tool labels without losing track of scripts.
  • [Feature] Better error reporting on missing scripts.

v1.06.01 - December 22, 2011
  • [MinorFeature] Added plaintext output script
  • [BugFix] Fixed bug where blank labels could cause crash
  • [Minor Feature] New option in output to remove labels from items
  • [MinorFeatures] Minor improvements

v1.05.01 - December 15, 2011
  • [MajorFeature] You can now download Python2 addon pack for the program, rather than using a standard python installation.
  • [MinorFeatures] Lots of small user interface improvements.

v1.03.01 - December 13, 2011
  • [MajorFeature] Embedded Python 2.7 DLL mode option -- MUCH faster
  • [Feature] Improved all python scripting invocation features and options and error reporting
  • [MinorFeature] Added more gui options including ability to hide help panels
  • [MinorFeature] Small improvements to tools (ability to show readme links)

v1.02.01 - December 13, 2011
  • [MajorFeature] You can now use an embedded DLL python interpretter option
  • [Feature] Cache controls from options
  • [Feature] Includes additional search scripts from ewemoa (thanks) [note they require some additional module installations]
  • [MinorFeature] Reports elapsed time for operations
  • [MinorFeature] Scripts can now include readme files and link to them from their tool descriptions.
  • [MinorFeature] Some reorganization of script files
  • [MinorFeature] Hover over links shows url in statusbar of web preview.

v1.01.05 - December 11, 2011
  • Regular expression script improvements -- should be faster and may work on Python 2.6 now

v1.01.04 - December 11, 2011
  • Tools/Scripts improvements
  • Minor GUI tweaks



You can read the original NANY2012 discussion that led to this program here.

« Last Edit: December 17, 2013, 10:51:53 AM by mouser » Logged
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #1 on: December 11, 2011, 04:24:37 AM »


The program will be officially released as part of NANY 2012, on January 1st, 2012.

BUT If you would like to try it early, just send me a personal message via the forum or email me (mouser@donationcoder.com) and I will send you a link.  Thmbsup
« Last Edit: December 11, 2011, 04:32:43 PM by mouser » Logged
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #2 on: December 11, 2011, 04:45:35 AM »

As a super simple example of the kinds of things that WLC can do, I put in a list of a few of my favorite bands, and click "Run" to get the following output, ready to paste in a forum post:

Input:
Quote
Cat Power - The Greatest
Bikini Kill - Reject All American
The Doo Wop Box - Volume one
Dexy's Midnight Runners - Greatest Hits


Output (raw):
[copy or print]
[url=http://www.amazon.com/Greatest-Cat-Power/dp/B000C0X3ZC]Cat Power[/url]
[url=http://www.amazon.com/Reject-All-American-Bikini-Kill/dp/B00000373N]Bikini Kill[/url]
[url=http://www.amazon.com/Doo-Wop-Box-Various-Artists/dp/B00000333M]The Doo Wop Box[/url]
[url=http://www.amazon.com/Too-Rye-Ay-Dexys-Midnight-Runners/dp/B00005Y1ZF]Dexy's Midnight Runners[/url]

Output:



And here's output from a different kind of search and output on those same bands:

Cat Power:

Bikini Kill:

The Doo Wop Box:

Dexy's Midnight Runners:
Logged
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #3 on: December 11, 2011, 04:57:22 AM »

One of the major goals of the program is to provide a flexible way to rank and filter search results.

This can be done interactively using the result grid, by using pattern matching to modify scores, by adding terms and options to web searches, and by using external processing scripts to apply heuristics.

In such a way one can hope to filter out results that are undesirable and favor sites that are more likely to have good results.

I'm hoping that people will write and share additional such filter and search scripts with us to improve the program.  The included scripts are written in python and it's (relatively) easy to add your own new ones to the system.
Logged
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #4 on: December 11, 2011, 04:59:58 AM »

I hope the program is useful to people.  There is a lot that could still be added to it.  I will be posting a screencast video tutorial in the next day or so.

And lastly.. If you'd like to see this program continue to be developed, please consider making a donation -- It would be very appreciated. Thank you  Thmbsup
Logged
CleverCat
Supporting Member
**
Posts: 1,126


Cat's Are Fun!

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #5 on: December 11, 2011, 05:25:27 AM »

This looks very promising - can't wait to try it!  Thmbsup
Logged

If you need help - JUST ASK!
kyrathaba
N.A.N.Y. Organizer
Honorary Member
**
Posts: 3,014



while(! dead_horse){beat}

see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #6 on: December 11, 2011, 06:39:51 AM »

That is a really useful application!
Logged

Win 7 Home Premium 64bit-SP1 AMD Athlon II X2 220 Socket AM3 (938) @ 2.1GHz 6GB RAM Firefox 26.0
_________________________________________________________________________________________

I'm fighting against patent trolls. Join me and tell your representative to support the #SHIELDAct: https://eff.org/r.b6JJ /via @EFF

My DC page: http://kyrathaba.dcmembers.com | My blog: http://williambryanmiller.com/ | Proofreading Service: http://bit.ly/1fQSqQP

mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #7 on: December 11, 2011, 10:50:01 AM »

I've uploaded two video screencasts of the program in action:

Video 1 is four minutes and just shows the program in action:
.

Video 2 is ten minutes long and goes into more detail:
.
« Last Edit: December 24, 2011, 08:06:49 AM by mouser » Logged
Ath
Supporting Member
**
Posts: 2,241



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #8 on: December 11, 2011, 01:08:26 PM »

Just watched the video, and I'm very impressed Thmbsup Thmbsup Thmbsup
Great program.

Had to use the DC hosted version, as the screen-text in the youtube release was nearly unreadable for me ohmy
Logged

IainB
Supporting Member
**
Posts: 4,809


Slartibartfarst

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #9 on: December 11, 2011, 03:46:01 PM »

+1 from me for what Ath wrote.
Quite elegant!

Just curious:
On the input: would it be possible to direct the proggy to recognise and pick up the search terms from CHS, if they were already in the CHS database?
On the searched sources: would it be possible to have the proggy also search CHS for matches? (I think this would imply that the data to be matched was in the Clip data, and the related URL reference would be a string (or strings) in the Notes column/field or the Clip data.)
Logged
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #10 on: December 11, 2011, 03:49:51 PM »

Leaving aside the specific issue of getting data from CHS.. Let me say that it would absolutely be possible to have the program "search" for "results" not on the web, but in a local database or text file catalog, etc. and there is no reason that the results would necessarily have to be urls.

And in a more general sense, the codebase for this program is really all about instantiating chains of parameterized processing scripts that work on (text) files.. and the code may have additional uses in creating different programs that do different things but use the same paradigm of letting user assemble and configure chains of batch scripts to run.
Logged
Deozaan
Charter Member
***
Posts: 6,439



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #11 on: December 11, 2011, 09:48:45 PM »

Watch the original full size and much higher quality video from the DC server:

Great screencast. But I'm curious why YouTube couldn't handle the full size/high quality video? Was it a limitation or a choice to have lower quality on YouTube?

WLC looks really awesome! It makes me wish I wrote more lists that needed links.
Logged

mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #12 on: December 11, 2011, 10:06:09 PM »

Quote
Was it a limitation or a choice to have lower quality on YouTube?

certainly wasn't by my choice.. i also don't know why it shows it in bad quality.  the original file i 800x600.
Logged
ewemoa
Honorary Member
**
Posts: 2,484



View Profile Give some DonationCredits to this forum member
« Reply #13 on: December 12, 2011, 06:03:33 AM »

I've uploaded a long (sorry) video demonstration of the program in action:

Watch on youtube:
...

I found that it was easier to see the text in the 480p version.

Also, it's at about 8:40 that one gets to see the first generated web preview.  May be that will make 20 minutes seem like less smiley
Logged
ewemoa
Honorary Member
**
Posts: 2,484



View Profile Give some DonationCredits to this forum member
« Reply #14 on: December 12, 2011, 06:28:01 AM »

In the output preview section, is there a recommended way to transfer the output elsewhere?  With the appropriate tab selected Control+A, Control+C, and Control+V at the destination?
Logged
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #15 on: December 12, 2011, 06:49:12 AM »

Quote
In the output preview section, is there a recommended way to transfer the output elsewhere?  With the appropriate tab selected Control+A, Control+C, and Control+V at the destination?

yeah i suppose i really should have a Copy To Clipboard button shouldn't I?
Logged
IainB
Supporting Member
**
Posts: 4,809


Slartibartfarst

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #16 on: December 12, 2011, 03:32:27 PM »

Quote
In the output preview section, is there a recommended way to transfer the output elsewhere?  With the appropriate tab selected Control+A, Control+C, and Control+V at the destination?

yeah i suppose i really should have a Copy To Clipboard button shouldn't I?

Maybe also consider implementing a design concept of structured integration with CHS, rather than ad hoc extensions?
So you might get, for example (from this thread so far), options to:
  • INPUT: recognise and pick up the search terms from CHS, if they were already in the CHS database.
  • PROCESS: include CHS database records (of some definable type) in the search for matches.
  • OUTPUT: output results to CHS.
That could save a lot of duplication of effort and mucking about when using the WLC (assuming that you already had the relevant data in CHS and wanted the output in there too.)

On another point, I would be very interested in ß testing WLC if you were opening it up for "...additional uses in creating different programs that do different things but use the same paradigm of letting user assemble and configure chains of batch scripts to run" (as you had intimated above).
Logged
mahesh2k
Supporting Member
**
Posts: 1,408



see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #17 on: December 12, 2011, 04:02:23 PM »

I see one more use of WLC : Capture the URLs from various web pages into clipboards and later add it to WLC for uploading these urls into delicious or similar other sites. This saves time, instead of using browser widgets and bookmark link, this can process large chunk of URLS. I hope this could be possible with extensions.
Logged
IainB
Supporting Member
**
Posts: 4,809


Slartibartfarst

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #18 on: December 12, 2011, 04:28:49 PM »

In terms of "other uses", I'm not sure, but I wondered if this app. might be relevant: Boxie - Fix Music File Names (from id3 tags)
I wanted to take a look at their website but Malwarbytes tells me it is a dodgy URL, so I didn't.
Logged
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #19 on: December 12, 2011, 04:36:04 PM »

Well one very very natural extension (simplification really) of WLC would be as a combination GUI and extensible script collection for performing intricate text manipulation tasks.  It wouldn't be able to DO things you couldn't do using a scripting language, but it could make creating, configuring, and previewing chains of operations much easier to work with.
Logged
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #20 on: December 12, 2011, 05:29:04 PM »

Ive replaced the video with one half as long.
Logged
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #21 on: December 22, 2011, 10:48:53 PM »

Version History

v1.07.01 - December 23, 2011
  • [Feature] New script combobox is hierarchical.
  • [Feature] Script Tool files now specified hierarchical path and unique identifier which will make it easier to change tool labels without losing track of scripts.
  • [Feature] Better error reporting on missing scripts.

v1.06.01 - December 22, 2011
  • [MinorFeature] Added plaintext output script
  • [BugFix] Fixed bug where blank labels could cause crash
  • [Minor Feature] New option in output to remove labels from items
  • [MinorFeatures] Minor improvements

v1.05.01 - December 15, 2011
  • [MajorFeature] You can now download Python2 addon pack for the program, rather than using a standard python installation.
  • [MinorFeatures] Lots of small user interface improvements.

v1.03.01 - December 13, 2011
  • [MajorFeature] Embedded Python 2.7 DLL mode option -- MUCH faster
  • [Feature] Improved all python scripting invocation features and options and error reporting
  • [MinorFeature] Added more gui options including ability to hide help panels
  • [MinorFeature] Small improvements to tools (ability to show readme links)

v1.02.01 - December 13, 2011
  • [MajorFeature] You can now use an embedded DLL python interpretter option
  • [Feature] Cache controls from options
  • [Feature] Includes additional search scripts from ewemoa (thanks) [note they require some additional module installations]
  • [MinorFeature] Reports elapsed time for operations
  • [MinorFeature] Scripts can now include readme files and link to them from their tool descriptions.
  • [MinorFeature] Some reorganization of script files
  • [MinorFeature] Hover over links shows url in statusbar of web preview.

v1.01.05 - December 11, 2011
  • Regular expression script improvements -- should be faster and may work on Python 2.6 now

v1.01.04 - December 11, 2011
  • Tools/Scripts improvements
  • Minor GUI tweaks

Note:
If you are upgrading your old script configurations will not be found, as the identifiers for them have changed.  Simply double click on each script entry and reselect it to fix.
Logged
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #22 on: December 24, 2011, 03:19:39 AM »

v1.08.01 - December 24, 2011
  • [Feature] Now packaging with a minimal python addon included
Logged
mouser
First Author
Administrator
*****
Posts: 33,614



see users location on a map View Profile WWW Read user's biography. Give some DonationCredits to this forum member
« Reply #23 on: December 30, 2011, 03:36:19 PM »

Web Link Captor download is now open to the public on the program web page listed above.
Let me know how you like it.  Thmbsup
Logged
SFdude
Participant
*
Posts: 3

View Profile Give some DonationCredits to this forum member
« Reply #24 on: January 08, 2012, 07:28:45 PM »

Web Link Captor: great and useful!  smiley

Quick question:

Does WLC in any way, search and/or display
any of its web results
(for ex: in the "Web Preview Tab"),
using the MS Internet Explorer browser,
or any MS-IE tools?

I'm concerned about using the IE Browser, any version,
because of security concerns.
(I never use IE, only Firefox & Chrome in XP-SP3 f/browsing...that just me).

Thanks for your answer!
SFdude
« Last Edit: January 08, 2012, 07:34:15 PM by SFdude » Logged
Pages: [1] 2 Next   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.062s | Server load: 0.14 ]