Harvest the Web with OutWit (Firefox)

[Curt]

For Firefox only, and still in first beta:
http://www.outwit.com/ or
http://www.snapfiles.com/reviews/http-wwwoutwitcom/outwit.html 
- or log in to get it from Mozilla at https://addons.mozilla.org/en-US/firefox/addon/7271

Harvest the web

If you desperately need...
- a list of all the design studios in London,
- your dream job ads in an Excel file every day,
- hundreds of photos of your favorite movie star,
- all available PDF files about Semantic P2P...

And if you are tired of scrolling down Web
pages, scanning text and compulsively
clicking, cutting and pasting for hours:
Here is the first beta release of
OutWit Hub, your Web Collection Engine.

http://www.outwit.com/

Our mission is to provide the Web community with a simple Web automation environment (finally) allowing everyone to harvest data elements, documents or media from virtually any public (and legal) source of content. The technology is open and will eventually provide an API and wizards to build simple and efficient tools.

The OutWit Platform is composed of a kernel that contains a large library of data recognition and extraction functions, around which an unlimited number of original extensions —called outfits— can be developed, using the kernel's features for specific applications.

An outfit is a small extension with its own user interface, features, scripts and directory of Web sources. Some outfits will be developed by us, but most, hopefully, will be developed by our users who have a specific need or passion.

Our first outfit, OutWit Hub, is a multi-purpose development & showcase application, in which we have gathered the largest possible number of features, hoping to cover a large spectrum of needs. The Hub will keep evolving and should become a very useful tool for advanced users. However, the real objective of this technology is to build simple, straight-to-the-point applications:

  1. a simple tool to collect images
  2. a simple tool to find a job
  3. a simple tool to follow the news in handball
  4. a simple tool to ... (this is why we will never succeed without you.)

At this point, the applications are countless; only you can steer us in the right direction(s).

The OutWit team, May 2008

http://www.outwit.com/

[electronixtar]

Sometimes I think it's not worthy to 'harvest' webpages, keep a webpage versioned, revised and updated is more and more popular. 

Just save URL to del.icio.us and check it out later.

Is the Web evolving from user-generated to user-refreshed ?

[kartal]

No screenshots, no faq!!

I use wikidpad, zotero for firefox  and pdf printing for collecting material of any kind off internet

http://www.zotero.org/

[Perry Mowbray]

Thanks Curt: this looks interesting. Are you using it?

I'm having a play/try at the moment... 

[Curt]

No, Perry Mowbray, I don't use it myself -
the subjects I would be looking for don't come in great numbers.

[masu]

Does't work on my system, Firefox crash each time I want to start it 

[f0dder]

Sometimes I think it's not worthy to 'harvest' webpages, keep a webpage versioned, revised and updated is more and more popular. 

Just save URL to del.icio.us and check it out later.

Is the Web evolving from user-generated to user-refreshed ?

Sometimes, pages go missing from the internet, though - which sucks. For some kinds of information (research, source code snippets, technical info, reverse-engineering related matters, ...) keeping a local copy can be very nice.

[cmpm]

Sounds like one can cut through a lot of wasted hits in a search on the net.

Good comments so far at Firefox-only 6 but 6 for 6 is actually good for these things, unless there are fake, but I don't think that's the case.

Going to give it a work out soon.

Google searches find too much unless you use quatation marks, and then you are not likely to get much at all, perhaps this will be more to the point.

[cmpm]

http://www.outwit.com/products/firstRun.php?version=0.7.0.183&id={5fb1186a-3398-4c47-b579-0f2eee222ad1}

[blackaj]

the extension has been updated for firefox3: https://addons.mozilla.org/en-US/firefox/addon/7271

and some (few) tutorials here: http://blog.outwit.com

[tomos]

http://www.outwit.com/products/firstRun.php?version=0.7.0.183&id={5fb1186a-3398-4c47-b579-0f2eee222ad1}

So, what is OutWit, in a word? OutWit is a Web collection engine for everyone. It runs on your Windows, MacOS or Linux machine and allows you to browse through and easily grab information, images, contacts or files from the Internet, in a few clicks. The question, when looking for anything on the Internet, is two-fold: find the pertinent data and make it usable for your purposes. Both processes can prove extremely time-consuming and both can be vastly improved using OutWit Hub. Originally conceived for researchers and data managers, the program is bringing Web scraping tools to everyone for both business and personal use. Just browse the Web for pages that include the information you are looking for; OutWit will scan the pages to recognize the data structure and format it into tables, allowing you to rate it and easily export it to files, spreadsheets or databases for later use.

sounds very interesting

[blackaj]

Hey guys,

I must say I am a friend (let's say, really close friend) of OutWit Hub's developers, but I honestly think it's a very cool piece of software.

OutWit Hub is a Web collection platform for FF3, it's a download manager / Web harvester that detects all kinds of data (images, links, rss feeds, address emails, doc and pdf, html tables..) in Web pages, sorts it, and lets the user work on it and export it. Applications are countless from automating image downloads (with really one or two clicks), to building databases of anything you want.

OutWit apparently has tons of downloads from their own site but in addons.mozilla.org, they are still stuck in the sandbox, waiting for reviews.
So, dear ones who are interested, (and I am sure all of you are!) you can help OutWit with just a few clicks of the mouse, and taps on the keyboard.

It's really easy, just head over to the Mozilla Addon site and sign up for an account if you don't have one. Once you are logged in, install OutWit and try it. It takes a little time to discover the real value of this program, but I think you'll see it's worth it. Then, if you like it, head back to the addon page and write a review.

For people who doesn't know about the Mozilla addons Sandbox, take a look at :
https://addons.mozilla.org/en-US/firefox/pages/sandbox
http://www.twistermc.com/blog/2007/07/24/save-addon

[Curt]

Updated today, but still in beta

File size:   970kb
Version: 0.81.83 Beta
Added:         Sep 01, 2008


http://www.outwit.com/ or
http://www.snapfiles.com/reviews/http-wwwoutwitcom/outwit.html
- or log in to get it from Mozilla at:
https://addons.mozilla.org/en-US/firefox/addon/7271