topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Wednesday February 28, 2024, 4:48 pm
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: Any free web/cloud solution to monitor a webpage for changes?  (Read 454 times)

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,814
    • View Profile
    • Donate to Member
Hello!

Do you know any free web/cloud solution to monitor a webpage for changes and send you an email notification?

I have been using followthatpage.com but it gives me errors 426, 401 for some websites especially HTTPS.

Thanks!

Shades

  • Member
  • Joined in 2006
  • **
  • Posts: 2,921
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #1 on: February 20, 2024, 05:15 PM »
The tool type you search for is called:  Web Scraper.

There are many tools like that, the search engine of your choice will provide more than enough links. Some of these tools have a notification feature, email support is often included.

These tools are not that easy to use/configure, but once you get your head around these, they are pretty reliable at what they do. Realize though all these things you yourself will need to figure out as only you know on which information change has to trigger the notification. As most tools actually provide manuals and sometimes even step-by-step instructions, that shouldn't be too big of a problem.


publicdomain

  • Honorary Member
  • Joined in 2019
  • **
  • Posts: 708
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #2 on: February 20, 2024, 07:38 PM »
... it gives me errors 426, 401 for some websites especially HTTPS.

426 is "upgrade required" (the destination server refuses to accept the current protocol).
401 is "unauthorized", since it needs authentication.

You may want to try another scraper but since cloud based means using someone else's server resources, they are likely to ask for some $ at some point. There is a list of "cloud based web scrapping solutions" at:

You can find some free tiers there :up:

Do notice cloud scraper prices can be way overpriced when compared with hosting your own.

The most efficient way is of course having your own web scraper configured in a server under your control -even if the hosting/server fee itself is free. The important part is having the ability to modify the scraper to account for HTML changes and being flexible in the ways to interact with the destination server as time goes by.



Relevant questions:

What frequency do you need to have? (daily, hourly, other)

When you say "monitor a webpage for changes" do you mean any change on the page or is it a certain portion of the contents within the page?

...You can actually get away with using only the "Last-Modified" header in the first case.

If it has to do with monitoring specific contents, then yes, a regular web scraper is due.



BTW I'm coding a "Webpage to Address book" program right now, so I can be of help/assistance since the first part of parsing data from the web is essentially scraping .

Cheers!
Vic
My name's Victor but do feel free to call me Vic!

Support with your DonationCredits!
❤️ Support on Patreon @ www.patreon.com/publicdomain
🎁 One-time Paypal @ www.paypal.com/paypalme/victorvls
Email/Paypal: publicdomainvicgmail.com
« Last Edit: February 20, 2024, 07:50 PM by publicdomain »

sphere

  • Participant
  • Joined in 2018
  • *
  • default avatar
  • Posts: 171
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #3 on: February 21, 2024, 08:28 AM »
Do you know any free web/cloud solution to monitor a webpage for changes and send you an email notification?

You mentioned a web/cloud service.  There is often a free tier that you can use for less than 10 or 25 websites. There are also some selfhosted options. It used to be possible to self-host something like this using some free tier of a cloud service.

But in a pinch if I had to pay I would support an opensource project that had a reasonable pricing model. 

https://github.com/thp/urlwatch  /  https://thp.io/2008/urlwatch/ URLwatch  (I believe open source)
https://github.com/d...n/changedetection.io   opensource, docker image but I think there are limits of what you can do with selfhosted option.  Noticed this https://torvald.no/web-change-detection when looking for the link.
https://github.com/huginn/huginn a hackable IFTTT  docker
https://github.com/louislam/uptime-kuma  https://uptime.kuma.pet/   Opensource, there is a dem that has some site query information.


kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,814
    • View Profile
    • Donate to Member
Thanks but I really need a third-party cloud service, any idea?