topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Wednesday July 24, 2024, 9:28 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: Any free web/cloud solution to monitor a webpage for changes?  (Read 3073 times)

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,824
    • View Profile
    • Donate to Member
Hello!

Do you know any free web/cloud solution to monitor a webpage for changes and send you an email notification?

I have been using followthatpage.com but it gives me errors 426, 401 for some websites especially HTTPS.

Thanks!

Shades

  • Member
  • Joined in 2006
  • **
  • Posts: 2,925
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #1 on: February 20, 2024, 05:15 PM »
The tool type you search for is called:  Web Scraper.

There are many tools like that, the search engine of your choice will provide more than enough links. Some of these tools have a notification feature, email support is often included.

These tools are not that easy to use/configure, but once you get your head around these, they are pretty reliable at what they do. Realize though all these things you yourself will need to figure out as only you know on which information change has to trigger the notification. As most tools actually provide manuals and sometimes even step-by-step instructions, that shouldn't be too big of a problem.


publicdomain

  • Honorary Member
  • Joined in 2019
  • **
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #2 on: February 20, 2024, 07:38 PM »
... it gives me errors 426, 401 for some websites especially HTTPS.

426 is "upgrade required" (the destination server refuses to accept the current protocol).
401 is "unauthorized", since it needs authentication.

You may want to try another scraper but since cloud based means using someone else's server resources, they are likely to ask for some $ at some point. There is a list of "cloud based web scrapping solutions" at:

You can find some free tiers there :up:

Do notice cloud scraper prices can be way overpriced when compared with hosting your own.

The most efficient way is of course having your own web scraper configured in a server under your control -even if the hosting/server fee itself is free. The important part is having the ability to modify the scraper to account for HTML changes and being flexible in the ways to interact with the destination server as time goes by.



Relevant questions:

What frequency do you need to have? (daily, hourly, other)

When you say "monitor a webpage for changes" do you mean any change on the page or is it a certain portion of the contents within the page?

...You can actually get away with using only the "Last-Modified" header in the first case.

If it has to do with monitoring specific contents, then yes, a regular web scraper is due.



BTW I'm coding a "Webpage to Address book" program right now, so I can be of help/assistance since the first part of parsing data from the web is essentially scraping .

Cheers!
Vic
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com
« Last Edit: February 20, 2024, 07:50 PM by publicdomain »

sphere

  • Participant
  • Joined in 2018
  • *
  • default avatar
  • Posts: 176
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #3 on: February 21, 2024, 08:28 AM »
Do you know any free web/cloud solution to monitor a webpage for changes and send you an email notification?

You mentioned a web/cloud service.  There is often a free tier that you can use for less than 10 or 25 websites. There are also some selfhosted options. It used to be possible to self-host something like this using some free tier of a cloud service.

But in a pinch if I had to pay I would support an opensource project that had a reasonable pricing model. 

https://github.com/thp/urlwatch  /  https://thp.io/2008/urlwatch/ URLwatch  (I believe open source)
https://github.com/d...n/changedetection.io   opensource, docker image but I think there are limits of what you can do with selfhosted option.  Noticed this https://torvald.no/web-change-detection when looking for the link.
https://github.com/huginn/huginn a hackable IFTTT  docker
https://github.com/louislam/uptime-kuma  https://uptime.kuma.pet/   Opensource, there is a dem that has some site query information.


kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,824
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #4 on: February 28, 2024, 01:31 PM »
Thanks but I really need a third-party cloud service, any idea?

publicdomain

  • Honorary Member
  • Joined in 2019
  • **
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #5 on: March 01, 2024, 10:17 AM »
Thanks but I really need a third-party cloud service, any idea?

Idea: self-host with a Linux VPS in the cloud.
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com
« Last Edit: March 15, 2024, 11:56 PM by publicdomain »

dantheman

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 742
  • Be good if you can!
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #6 on: March 15, 2024, 08:22 AM »
Have never tried it as registered user, but it does have basic free option and seems to be able to check even sites without RSS.

https://newsblur.com/

4wd

  • Supporting Member
  • Joined in 2006
  • **
  • Posts: 5,644
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #7 on: March 23, 2024, 06:28 AM »
You can also run your own: changedetection.io

Web Site Change Detection, Restock monitoring and notifications.

Detect website content changes and perform meaningful actions - trigger notifications via Discord, Email, Slack, Telegram, API calls and many more.

kalos

  • Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 1,824
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #8 on: April 20, 2024, 02:06 PM »
I found this which seems interesting: https://webalert.me/

It says it will work as if you visit the website yourself with your browser so it will work with cookies, web forms etc.

I have not tried it yet, will see!

Shades

  • Member
  • Joined in 2006
  • **
  • Posts: 2,925
    • View Profile
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #9 on: April 20, 2024, 05:52 PM »
Do you know any free web/cloud solution to monitor a webpage for changes and send you an email notification?

You mentioned a web/cloud service.  There is often a free tier that you can use for less than 10 or 25 websites. There are also some selfhosted options. It used to be possible to self-host something like this using some free tier of a cloud service.

But in a pinch if I had to pay I would support an opensource project that had a reasonable pricing model. 

https://github.com/thp/urlwatch  /  https://thp.io/2008/urlwatch/ URLwatch  (I believe open source)
https://github.com/d...n/changedetection.io   opensource, docker image but I think there are limits of what you can do with selfhosted option.  Noticed this https://torvald.no/web-change-detection when looking for the link.
https://github.com/huginn/huginn a hackable IFTTT  docker
https://github.com/louislam/uptime-kuma  https://uptime.kuma.pet/   Opensource, there is a dem that has some site query information.


  • Huginn - Not easy to use. I have the Docker deployed in a Proxmox environment. Haven't spent all too much time with it, but the examples I saw when glancing over those, they looked involved.
  • UpTime Kuma - Run that also in the same Docker setup. This is not really useful for tracking content. It is great in hooking up any kind of service you run inside your network to let it check if it is up or not. You can then use webhooks to get informed by UpTime-Kuma via its interface, mail, Teams/Slack/Discord, Telegram and so many other (web-)services. In that guise, UpTime-Kuma is excellent. But for tracking changes in web-based content, not so much.

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 9,756
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: Any free web/cloud solution to monitor a webpage for changes?
« Reply #10 on: April 21, 2024, 06:27 PM »
You can also run your own: changedetection.io

Web Site Change Detection, Restock monitoring and notifications.

Detect website content changes and perform meaningful actions - trigger notifications via Discord, Email, Slack, Telegram, API calls and many more.

Combine the above with this:

https://www.oracle.com/cloud/free/