topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Monday December 2, 2024, 6:16 pm
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: AddressBooker - Webpage to Address Book  (Read 6006 times)

publicdomain

  • Moderator
  • Joined in 2019
  • *****
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
AddressBooker - Webpage to Address Book
« on: February 23, 2024, 09:48 PM »
This is based on fellow member @sphere's request:


After some brainstorming & discussion, the planned feature set for the program is:

AdressBooker v1 roadmap:

  • Extract contact information, contact photo (see original forum post)  from web page (by monitoring the clipboard) and bulk process saved pages from a folder directory ( https://github.com/gildas-lormeau/SingleFile/wiki )
  • Pull information from  ‘+” link on (ie instagram)
  • User will be presented with a dialog when an URL that contains a website from the template/definitions list is copied. 
  • Dialogue box will have:
  • Option to select and/or add group/category,  tags and short note from dialogue box/
  • Option to include webpage title and source url when collecting information.
  • User definable hotkey to pause clipboard monitoring in case user wants to clip address without triggering app.

Additional settings:
  • Ability to edit and create new page templates.

Information will be collected into an Addressbook that will be developed later to group/sort contacts add some notes etc.
-sphere

GitHub Repo @ https://github.com/publicdomain/addressbooker

Development of AddressBooker has officially begun! :Thmbsup:

Cheers!
Vic
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com
« Last Edit: February 25, 2024, 04:23 PM by publicdomain »

sphere

  • Participant
  • Joined in 2018
  • *
  • default avatar
  • Posts: 176
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #1 on: February 24, 2024, 01:42 AM »
Thats great news!  Looking forward to it. Thank you.

publicdomain

  • Moderator
  • Joined in 2019
  • *****
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #2 on: February 24, 2024, 10:46 AM »
Thats great news!  Looking forward to it. Thank you.

I'm happy to assist in bringing your brainchild to life! :-* Having the first ALPHA of AdressBooker available for commenting on its functionality is the goal for this weekend :Thmbsup:

Stay tuned!
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com
« Last Edit: February 24, 2024, 10:58 AM by publicdomain »

publicdomain

  • Moderator
  • Joined in 2019
  • *****
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #3 on: February 25, 2024, 04:23 PM »
Extract contact information, contact photo (see original forum post) from web page (by monitoring the clipboard) and bulk process saved pages from a folder directory ( https://github.com/g...meau/SingleFile/wiki )

As the actual implementation of this main feature happens, there have been some considerations:

- Libraries dealing with direct HTML scrapping were all discarded (sites can and do load dynamically via scripts regularly).

- We originally discussed Selenium + chromedriver. I am familiar with good ol' Selenium since many years ago but I'm using this first web scrapper of 2024 for having a go at some of the more-modern libraries, particularly Google's Puppeteer (https://github.com/puppeteer/puppeteer), which is looking like the way to go.

It's a bulky package when working with Chrome for testing, but it is guaranteed to work since the integration is made at the developer level as a first-party tool (an "indivisible entity with Chromium").

It may mean a bit of an upfront learning curve for a couple more days but seems that investing in Google's Puppeteer can make AddressBooker a proper "headless" scrapper for the modern web (given the requirements are the likes of Instagram, Facebook and the program is likely going to be used with more of the modern social webs).
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com

publicdomain

  • Moderator
  • Joined in 2019
  • *****
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #4 on: February 25, 2024, 08:10 PM »
@sphere

- What's your current Windows version?

I'm flipping and flopping between C# and JavaScript/TypeScript. I have more experience with C# for the desktop but it sounds like some form of node integration is unavoidable here.

My aim is doing this right, so any webpage that Chrome can display we can scrape correctly (regardless of size/megabytes or old Windows compatibility). This solves the many (many!) configuration issues that working with multiple browsers and versions of such  browsers can bring, right off the bat.
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com

sphere

  • Participant
  • Joined in 2018
  • *
  • default avatar
  • Posts: 176
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #5 on: February 25, 2024, 11:46 PM »

- What's your current Windows version?

I run windows 10 currently but expect to have to shift to windows11 at some point.   I also have a linux Mint machine but was not expecting this to work there. 

publicdomain

  • Moderator
  • Joined in 2019
  • *****
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #6 on: February 29, 2024, 05:06 PM »
I run windows 10 currently but expect to have to shift to windows11 at some point.   I also have a linux Mint machine but was not expecting this to work there. 

Thank you. The first ALPHA version should be coming up. I'm running an Ubuntu-based distro (20.04) so it might as well be Linux since I'm using it with good results.

Puppeteer programming is logical and I'm confident this is the way forward since basically all the sites & webmasters account for Chrome/Chromium. The only complaint is the ~400MB overall release package size but we'll cope with it.

My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com
« Last Edit: February 29, 2024, 06:27 PM by publicdomain »

sphere

  • Participant
  • Joined in 2018
  • *
  • default avatar
  • Posts: 176
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #7 on: February 29, 2024, 08:19 PM »
Thank you. The first ALPHA version should be coming up. I'm running an Ubuntu-based distro (20.04) so it might as well be Linux since I'm using it with good results.
Thats great. I have Linux Mint.
Puppeteer programming is logical and I'm confident this is the way forward since basically all the sites & webmasters account for Chrome/Chromium. The only complaint is the ~400MB overall release package size but we'll cope with it.
That is pretty large, but totally doable.  You had mentioned it being portable. So I am assuming if need be I could run it off of an external USB 3.0 or more.   

publicdomain

  • Moderator
  • Joined in 2019
  • *****
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #8 on: March 05, 2024, 11:54 PM »
That is pretty large, but totally doable.  You had mentioned it being portable. So I am assuming if need be I could run it off of an external USB 3.0 or more.

Currently doing all development on Linux. Portable looks doable, but I'm only focused on getting it going (and doing it right, including userDataDir for consistent profile + xPath definitions).

Please tell me if this command runs on your end (Mint):

npm i puppeteer

That should do the setup to run the program cloning from GH.

(Let's handle making it portable later).
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com
« Last Edit: March 06, 2024, 12:44 PM by publicdomain »

sphere

  • Participant
  • Joined in 2018
  • *
  • default avatar
  • Posts: 176
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #9 on: March 06, 2024, 01:03 PM »
Worrying about portable later makes sense. I will try and run it tonight when I get home, but it will be pretty late. Otherwise I might not get to it until tomorrow.
Hope that does not delay you to much.
Thanks

publicdomain

  • Moderator
  • Joined in 2019
  • *****
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #10 on: March 06, 2024, 02:33 PM »
No worries AMIGO :up: I just want to see where your Linux box is at (the full list of commands is larger; the AddressBooker program is a Node script at this time hence it needs you to have a suitable runtime set-up).

Realistically, it'll be until Friday & the weekend when I devote full days to finishing DC programs entirely so go at ease

(Thank to you for accepting the penguin for a platform as it streamlined things)
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com

publicdomain

  • Moderator
  • Joined in 2019
  • *****
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #11 on: March 11, 2024, 03:59 PM »
Notice: part of the testing is being handled by PM, but the working result is definitely being made open upon release :up:
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com

publicdomain

  • Moderator
  • Joined in 2019
  • *****
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #12 on: March 11, 2024, 06:02 PM »
npm i puppeteer

That should do the setup to run the program cloning from GH.
-publicdomain (March 05, 2024, 11:54 PM)

In case someone else is following, this is the current command line setup:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash

source ~/.bashrc

nvm install 18

nvm use 18

npm i puppeteer

We're about to start the actual public testing of definitions, etc.
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com

publicdomain

  • Moderator
  • Joined in 2019
  • *****
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #13 on: March 23, 2024, 07:33 PM »
App with clipboard monitoring and xpath definitions for the holidays.

This is the final "larger" release to close the month (after delivering this one, we open our new software project).
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com
« Last Edit: March 23, 2024, 07:59 PM by publicdomain »

publicdomain

  • Moderator
  • Joined in 2019
  • *****
  • Posts: 736
  • Call me Vic!
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #14 on: March 25, 2024, 03:43 PM »
App with clipboard monitoring and xpath definitions for the holidays.
-publicdomain (March 23, 2024, 07:33 PM)

Okay! I've started pushing versions to GitHub incrementally, for confirmation of at least one (1) user --likely fellow user "sphere" :)



First ALPHA confirms the program is not detected as a bot:


Setup:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash

source ~/.bashrc

nvm install 18

nvm use 18

npm install puppeteer puppeteer-extra puppeteer-extra-plugin-stealth
 
=====

Clone:

git clone https://github.com/publicdomain/addressbooker.git

=====

Run:
 
node addressbooker.js
My name's Victor but do feel free to call me Vic! (now known as "paradisusvic")

❤️ Support on Patreon @ www.patreon.com/paradisusis
New Email/Paypal: paradisusvicgmail.com
« Last Edit: March 25, 2024, 11:08 PM by publicdomain »

sphere

  • Participant
  • Joined in 2018
  • *
  • default avatar
  • Posts: 176
    • View Profile
    • Donate to Member
Re: AddressBooker - Webpage to Address Book
« Reply #15 on: March 26, 2024, 12:26 AM »
Thanks.  :Thmbsup: Ran it.