topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • December 17, 2018, 12:41 PM
  • Proudly celebrating 13 years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Author Topic: Why AdGuard? Why do they detect ad-blockers to begin with?  (Read 664 times)

ital2

  • Member
  • Joined in 2017
  • **
  • default avatar
  • Posts: 108
    • View Profile
    • Donate to Member
Why AdGuard? Why do they detect ad-blockers to begin with?
« on: December 01, 2018, 11:23 AM »
Since my question in the discount section was not answered:


"Re AdGuard: Since it's by subscription, can anybody confirm that it allows for viewing/browsing main "newspaper" and other propaganda sites which systematically leave you out if you use the usual adblocker (e.g. Adblock, Adblock Plus), and if so, which ones (and understood of course that you do NOT make an "exception")?

If there is some link to something potentially important, I open it then in Edge instead of FF, but I don't browse such sites anymore.

If the above is not the case, why is AdGuard so well regarded her in this forum? Since Adblock Plus works fine, too, for the sites remaining for me. Is it just because of "for once, the developer gets paid!"? Well, Adblock Plus's one gets paid, too - some call that extortion, but he wins the actions against him, up to now.

So I would really be interested in some pertinent information, and ideally along the lines, "is not blocked by adblocker-blocking sites, for most of them.""


So, what the heck with a paid / subscription ad-blocker, since the others work well, except for being detected by more and more sites?


Or in other words, yes, a non-(frequently) detectable ad-blocker would be worth its money again!

But then, why is that so difficult, since even with the usual ad-blockers of today, the whole crap is downloaded to your system first, THEN only your ad-blocker begins it discarding work?

Deozaan

  • Charter Member
  • Joined in 2006
  • ***
  • Points: 1
  • Posts: 8,597
    • View Profile
    • The Blog of Deozaan
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #1 on: December 01, 2018, 02:29 PM »
It might help if you provided some links to some "newspaper" or "propaganda" sites which prevent access due to ad blockers so people here can test them and let you know.

Shades

  • Member
  • Joined in 2006
  • **
  • Posts: 2,417
    • View Profile
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #2 on: December 01, 2018, 04:40 PM »
I use uMatrix in FF. All news site open just fine. uMatrix blocks any or all of the following ads/scripts/media/images/frame/css/xhr in any of the combinations you configure it to. And it does so per website. I am barely bothered by any ad, cookie, tracker scripts, (external) images etc.

Don't know why people keep bothering so much with ad-blocker this or that. In my anecdotal experience, uMatrix has all of them beat. Yes, all those items being blocked can lead to layout mishaps and some things might not function as you would expect. But then you can start enabling items in the matrix until the site and/or functionality works again and store those settings. And you can do this per website. All that configuring could be considered more of a job, instead of an easy-peasy solution, but I think the extra configuration work will be to your benefit in the end.

I visit a lot of Dutch newspaper web sites, so checking those would be useful to no-one, except for fellow Dutchies or Flemiings here in the forum. Also, whenever it is possible, I do not click on the 'Accept cookies' button from a web site. On some sites that stops a lot of scripts running in the background as well.

ital2

  • Member
  • Joined in 2017
  • **
  • default avatar
  • Posts: 108
    • View Profile
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #3 on: December 05, 2018, 02:48 AM »
Thank you, both of you, and as always, Shades' contributions really constructive!

"Also, whenever it is possible, I do not click on the 'Accept cookies' button from a web site." - I said so here, some weeks ago, in some other context, complaining about the fact that then, often, just 2/3, or even just the half of the screen height is left for my reading! ;-(

As for propaganda sites (formerly "Newspapers" ' sites) I don't have access without allowing ads:

http://www.bild.de (Springer for the masses, but for the "very latest news", i.e. the things that have happened within the very last 2 hours or so, they're often really informative; the alternative for this being dailymail.com, though - but then, no day without "royals" et al. on dailymail.com, and that makes me vomit)

http://www.faz.de (Frankfurter Allgemeine Zeitung, 30 years ago, that was the best (continental) European newspaper, together with Le Monde, at the time; both are finished now as serious information sources)

http://www.sueddeutsche.de (Süddeutsche Zeitung, informally "Prantl-Prawda", was very good, too, before Prantl as editor-in-chief)

http://www.spiegel.de (Der Spiegel, world-best news-weekly 30 years ago; at the time, Der Spiegel was sufficient reason for some to learn German, not kidding! - their incredible, mythical, multi-km-broad archives were the starting point for my interest in information management / technology; today, their site - the printed publication remains slightly more acceptable in parts - is almost-pure propaganda shit; it's from Bertelsmann, the censorship specialists - not kidding here either, they've made a business model out of censoring)

So why my interest in these sites then? Since the above leaves me with http://www.focus.de for getting some Germany-specific news, and the latter is a brainwashing site for 5-year-olds.


Again, I think a good ad-blocker should SIMULATE that the ads have been displayed, or then, stop any page-specific feedback altogether / send false data back.


Not knowing about AdGuard's possible advantages over the "free" alternatives. Again setting up an alternative system for such crap use like trialing software; my XP on that one just not being successful anymore with most modern software; last time I had bought a cheap W7 Pro version from some British crook, it seemed to work fine, I left a nice appreciation for the (commercial) vendor, and then, after some 30 days, my W7 stopped working, the MS screen telling me it wasn't legitimate, and then the vendor never wrote back upon my several reminders.

So never leave positive appreciations for somebody on ebay before 55 days!

(Currently installing some cheap W10 Pro version from some German commercial vendor (English version again - with any other version you just multiply problems of all kinds, not the least not remembering by which search terms you'll get help) just asking me to pay again by PayPal, after me having paid by PayPal upon purchase - let's see if PayPal took the money and ran, or if just vendor's accounting is shit.)

(I have to admit that when installing-activating the Brit W7, I had not thought about setting my VPN to Minuscule Britain, assuming MS - which doesn't pay much taxes within the EU but presumably insists on "users" only using their OS within the country of original commercialization - stops W versions if they detect they've left their "original" country, so this time, for activation of my new "alternative W10" now, I'll set it to Germany then; if after that, they even check for use later use, I'll be probably screwed again, sooner of later, not remembering to do ALL web access by alleged-German VPS afterwards, should certainly make it the default setting to begin with, in order to minimize risks...)

(So much info is floating around; in real life, you just happen to not remember all of it at every little moment.)

wraith808

  • Supporting Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 9,894
    • View Profile
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #4 on: December 05, 2018, 08:42 AM »
Again, I think a good ad-blocker should SIMULATE that the ads have been displayed, or then, stop any page-specific feedback altogether / send false data back.


One reason that they don't do this is bandwidth.  They tout saving bandwidth because they never actually touch the asset.

ital2

  • Member
  • Joined in 2017
  • **
  • default avatar
  • Posts: 108
    • View Profile
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #5 on: December 06, 2018, 02:47 PM »
@wraith808: I don't get this. In fact, the page itself (javascript) sends data back, telling the respective server that the ads have not been displayed, or something along these lines. Or/and the other way round, in case of successful display of the ads, the page sends some "ok" string (which may be quite elaborate, unfortunately).

So I understand that these "false positives" I'm asking for would be individual for each such site, which is why this could only be done for those "big" sites of if not universal then at least national appeal (in big nations), like the examples I gave above. Also, the site owners would frequently "update", i.e. modify those "got thru" or "were banned" strings; it'd be a little bit like downloading from Flickr and the like; frequent ad-blocker updates would be needed for given "standard sites", by subscription, for once.

Site owners could monitor what the ad-blockers did, by running a dummy pc with those ad-blockers installed; ad-blockers' developers could monitor the changes in the sent-back strings by running pc's with and without ad-blocking.

Sites would change those strings several times a day in the end; ad-blockers would not keep up (with sending updates).

So that's probably why that isn't done, but if it's NOT a mass market, but a quite confidential one, it'd be doable - the question arises of course how long a reasonably-priced such offer would remain confidential...

(Additional problem: The site owners could encode those strings in some individualized way, according to the page's url, title or the like; basic problem here: we all have accepted that downloaded pages (or even before download) also SENT BACK data; if that was not the case, they simply wouldn't know.)



EDIT: Misunderstanding since I hadn't had in mind everything from my original post. You mean that for processing the advert part(s) of the page, the ad-blocker should sent the page, from the user's pc to the ad-blocker's server, then send the purified core part back; what I had in mind was something like "whiting it all" (since the data is there already, so make it invisible at least, and non-interactive). -

I had in mind that this processing of the ads would be quite simple, technically, and that the ad-blocker, installed as browser add-on, would have the necessary code for doing this, installed on the user's pc: just enough javascript in order to successfully identify the ad-parts, and make them invisible.

Of course, and according to my further thoughts above, it's to be feared that this "whiting" is then detected by some (upgraded, for this new necessity) page code.

What I had in mind, was something less consequential than what's done today, and which would be less detectable, but I suppose that's illusionary, the (spiced-up) page code would be able to detect pixel colors (or its own code to be invalidated), and then some preset, encoded string could be sent.

Of course, there could be another ad-blocking trick being envisioned: NOT interfering up to the full page being displayed (so there is a slight annoyance for the user indeed), THEN "do-it-all", and block any further sending back info to the server, like "bad connection", up to the user asking for some new page, and then ditto as before - in this scenario, the server(s) would get that AFTER each full download, the connection gets bad, and make their conclusions...

Or then, creating (user-pc-sided) a virtual representation, and infering from that, the real one, the server(s) not "getting" that the real representation on the user's screen isn't identical to the virtual page they will have created.

In this context, let's remind ourselves that the ads often come from third-party servers, and that was the aspect I hadn't in my mind; you mean that for bandwidth minimization, the ad-blockers block the download of the ads already.

Hence:

I think there should be possible technical means for those prominent, "special" sites, that you can NOT see without ads today: If I want to see them currently (with a browser exception or in another browser), I have to convene to any download they want to force upon me, anyway, SO the ad-blockers, in these instances, by special option, should NOT block the download (i.e. my traffic would be the same anyway), just block the display (and the interactivity, too) - possibly, there ARE some means for this, along my ideas here, as soon as ad-blockers free themselves from the conception that blocking ads necessarily implies blocking their download, too.

Considering there are "prominent" sites which at the end of the day would be worth the traffic, just not all the visual annoyance.
« Last Edit: December 06, 2018, 04:37 PM by ital2 »

wraith808

  • Supporting Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 9,894
    • View Profile
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #6 on: December 07, 2018, 12:03 AM »
@wraith808: I don't get this. In fact, the page itself (javascript) sends data back, telling the respective server that the ads have not been displayed, or something along these lines. Or/and the other way round, in case of successful display of the ads, the page sends some "ok" string (which may be quite elaborate, unfortunately).

No, that's not how this works.  When you open a connection via http to get the ad, it streams it back in the browser.  What the ad blocker is doing is _not_ making that connection at all.  It could make the connection then abort, I suppose, but they can also see how much is transferred if you do that.  It could also stream the ad and not show it, I suppose, replacing the element in the page and streaming it using another connection.  But that would require you to download the ads, which again, is one of the advantages that they tout, i.e. we saved you this much bandwidth.

ital2

  • Member
  • Joined in 2017
  • **
  • default avatar
  • Posts: 108
    • View Profile
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #7 on: December 07, 2018, 08:57 AM »
"When you open a connection via http to get the ad" - I'd never do this! ;-) Also, your "they never touch the asset" was completely obscure for me - btw, I hereby copyright the comedy line "Don't touch my assets" (for "my private parts") - you meant they (the ad-blockers) don't download (what you call "stream"?) the ads, from third-party (i.e. not the page's) server? Or better, they intercept/block that download? ("not touching the asset" for "block the download of the ads" was too exotic for me, in order to understand, hence parts of my meanderings; in fact, the ads are the crap, the core page would be the "asset", but I would never tell it that...) ;-)

As far as my knowledge goes, the page is downloaded.

In its html, there is javascript which downloads ads from third-party servers.
(Also, some ads, here and there, can be included within the page html, but that's quite rare.)

Now, before the browser displaying the page (and thus following the download links in the html, too), the ad-blocker purifies the page script and gives it to the browser only then, for the browser to display what's left from the original page script.

Also, the page script sends back info to the page's server, in order for that server to check if everything has been displayed, incl. the ads; the ad-blocker cannot block that sending-back since then the server would know the ads have not been downloaded (and of course, the amount of downloaded (what you call "stream", or is that something different?) bytes can be monitored and sent back to the server.

Since that (in case, encoded) "everything's ok" info sent back to the server could only be sent after everything's fine, I even suppose the core info (i.e. the page to be displayed) isn't even included in what the server sends first, since if it did, the page could be displayed in full - all the info was there already -, even if then there was "problems" with the ad-loading: the server would be unable to stop the page display, even by knewing the ads weren't displayed - but could block the IP address for further downloads then.

The sent amounts of bytes downloaded (streamed???) would be identical, if the page is fully displayed (i.e. with the ads) or not (i.e. without the ads); the difference between download and stream, as far as my knowledge goes, is with download = stored in the browser cache and stream = just displayed on screen, without any storage on the user pc, right?

So I suppose that if download, not stream, there would be means of not displaying the (downloaded) ads, without the page's server knowing about this.

We convene that one of the tasks of the ad-blocker consists in blocking unnecessary traffic, i.e. blocking ads-download (stream???), hence the knowing of blocking of the page's server, by counting.

My argument above concerns pages which are sufficiently important for the user in order for them to accept the traffic caused by additional downloads (streams???), but where they do not want to be visually bothered with all that crap. Hence my idea that for "renowned" "standard" sites, a (reasonably-priced) ad-blocker could, by individual option for every one of these sites (the ones for Germany, I mentioned above), process these sites - IF that was possible along my ideas, without having to cope with every single page individually for that (which would be practically impossible, see above) - differently from the "regular" ones: allow for download/streaming of the ads, but kill/hide them afterwards, and without the page's server knowing about this.

Don't ask me about the advertizer's interest here since in fact, I happen to avoid them whenever possible if they had bothered me before, so it's to their advantage, at the end of the day, if I do NOT see their advertizing; I would not go so far as to say they pay for this favor, though, hence my idea to pay a reasonable fee p.a. for such a spice-up ad-blocker (20 bucks or so).

Also, sending back of info to the third-party, the ads', servers, would be possible, but here again, once they have been downloaded (streamed???) - for these special pages, those advertisers should NOT be able to detect their ads have not been displayed.

Errors of mine, ideas / info of yours? ;-)

wraith808

  • Supporting Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 9,894
    • View Profile
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #8 on: December 07, 2018, 11:59 AM »
"When you open a connection via http to get the ad" - I'd never do this! ;-)

O...k.  I was referring to from a programming perspective.

As far as my knowledge goes, the page is downloaded.


It is not.  The assets are processed separately, which is why you might get them at different speeds.  I'm telling you how things work on the internet from experience coding.  There are cases where the images are encoded in the streams, but they wouldn't be for this.  Each asset is streamed separately.

But I don't seem to be helping the conversation, so I'll bow out.

ital2

  • Member
  • Joined in 2017
  • **
  • default avatar
  • Posts: 108
    • View Profile
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #9 on: December 07, 2018, 06:17 PM »
"O...k.  I was referring to from a programming perspective." - and since I had perfectly grasped that, I had added the ";-)".

As for the rest, ok, no problem, I don't understand your telegram style anyway, my fault assuredly.


In order to clarify: I never meant the page is in one piece*, but there is a core html code of the page which triggers download of any sorts of elements, be that pics (mostly from the same server) or ads (mostly from other servers), and of course, these disparate elements are downloaded separately, but by the code in the "page source code" (which in case could trigger some server code which then only downloads those elements, or not), BUT there is some group of elements which, together, form the core page, and then there are unwanted elements, ads or others; the term "assets" here is obviously misused for "page elements", and "stream" is left as is, whilst it's obvious from the above there would have been some need to differentiate it from "download".

It's called "playing on words, in order to obfuscate, in order to display knowledge, without giving any knowledge away".
« Last Edit: December 07, 2018, 06:26 PM by ital2 »

wraith808

  • Supporting Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 9,894
    • View Profile
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #10 on: December 07, 2018, 07:07 PM »
Are you serious right now?

You can look up in relation to programming stream and find the following definition:

transmit or receive (data, especially video and audio material) over the Internet as a steady, continuous flow.

There is no need to separate that for any sort of download, as should be obvious from that definition.  Each one of those connections its own separate communication stream to be rendered in the browser as indicated by the tags.

And the word asset is not "misused", but a standardly accepted term to refer to anything linked from the web page, whether it's external CSS, images, fonts, video, etc.

I have no idea what you mean about "telegram style", other than to toss it out as an aspersion to the fact that I like to get to the point of the matter.  I'm not a long-form writer when it isn't needed, i.e. I don't throw words at something to get my point across, but try to make things as succinct as possible and elucidate complex subjects, knowing that people don't necessarily have time to read multi-page treatises on a simple subject.  To bastardize a phrase by Einstein, Any intelligent fool can obscure his speech with words to make explanations longer and more complex ... it takes a touch of genius -- and a lot of courage -- to move in the opposite direction.
« Last Edit: December 07, 2018, 07:43 PM by wraith808 »

ital2

  • Member
  • Joined in 2017
  • **
  • default avatar
  • Posts: 108
    • View Profile
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #11 on: December 08, 2018, 06:56 AM »
ATTN Sophists You Can Learn Something Here (no, not from me for once):

Even some 3-years-olds in the sandpit are able to play foul already; if you have such people as our boss, more than 20 years later, you're more or less done; change employer asap, without them letting the time to destruct in part your credentials.

When you do the "complete style" (or whatever you like to call it), you type lots of characters, and if you don't redact then - being paid for what you will have written -, there will not only be typos, but also sometimes some obscure elements where you will not have formulated your thought to the best; in many occurrences, the context at least will then indicate what you will have meant.

On the other hand, if you do the very sparse style, there is no context to explain if needed, so you better be perfectly clear, and the above specimens are such wonderful examples of being almost inscrutable that you invariably end up - at the very moment the author announces the break of the "conversation" - with thinking that that's been on purpose; as implied above, we know that style from within the corporate world when it's not about clarifying things together, but to butcher an opponent/subordinate.

For example, if you're able to make any sense of "There are cases where the images are encoded in the streams, but they wouldn't be for this.", you're really, really strong; being sparse, AND leaving out core elements then, is no "conversation" at all.

Btw - and not even mentioning the fact that the usual meaning of streaming is "sending data to the pc, without that data being stored on that pc" (so the usual downloaders cannot save it but by filming the screen) - e.g. pics are indeed embedded in a very weird way sometimes, e.g. in rtf where the whole pic code is simply replicated (copied) into the rtf text, just with some special code before and after that, in case, millions of characters describing the pic; on the other hand, I've never heard of such a thing in web programming, where the pics and other elements are not embedded within the page's code - would that be even possible? -, but linked, for separate download indeed (i.e. all the pic or other code is within its own, dedicated file), be that from the same server or from others; as said, some code complicates things in order to make downloads more or less impossible, by the http links referring not to the special files, but back to the php server code which then sends back (or not) the data, perhaps even just "streaming" it, so that it doesn't get onto your hdd (browser cache) - don't know how they make it, but doesn't matter here.

The above "conversation" is a classic exemple where one participant naively thinks of getting things clarified, whilst the other strives to make appear his "opponent" stupid, systematically just giving away tiniest bits, from which the naive participant tries to get some meaning, instead of saying, early on, if you express yourself in just some sparse words, make them meaningful; just the sudden aggression some sparse posts later will then open his eyes and make him discover the foul play he's being subjected, his naive thinking "he doesn't do it on purpose, he just expresses himself badly, so let's see if we can make any sense out of it anyway" prevents him from seeing the situation as it is.

When in the corporate world, such people begin to aggress you openly, in the end, it'll be already too late, so you're probably well-advised to never even enter such foul play of others, and that means, never try to obscure things but immediately say you don't understand - of course, and as you see here in this current example, they will say that's your fault, so be it, but break their foul play and get a new employer, but don't enter their scheme since you will end up with taking decisions "for them" they will then fervently disown, making you responsible for big losses for the corporation (of which these people give a shit, their agenda being a sadistic one).

Many of you will have (out)lived (or will have been broken in) such a situation, which is a frequent one in the corporate world (or, of course, in public administration); of course, you will not be aware of such a stratagem in "play" situations like ours here, but you should be, at any moment, in professional ones:

Trying to fill up the allegedly unintentional omissions of others is among the most dangerous situation you could ever encounter in your professional life; you'll find yourself on some icecap which then will break under your feet.

Of course it's a shame you'll need to be paranoid in order to survive, but there are indicators for WHEN it's necessary to enter paramode, and the above "conversation" is a classic example of the victim not having seen the strong indicator, so you can indeed learn something here indeed, and be it just for a refresh and reminder.


P.S. That's "website assets" indeed, but quite rarely "assets"; even on Quora, you'll find a "what the hell is that", and the problem doesn't lie within the (unusual) expression, but in the obvious and repeated (above) refusal to clarify, i.e. the will to maintain the misunderstanding.


EDIT: Oh, I forgot: In the old days, there were even school teachers for whom the day wasn't a successful one if, in the way described above, they hadn't made cry at least one of their pupils (10, 12 years old) in at least one of their classes. Those times are over now in civilized countries; corporations and administrations, as said, remain a jungle in which such predators can run rampant still.
« Last Edit: December 08, 2018, 07:06 AM by ital2 »

hollowlife1987

  • Honorary Member
  • Joined in 2006
  • **
  • default avatar
  • Posts: 91
    • View Profile
    • Donate to Member
Re: Why AdGuard? Why do they detect ad-blockers to begin with?
« Reply #12 on: December 08, 2018, 10:25 AM »
I've never heard of such a thing in web programming, where the pics and other elements are not embedded within the page's code - would that be even possible? -, but linked, for separate download indeed (i.e. all the pic or other code is within its own, dedicated file), be that from the same server or from others
-ital2

Yes it is possible and used most the time
<img src="example.png" />
the page code does not include the image directly although that is possible too.

Embedding the image would be more like this
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAADSCAMAAABThmYtAAAAXVB" />
Lets assume the web developer didn't base64 encode all his images. When loading a web page your browser opens up a few http/https connections to download/stream the resources (The browser downloads a stream of bytes for the image in this case)

Now the same can be done with javascript scripts
<script src="./ads.js"></script>

Now an adblock program of some kind may assume that ads.js contain well ads and prevent the browser from requesting the resource.
The server doesn't need to know if the resource was requested or not.   Assuming ads.js is an adblock detector script; if the ads.js was requested then the browser would execute the javascript and remove the content dynamically that shows the real page content vs the adblock in use content.  Only when that "fake ad" script gets loaded does the real content get shown.

While there will be exceptions to this, it is the general basis on how to detect adblock is to leverage the blocklists and load fake ad scripts that when blocked from being downloaded and executed the content doesn't show.

Now to the discussion about AdGuard. 
I noticed that some adblock detectors will still pick it up others won't.  Also the https filtering works by MITMproxy and installing its own root certificate into the system.