The issue of Ad-Blocking in our browsers.

[IainB]

I thought this was priceless. A rather fatuous and self-aggrandising post in http://blog.pagefair.com was somewhat pwned in classic manner in the comments - where is made a lot of sense arguably reflecting the feelings of a lot of users (including myself).
For posterity (in case it gets deleted), an .mht copy of the page is attached as a .txt file, if you want it (just change the extension to view it in a browser).
Detect Adblock_ Our Secret Sauce- PageFair Blog mht.txt (254.79 kB - downloaded 714 times.)

Detect Adblock: Our Secret Sauce
Published October 2, 2013 by Cody Beck

How do we do it?
[Image] How do they do it?How do they do it? [Discovery Channel]
We’re often asked how our adblock detection script works its magic: how do we detect that someone is blocking ads? Most people expect us to guard this secret closely, but the truth is we use an approach that’s widely discussed online. We observe what happens when a web page loads and detect the effects of adblocking plugins.

Understand Ad Blocking
In order to know what effects to look out for we need to understand how adblock stops ads from loading. The first technique used to block ads is to intercept requests from the browser to particular domains or for particular files. Most publishers use hosted ad servers that operate from well known domains; for example Google’s display ads are served from doubleclick.net. The adblock community maintains ‘filter lists’ of these domains that are updated regularly with the latest ad server domains. Filter lists also name particular files for which requests should be blocked regardless of domain; for example any javascript file called ads.js.

The second technique used to block ads is to hide ad-related page elements based on css rules. Publishers carefully design their web pages with space for both content and advertising, but when ads are blocked this could leave large, empty areas on screen. The adblock community’s filter lists specify page elements that should be hidden, for example any element with the ID ‘leaderboard-ad’. Page elements that match standard ad dimensions are also hidden. By hiding these page elements, adblock ensures that the space they would have taken up can be re-used by other parts of the page, such as the main page text. This has the bonus side-effect of also hiding any ads that slip by the first blocking technique.

Choose Your Bait Carefully
With these techniques in mind we insert bait elements into the page that adblock will attempt to block; including a javascript file, an image and an iframe. We then carefully observe what happens when a page loads. onLoad and onError events tell us if they’re successfully retrieved or if requests have been blocked. Their css style tells us if they are visible or have been hidden. We have run these tests billions of times, and have now refined them to the point that we can accurately detect when a user is blocking ads using adblock.

The Devil is in the Detail
As always, there’s more to this than meets the eye. Anyone who’s tried their hand at web development will be familiar with the frustration of cross-browser (in)compatibility and the challenge of staying current with a shifting landscape of browser and plugin technologies. Not to mention the challenge of  building a scalable server infrastructure that can handle vast quantities of analytics traffic in real-time.  We won’t bore you with complaints here though; hopefully you’ve now got enough information to understand what’s going on in the background when you sign up to use our free adblock measurement service.

Tags: adblock, detect adblock, technology
← Ad-news For Publishers
END OF POST============================

Comments:

    PhasmaFelis
    So has anybody ever tried to address the root cause of adblocker use, i.e. ads are really fucking annoying? I don’t like ads in general, nobody does, but that alone wouldn’t be enough to make me bother to install and maintain AdBlock. What does it is strobing “YOU MAY HAVE ALREADY WON” and animated “one weird trick” scams and softcore porn. This shit is *everywhere*, even on allegedly respectable news sites.

    If you want me to turn off AdBlock, you need to insist on reasonable, non-offensive, non-animated ads. If your ad provider doesn’t do that, get a better one, or lean on yours until they do. If this industry spent one-tenth as much energy pushing ad services for better quality standards as they do wringing their hands about ad blocking, shit would happen.
        http://brandonbrown.io/ Brandon Brown

        I whole-heartedly agree.
        oGMo

        Yeah seriously. And if your site refuses to load or whatever due to adblock, it wasn’t worth reading anyway. Your content is not that special. I’ll just go elsewhere.

        (Also the techniques discussed in the article are pretty much nothing special and exactly what I would have expected. Expect the next wave of adblockers to alter reporting for elements they block if this becomes necessary.)
            Rick Burgess

            How would you suggest paying for the content? are paywalls less annoying?
                PhasmaFelis

                I would, and did, suggest less offensive ads.
                    Rick Burgess

                    Lets be honest though, ad blockers originally came about to block pop-ups because they were horrible. pop-ups are dead on all but adult sites for the most part and what you have now are small text or image ads thats really aren’t a big deal.

                    You could also argue that even if they made ads that were less “offensive” you would never see them because of your blocker :p
                http://www.trisweb.com/ Tristan

                I suggest a simple and universal “tip” service. You would dedicate a certain amount, such as $5 per month, to be used for tips, and every time you click “tip” (a universal and recognizable UI) it’s recorded. Your $5 budget is then split amongst all the tips clicked that month.

                Obviously you could do a certain micro-payment amount as well if required. Or a simple pay-switch (rather than a wall) — the key is to make it a seamless and universal experience, as easy and ubiquitous as this Disqus form, so you’re not inconveniencing users at all.

                Add a “pay wall” and people will not climb over it. Hell no. But make it easy and fun to pay for content, and people will embrace it and feel good about it.

                This is a UX problem, not an economic problem.
                    Toranaga

                    marketing is the tax you pay for not being interesting. Pony up!
                    Rick Burgess

                    I agree that solution would be better in theory, if people were to actually use it. I fear that the majority of users are used to what they perceive as free content online and will simply not pay if they don’t have to.

                    We have the paypal donate type functionality which has been around for years and has a standard (although not nice) UI but I would guess (as i have no data to back it up) that the actual donation rate is pretty low.

                    The only way I can see a tipping type service working is if there is some benefit to the user for doing so, much like subscriptions on twitch.tv for example.
                oGMo

                First you need actual content. This means not a link chain to some other site or some blathering commentary piece. Hint: If I can skip your site and find the same or very similar content in the next link down the chain, your content is worthless.

                Yes, this means doing real work. This means having something to actually say, some research or something of value you’ve actually done. Then a paywall isn’t even necessary: I’ll subscribe to you even if I block your ads. For instance, I subscribe to places like di.fm and Destructoid which provide real, actual content I can’t find elsewhere.

                If you can’t be bothered to do the work and you just have drivel that no one is even willing to see an ad to read, your “content” doesn’t deserve monetary support. However, if you have quality content, you will get support.
        pagefair

        We agree that intrusive ads are bad! In fact we highlighted this issue in a previous blog post ‘Dealing With Adblock: 5 Options That Don’t Work’. The problem is that existing alternatives are bad for both publishers (less revenue) and web users (less access to information). The ad industry is gigantic and sadly change is slow to happen.
        Justizin

        Having worked at (and left in disgust) an ad-driven company, the answer is simple and clear: The most annoying ads yield the best click-through rates, but I can’t possibly believe they yield the best consumers. Companies like Google who once tried to challenge the shittiest ad strategies are now serving them up, and many companies relying on those ads have offices full of people using AdBlock “because our fucking site doesn’t load otherwise”. I think we should organize to boycott sites with the most visually distracting and CPU intensive ads (FLASH).

        What would astound you, BTW, is that for direct sales, you can barely sign an agreement anymore without a large % of video ads, so sites with no substantial video content have to invent an excuse to have video content and TRY THEIR BEST to distract their users away from the actual site, to watch these 1:00 ads on top of :15 video clips, sometimes bought wholesale.

        It’s abhorrent and it’s a fucking ponzi scheme, but it pays. :/
    kjs3atl

    An even more important reason to run an ad blocker is that the ad networks have become very effective malware distribution mechanisms. I have huge numbers of events like the following (edited slightly for readability):

Code: Text [Select]

1. GET /7f01baa99716452bda5bba0572c58be9/afr-zone.php HTTP/1.1::
2.     ~~Accept: text/html, application/xhtml+xml, */*::~~
3.     Referer: http://www.huffingtonpost.co.uk/_uac/adpage.html::
4.     ~~Accept-Language: en-US:: ~~
5.     User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64;Trident/5.0)::
6.     ~~Accept-Encoding: gzip, deflate::~~
7.     Host: delivery.globalcdnnode.com::
8.     ~~Connection: Keep-Alive::~~::~~

   See that referer? HuffPo ad launch page. That host? That’s not a CDN, it’s a malware farm registered to some guy out of RU. That GET? It eventually leads to a Darkleech exploit toolkit.

    We’ve seen these coming from all sorts of legitimate sites and ad distribution networks. We’ve had to block a couple of big ad networks in our web hygiene proxies, and they’ll likely stay blocked until they clean up their content, and that’s probably going to take a lot of big sites to make them.

    Haven’t had a single complaint from the users, tho.
    ende

    We’ve tried the same technique to detect ad blocker and then show the people an unobtrusive “Please switch off the ad blocker, that’s how we are paid”-banner. It took 3 days for the ad blockers to adjust the rules to not block the bait. It seems the only way to have even such an unobtrusive banner displayed is pay ABP for the exception.
    flamer96845312

    Good job linking to jQuery’s API when talking about JavaScript events.
    Also, if you start complaining about compatibility issues when talking about a few lines of JS, you must be quite the pro.
    Don’t write “technical” posts to attract clients when they’re only a display of how skillless you are.

    Tweets by Pagefair
    tweets

[Stoic Joker]

   PhasmaFelis
    So has anybody ever tried to address the root cause of adblocker use, i.e. ads are really fucking annoying? I don’t like ads in general, nobody does, but that alone wouldn’t be enough to make me bother to install and maintain AdBlock. What does it is strobing “YOU MAY HAVE ALREADY WON” and animated “one weird trick” scams and softcore porn. This shit is *everywhere*, even on allegedly respectable news sites.

    If you want me to turn off AdBlock, you need to insist on reasonable, non-offensive, non-animated ads. If your ad provider doesn’t do that, get a better one, or lean on yours until they do. If this industry spent one-tenth as much energy pushing ad services for better quality standards as they do wringing their hands about ad blocking, shit would happen.

-Comments:

I'd say they nailed it quite nicely there. It infuriates me to no end when a page freezes for x seconds because so lame assed ad element hasn't managed to drag its poorly coded or BW provisioned carcass into memory. Hay add companies...you have 3 seconds to lose my attention...so keep it light, as in ultra light, ultra fast, ultra quiet. I don't need to have to keep cleaning my chair because I just shit myself after getting screamed at by some idiotic video in a silent room when I'm researching something in the middle of the night.

[Renegade]

Ads suck. Sponsors are better. And actually having something to sell is best.

Check out the right column at Techdirt. Some sponsors, affiliate programs, and 1 ad. Amazon book recommendations are really good there - topical.

Torrent Freak has affiliate programs.

Infowars sells products.

The Oatmeal sells products.

The Activist Post has mostly affiliate programs, a few ads.

Explosm has an ad, but sells products.

Free Domain Radio has no ads and runs entirely on donations.


As Sam Axe might say, you know PageFair customers, a bunch of whiny little bitches.

"I can get eyeballs" is not a real business model. These whiners would be better off actually offering some kind of service that people are willing to pay for. Ads just mean that people might be willing to pay for someone else's product.


The venom in the comments at the article there is pretty much to be expected. No surprises.

One good comment there:

But make it easy and fun to pay for content, and people will embrace it and feel good about it.

This is a UX problem, not an economic problem.

Buddy nails it. What it turns out to be, well, "easy and fun" is the key. That just requires some thought and creativity. (I think the biggest burden there is that payments are horribly difficult at the moment, and the only truly easy way to pay is through Bitcoin.) So, is that possible? Maybe. I have a feeling that it would require a huge amount of effort though in order to get a mainstream payment system that worked easily.

[Renegade]

I don't need to have to keep cleaning my chair because I just shit myself after getting screamed at by some idiotic video in a silent room when I'm researching something in the middle of the night.

-Stoic Joker (October 03, 2013, 06:40 AM)

If that has happened to you, then you know what "murderous rage" means.

[TaoPhoenix]

For other reasons, I am discovering that headphones on the desk stops this problem! On the few times I think I hear sound. I put them near my ear, then go "oh. you. right."

This service brought to you by a collaboration of your earphone and diaper departments! Posted on DonationCoder. Mouser owns DonationCoder. If you have a mouse, give it to Mouser! That's what he does! (When he's not working on Mewlo, which requires a cat.) Ever notice the disclaimers in ads are getting longer? That's Don Lapre's fault! But he passed on.

[40hz]

I'd be happy to pay for good content I'm interested in. I've paid for site access before, am paying some sites now, and remain completely open to paying for more content in the future.

Too bad there's so little content out there I personally consider worth paying for.

I suppose that's my acid test: If a certain site switched to subscription only - would I buy a subscription? If the answer is "no" then it's nothing I won't walk away from in a heartbeat if the "monetization" strategy the site is using becomes too obnoxious.

The sad truth (IMHO) is that very little of the information buffet making up today's Internet is worth paying for. Most of it is amateurish, badly researched (if researched at all), poorly presented, and painfully shallow.

If the Internet is a vast info-ocean, it's an ocean that's 10,000 miles wide - but only about a quarter-inch deep in most places.

Or so it seems to me.

[TaoPhoenix]

I'd be happy to pay for good content I'm interested in. I've paid for site access before, am paying some sites now, and remain completely open to paying for more content in the future.

Too bad there's so little content out there I personally consider worth paying for.

I suppose that's my acid test: If a certain site switched to subscription only - would I buy a subscription? If the answer is "no" then it's nothing I won't walk away from in a heartbeat if the "monetization" strategy the site is using becomes too obnoxious.

The sad truth (IMHO) is that very little of the information buffet making up today's Internet is worth paying for. Most of it is amateurish, badly researched (if researched at all), poorly presented, and painfully shallow.

If the Internet is a vast info-ocean, it's an ocean that's 10,000 miles wide - but only about a quarter-inch deep in most places.

Or so it seems to me.

-40hz (October 03, 2013, 11:16 AM)

Not seems ... is.

You eval it on a site by site basis. X site becomes known for x1 stuff. Y site becomes known for Y1 stuff. As joked by xkcd and quietly acknowledged by game show producers, the internet is good at "factoids". Books (and "post-books" etc) still seem to reign for deep knowledge. There's a reason I have 2000 books on my shelf ... because the internet can't yet match any one of them in sequential order. Yes, if I spent 100 hours carefully building 1400 search queries I might slowly assemble one, but ... see?

Books *by definition* have X amount of knowledge! (Yes, white space etc, but I'm Anti-WhiteSpace. Rant elsewhere.) So that is/was what a bookstore used to be for ...
A. You didn't know X book existed, and you can't search for what you don't know.
B. Yay it exists. So you can look at *all of it*. Limited basically only by store hours and maybe in a few cases a hyper manager. With some practice you can get good at speed-evals. If the book keeps impressing you every five pages for 400 pages... you buy it. Simple. None of the DRM limited junk where chapters 1, 8, 14, and 22 are good and the rest are junk...

Ad-infested layouts really make "content" seem more than it is. Yes, they are passably well designed. Blocked almost right, fills a screen page, etc. But yes, once you actually look at the "article", especially if you play the game where you copy it into notepad/other, it's pretty thin! Even with all the Ad blockers on, the page wastes space with self-promotion junk. Like Me, Feeds, Twitters, and more.

[TaoPhoenix]

This begins to overlap with my threads on the anti-javascript plugins. "Ad blocking" increasingly means site-served stuff. Forbes is coming to mind with horrible "sliders".

[CWuestefeld]

As others have said, it's the annoying ads that are the problem.

I run AdBlock Plus in my browsers. But I've disabled it for DC, and for some other sites. I don't object to ads as such, and indeed, when they're topical, they can even be useful.

But ads that interfere with my usage are the problem. Animated ads that demand your attention still are common. My wife uses Chinese pages all the time, and these frequently look like someone vomited on them; I've added dozens of custom rules to ABP for her.

At a minimum, if you don't allow the ads to detract from the experience of the web page you've worked so hard on, then I'm willing to go along with it. But when you let the ads try to invade my experience, then I don't want to allow that.

[Edvard]

The saddest part:

Having worked at (and left in disgust) an ad-driven company, the answer is simple and clear: The most annoying ads yield the best click-through rates

I can believe it, thoroughly and whole-heartedly, after talking with absolute idiots doing bizarre things with their cellphones and expecting me at customer service to fix it, after multiple times telling my mother-in-law what's OK and not OK to click while surfing political sites and I still have to spend 3 hours every time I visit because "the computer is running a little slow, can you take a look at it?" only to find that not only is it slow, but half the start menu and desktop icons are gone because "We have detected a virus on your computer.  Totally clean the virus now for only $29.99!", and taking a handful of Tylenol after reading my mother's posts on Facebook "This is not a hoax!  Confirmed by Snopes... "