BOT traffic – How to detect fake hits and what to do about it!

One of the biggest annoyances for every paid-traffic affiliate marketer is dealing with the bot traffic, which results from the high amount of various BOT’s, that are surfing the internet. According to some estimates, more than half of all internet traffic is actually caused by BOTs in one form or another – crawlers, spiders, rippers … some are legit tools, some not so much.

There is not much we can do about the google crawler, for example, SE crawlers are an acceptable part of the internet ecosystem. There are many legit reasons for running automatic systems that mimic user behavior, so I’m not gonna complain about such tools at all.

But what really SUCKS, is BOTs that are deployed by fraudsters, in order to defraud advertisers and media buyers. There is a whole lot of cheaters, who create ton’s of sites with fake traffic and sell these BOT traffic impressions to various traffic networks.

To get an idea about how HUGE this problem actually is, read this Forbes article : https://www.forbes.com/sites/thomasbrewster/2016/12/20/methbot-biggest-ad-fraud-busted/

Some traffic networks do their best to detect this fake traffic, but many of them just turn a blind eye to the problem – every impression they can sell brings them profit, whether it’s real or fake … so some networks simply don’t care and act like the problem is not theirs to solve.

One way or another, BOTs don’t buy, so we don’t want them in our campaigns, or at least as low amount as possible. Since these clicks totally screw up the optimization process, is in the interest of every affiliate, to detect them and block placements/sources with too high % of fraudulent traffic.

When optimizing a new campaign, you need to make sure that you are working with human traffic, it’s completely pointless to attempt any optimization with fake BOT traffic – they won’t register, they “click” too much, they screw up all the data and deteriorate the click distribution …

That’s why it’s absolutely crucial to know how big % of the traffic is fake, what are the placements with the highest % bots … and block those.

SO HOW DO WE DETECT BOT TRAFFIC?

In all honesty, there is no 100% reliable method out there. Some bots are so advanced that it takes literally detective work to uncover them, and that’s not something we could do. There are specialized companies, who audit traffic and combat AD fraud, but their services are expensive … forensiq.com for example.

The good news is, large part of the BOTs is “stupid” and we can catch them with a few simple tricks. Let me give you a few tips on how to do it now. Please keep in mind that each of them requires some kind of a tracker, like Voluum, for example. You need this to be able to see what placements/sources are actually sending the BOT traffic.

The best is to use a tracker that supports “multi-offer campaigns” – you will be sending legit traffic to offer 1 and BOT traffic to offer 2. In voluum for example, you can handle this easily with using “click/1, click/2 URLs”. Every tracker is different, so I’m not gonna get into details here, you will need to consult it with the support of your tracking solution.

1. BOT click trap : Put “invisible” clickable links on your landing pages. These can be 1×1 pixel images, text using the same (or very similar) color as the background, links placed very low on the page (below the fold) in areas that are not visible (by disabling scrolling).

Link these invisible links to the offer #2, if it’s getting clicks, it’s caused by BOTs… simply because no human could possible notice them and stupid BOTs click on everything.

This is the oldest detection method, so some BOTs are designed to pass these traps – especially text links in the same color are often ignored, that’s why it’s recommended to use SIMILAR color and not the same. Same goes for the 1×1 links, but if you change it to 10×2 for example, humans will still not notice 🙂

2. Javascript Redirect : this is another simple but effective method. Many poorly coded BOTs simply don’t “read” javascripts, so they can’t execute it either. All you need to do here is to make a simple blank Landing Page, that will serve as an automatic redirect.

Put this code into the source code of the html file, into the <head> part:

<head>
<script>
window.location.href = “http://www.yourvoluumurl.com/click”;</script>
</head>

You have to replace the http://www.yourvoluumurl.com/click with your actual click url, depending on what tracker you are using.

In this case, some BOTs will not understand the redirect and they will not get to the offer. Real users will be redirected though. As a result, the higher the CTR of this LP will be, the more human traffic you are getting. In case of a 0% CTR of this LP, all the traffic were BOTs.

3. Timed JavaScript redirect + Bot click trap : you can also combine the two methods for more effective testing and higher detection certainty.

You can use any Landing Page for this basically, there has to be at least one click-able link in it though. Link this link/button to offer #2.

Then place the snippet below into the <head> part of the source code. Link it to offer#1 :

<script>
function redirect(){
window.location.href = “http://www.yourvoluumurl.com/click1“;
}
</script>

And put this code into the <body> part, you can change the number 200 to 300, 400 or so :

<body onload=”setTimeout(redirect, 200);”>
</body>

What’s gonna happen now is this : the LP will load, but after 200 ms (or other) it will automatically redirect to offer #1. The redirect happens too fast for a human to be able to click on any links on the LP, but BOTs can do that, so if you see clicks on offer #2, it’s from BOTs. And additionally, if you don’t see clicks on offer #1, it means there are BOTs that don’t understand the redirect … so it’s like a double check.

NOTE: Let me repeat that all these methods are simple and pretty basic, so they will not detect advanced BOTs. BUT, large number of BOTs are really this simple, so it’s still very effective to use such methods, especially when testing a new traffic source for the first time.

DO I HAVE TO KILL EVERY PLACEMENT/SOURCE WITH BOTS?

Let me tell you one thing : there is NOT A SINGLE traffic source is free of BOT traffic. It is everywhere 🙂 The problem is, when the % of fake traffic reaches certain amount. As a rule of thumb, it’s virtually impossible to make a placement work, when then % reaches 70% or 80% … so feel free to block such placements immediately.

The rest CAN work, but it makes optimization complicated.

Here is a simple approach that I’m using and it should work for you too. When starting a new campaign, I run a BOT test initially and cut every placement that gives me more bots than the AVERAGE of that particular source. It becomes pretty visible after a rather small amount of traffic.

It’s pretty much always following this pattern : some placements have super high % of bot traffic, majority is in the middle, then some have very low % of bots. I cut the super high % ones and set a thresh-hold based on the rest, simply by estimating a rough average. Then I cut everything that has more bots than this average.

Now I know that I’m working with the BETTER part of the placements of this particular source. At this point, I switch to regular landers and start with the normal optimization process. In case I’m able to get into profit, I unblock some of the placements that I initially blocked due to higher than average BOT % … some of them will work too.

A TIP FOR YOU:

Keep one thing in mind when buying traffic : it’s an auction! All standard media buyers, CPA marketers or other affiliates … they all buy the traffic because they are able to turn profit on it, and that is what sets the bid levels. All these people are buying the impressions including the BOT traffic, just as you, so this factor has already been calculated into the price.

If all BOTs vanished right now, the prices would simply go up quickly, because the traffic would suddenly become more productive. With this in mind : the presence of BOT is mainly a major optimization problem, it also causes financial loses, but the auction model keeps them down somehow.

Except for one type of advertisers – branding companies who only look at numbers, without actually measuring the effect of their campaigns. They will buy anything that looks good on paper. That’s why traffic networks love them so much and that’s why performance marketers hare them 🙂

Thanks for reading!

[grwebform url=”https://app.getresponse.com/view_webform_v2.js?u=Sym6E&webforms_id=8162402″ css=”on” center=”off” center_margin=”200″/]