How referrer spam affects search engine rankings

A question I receive fairly often is: Wordfence ran a scan on my site and found a known malicious URL in one of my files. The file is a backup of my database or a log file. Where does this URL come from, what is it’s impact and what should I do about it?

I’d like to explain this in detail because it will help you understand referrer spam and what you see in your real-time traffic.

A common tactic on the Net that malicious webmasters use is to try and publicize their site by engaging in referrer spam. What this means is that they will create an automated web crawler much like Google’s Googlebot. They then send this crawler out to visit hundreds of millions of websites and the crawler pretends to be a normal web browser. The user-agent string (identification) that the crawler sends is Chrome or Internet Explorer or another human looking web browser. But the important difference is that it sends a fake referrer string to any website it visits. That means that it tells the logging application on any website it visits that it arrived from a website that it didn’t actually arrive from. Lets call that site example.com.

The effect is that there appears to be a fake person going around visiting a hundred million websites and arriving at those websites from someone else’s website.

You and I both enjoy looking at which websites are sending us traffic. When the bad crawler arrives at our site and visits a few hundred pages, our logging program will log that it came from example.com and of course we’re going to get very excited about going to visit example.com when our logging program tells us that example.com is sending our site all this new traffic. So we visit the site….

If it’s a phishing site we might get fooled into giving away some sensitive information. If it’s a hacking site we might end up getting our web browser hacked and malware installed on our machine.

If you see a new website in your list of referrers in your logging application, visit it using Chrome which uses Google’s safe browsing list to block bad sites. Or you can type the URL into this box and hit the button to get a report on it:

Referrer spam has other effects. Occasionally referrers are displayed publicly on websites or they accidentally end up in logfiles that are visible to the public and to Googlebot. That sends example.com (the malicious website) even more traffic because it gets a boost in the search engine rankings thanks to the new backlinks and it gets more visitors who see the URL appearing on some random web page and wonder what it is and pay it a visit.

So that is what referrer spam is and why people engage in it, particularly malicious websites.

When our WordPress administrators do a scan with Wordfence, we look for malicious URL’s in all your files. Occasionally we’ll find one in a log file that we scanned or in a backup of your database. I receive emails from admins from time to time thinking that they’ve been hacked when a malicious URL shows up in a file. It’s quite normal to find malicious URL’s in your log files or database backups because unfortunately referrer spam is commonplace. The danger comes from accidentally storing those log files or DB backups in a directory that is publicly accessible. You must NEVER do this. Not only will your sensitive data become public, but Google will start indexing that file with all those malicious URL’s and you may incur a severe search engine ranking penalty. This sounds like obvious advice, but you’d be appalled at how many people install a backup plugin or logging application and use the default settings which inadvertently delivers backups or log files into a public directory.

So in conclusion:

  • Referrer spam has sadly become commonplace and you will eventually find a malicious URL appearing in your database or in a log file.
  • As long as that log file or database record is not visible to the public and to crawlers, it won’t affect your search engine rankings.
  • Be careful which sites you visit when viewing your logs and referring websites.
  • Store your log files and database backup files in a secure private directory.

Did you enjoy this post? Share it!

Comments

4 Comments
  • When I originally commented I clicked the "Notify me when new comments are added" checkbox and
    now each time a comment is added I get several emails with the same comment.
    Is there any way you can remove people from that service?
    Many thanks!

    • Here's a thread that help resolve this: http://en.forums.wordpress.com/topic/unsubscribe-from-comments-1

  • Thank you so much for all of the info you guys share!

    Unfortunately, I've been dealing with these stupid referr spam for a year now. It's not just a little problem for me... These people filled up the whole 1st page of my google analytics! And they don't just do this once or twice... Most of the ones attacking my site say that they've visited 100-500 times- which really messes with your stats.

    Another bad thing about referrer spam, is that it affects your overall bounce rate. The bounce rate that appears next to these sites in your analytics is always 100%.

    What I had to do was download a plugin that has really really helped me keep these things under control.

    This is where my My question comes into play- the plugin that I use, has an easily accessible list of the reported spam referrals...
    Is that what you were saying not to do?
    (I've been blogging for over 2 years- but this part of it will always remain a mystery to me lol).

    Thank you again for your help! And I hope to hear back from you :-)

    Have a great weekend!
    -Heather Lynn