How can I block semalt.com crawler?
|
05-11-2014, 04:45 PM
Post: #1
|
|||
|
|||
How can I block semalt.com crawler?
I use sitemeter.com's free version, which shows me the last 100 visitors. Lately, on some of my infrequently read sites, 80 - 90 of the last 100 have been http://semalt.com/crawler.
They come from different URLs each time, as far as I can tell. It is distorting my visit count and hiding the real visitors. Is there some magic word I can put in my .htaccess file to block it? Ads |
|||
05-11-2014, 05:01 PM
Post: #2
|
|||
|
|||
# block visits from semalt.com
RewriteEngine on RewriteCond %{HTTP_REFERER} semalt\.com [NC] RewriteRule .* - [F] or you can contact with alex@semalt.com Ads |
|||
05-11-2014, 05:10 PM
Post: #3
|
|||
|
|||
RewriteCond %{HTTP_USER_AGENT} ^.*(semalt|otherbadbot|otherbadcrawler).*$ [NC]
RewriteRule . - [F,L] separate some bot that you want to block using pipeline | you don't need to write full name of the crawler only specific name like semalt, google, yahoo, baidu or other bot that you want to block. because it will be match with wildcard name wildcard |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)