This Forum has been archived there is no more new posts or threads ... use this link to report any abusive content
==> Report abusive content in this page <==
Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
robots.txt file and SEO?
06-15-2014, 09:07 PM
Post: #1
robots.txt file and SEO?
I currently have the standard robots.txt file that Wordpress includes in their themes. As a result many of my pictures on the website have been indexed by google. If I add a robots.txt file to my site now, will the 404 errors hurt my rankings? Will adding a robots.txt file like the one below help my site in terms of page rank and SERP's?

User-agent: *
Disallow: /wp-content/
Disallow: /wp-content/plugins/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-
Disallow: /trackback/
Disallow: /cgi-bin/
Allow: /wp-content/uploads/
User-agent: Googlebot
Disallow: /*.pdf$
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.cgi$
Disallow: /*.xhtml$
Disallow: /*.php*
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.txt$
Disallow: /*?*
Disallow: /wp-*
Disallow: */trackback/
Disallow: /cgi-bin/
Disallow: /go/
Allow: /wp-content/uploads/
User-agent: Googlebot-Image
Allow: /*

Ads

Find all posts by this user
Quote this message in a reply
06-15-2014, 09:17 PM
Post: #2
 
That robot txt file will more than likely hurt your rankings, you're blocking off a lot of important things which when indexed can help you get traffic, like for example - your images. I don't recommend it. You're also telling it not to index ".php" files --- if i recall, wordpress is a php based engine, you might end up preventing your website from being indexed.

I would stick with the default robots txt.

Ads

Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)