This Forum has been archived there is no more new posts or threads ... use this link to report any abusive content
==> Report abusive content in this page <==
Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
My robots.txt are not workking?
06-04-2014, 08:07 AM
Post: #1
My robots.txt are not workking?
and they look like this, all commands are in one line
http://www.kshayaiklhd.tk/robots.txt

instead of robots.txt of http://www.google.com/robots.txt
I tried that too, and still it looks the same
I tried that too, and still it looks the same

Ads

Find all posts by this user
Quote this message in a reply
06-04-2014, 08:17 AM
Post: #2
 
delete this file andgenerate new robots.txt file and upload it in servers.

Ads

Find all posts by this user
Quote this message in a reply
06-04-2014, 08:20 AM
Post: #3
 
Delete current robots.txt file ...

create new robots.txt file -

User-agent: *
Disallow:
Disallow: /cgi-bin/
Find all posts by this user
Quote this message in a reply
06-04-2014, 08:33 AM
Post: #4
 
Deleting and re-creating the file won't fix the issue.

Any web browser will render the content in one single line because it considers the content received as HTML instead of mere TEXT—due to the CONTENT-TYPE value in the HTTP response header that is set by default to HTML as shown below in the copy of the HTTP header of the request to your robots.txt file (last line):

HTTP/1.1 200 OK
Date: Fri, 18 Jan 2013 21:04:45 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Content-Length: 148
Connection: close
Content-Type: text/html

To fix the issue, header should instead contain "Content-Type: text/plain" as in Google HTTP response header. To do so, you can explicitly set the MIME type associated to the .txt extension by adding the following line in your .htaccess file:

AddType text/plain .txt

If it still doesn't work, the configuration has to be done in the administrative interface your ISP provides you with to manage your website—look for a section talking about "MIME Types". If you're lost, contact your ISP, they'd be able to do it for your at no charge.

—Pascal Perry
Online Findability and SEO Strategist
Find all posts by this user
Quote this message in a reply
06-04-2014, 08:34 AM
Post: #5
 
Create a new one by following your competitors
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)