Affordable SEO by SEOClerks
See the top rated post in this thread. Click here

+ Post New Thread
Results 1 to 5 of 5

Thread: What is robots.txt?

  1. #1
    Junior Member eagleeye dental's Avatar
    Join Date
    Sep 2014
    Location
    India
    Posts
    11
    Post Thanks / Like

    What is robots.txt?



    What is robots.txt?

  2. #2
    Junior Member comnetsolutions's Avatar
    Join Date
    Sep 2014
    Location
    singapore
    Posts
    4
    Post Thanks / Like


    1/1 found it usefull/helpfull Is this useful / helpfull? Yes | No
    robots.txt file instruct web crawler how to crawl and index web page in website

  3. #3
    Administrator Fli's Avatar
    Join Date
    Mar 2013
    Posts
    2,410
    Post Thanks / Like
    Blog Entries
    1


    Is this useful / helpfull? Yes | No
    example of a crawler is "GoogleBot" (bot of Google search engine)

  4. #4
    Administrator Fli's Avatar
    Join Date
    Mar 2013
    Posts
    2,410
    Post Thanks / Like
    Blog Entries
    1


    Is this useful / helpfull? Yes | No
    Example robots.txt file contents can be found via search engine like google: robots.txt example

+ Post New Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
 Protected by : ZB BLOCK  &  StopForumSpam