Affordable SEO by SEOClerks
See the top rated post in this thread. Click here

+ Post New Thread
Results 1 to 4 of 4

Thread: What is robots.txt?

  1. #1
    Junior Member eagleeye dental's Avatar
    Join Date
    Sep 2014
    Location
    India
    Posts
    11

    What is robots.txt?



    What is robots.txt?

  2. #2
    Junior Member comnetsolutions's Avatar
    Join Date
    Sep 2014
    Location
    singapore
    Posts
    4


    1/1 found it usefull/helpfull Is this useful / helpfull? Yes | No
    robots.txt file instruct web crawler how to crawl and index web page in website

  3. #3
    Administrator Fli's Avatar
    Join Date
    Mar 2013
    Posts
    2,455


    Is this useful / helpfull? Yes | No
    example of a crawler is "GoogleBot" (bot of Google search engine)

  4. #4
    Administrator Fli's Avatar
    Join Date
    Mar 2013
    Posts
    2,455


    Is this useful / helpfull? Yes | No
    Example robots.txt file contents can be found via search engine like google: robots.txt example

+ Post New Thread

Similar Threads

  1. What is robots.txt?
    By himanshu in forum SEO - Search Engine Optimisation
    Replies: 6
    Last Post: 10-03-2019, 11:31 AM
  2. Which is better Robots.txt or Meta Robots Tag?
    By swatijain2233 in forum General Webmaster Discussion
    Replies: 7
    Last Post: 09-19-2019, 12:42 PM
  3. What is robots.txt
    By Sonamkumari in forum SEO - Search Engine Optimisation
    Replies: 1
    Last Post: 03-24-2018, 08:50 AM
  4. Which is better Robots.txt or Meta Robots Tag?
    By Sonamkumari in forum SEO - Search Engine Optimisation
    Replies: 3
    Last Post: 08-11-2017, 07:28 AM
  5. What is the use of robots.txt?
    By artijain1900 in forum SEO - Search Engine Optimisation
    Replies: 1
    Last Post: 09-23-2016, 01:42 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
 Protected by : ZB BLOCK  &  StopForumSpam