See the top rated post in this thread. Click here

+ Post New Thread
Results 1 to 4 of 4

Thread: What is robots.txt?

  1. 09-11-2014, 04:49 AM #1
    eagleeye dental
    eagleeye dental is offline
    Junior Member eagleeye dental's Avatar
    Join Date
    Sep 2014
    Location
    India
    Posts
    11

    What is robots.txt?

    What is robots.txt?
  2. 09-15-2014, 09:21 AM #2
    comnetsolutions
    comnetsolutions is offline
    Junior Member comnetsolutions's Avatar
    Join Date
    Sep 2014
    Location
    singapore
    Posts
    4

    1/1 found it usefull/helpfull Is this useful / helpfull? Yes | No
    robots.txt file instruct web crawler how to crawl and index web page in website
  3. 09-16-2014, 08:57 AM #3
    Fli
    Fli is online now
    Administrator
    Join Date
    Mar 2013
    Posts
    2,763

    Is this useful / helpfull? Yes | No
    example of a crawler is "GoogleBot" (bot of Google search engine)
    Register to reply this topic
    Check ILF rules
    I live on ecological debt
  4. 09-16-2014, 08:17 PM #4
    Fli
    Fli is online now
    Administrator
    Join Date
    Mar 2013
    Posts
    2,763

    Is this useful / helpfull? Yes | No
    Example robots.txt file contents can be found via search engine like google: robots.txt example
    Register to reply this topic
    Check ILF rules
    I live on ecological debt
+ Post New Thread
« Previous Thread | Next Thread »

Similar Threads

  1. What is robots.txt?
    By himanshu in forum SEO - Search Engine Optimisation
    Replies: 6
    Last Post: 10-03-2019, 11:31 AM
  2. Which is better Robots.txt or Meta Robots Tag?
    By swatijain2233 in forum General Webmaster Discussion
    Replies: 7
    Last Post: 09-19-2019, 12:42 PM
  3. What is robots.txt
    By Sonamkumari in forum SEO - Search Engine Optimisation
    Replies: 1
    Last Post: 03-24-2018, 08:50 AM
  4. Which is better Robots.txt or Meta Robots Tag?
    By Sonamkumari in forum SEO - Search Engine Optimisation
    Replies: 2
    Last Post: 08-11-2017, 07:28 AM
  5. What is the use of robots.txt?
    By artijain1900 in forum SEO - Search Engine Optimisation
    Replies: 1
    Last Post: 09-23-2016, 01:42 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Forum Rules

 Protected by : ZB BLOCK  &  StopForumSpam