+ Post New Thread
Results 1 to 2 of 2

Thread: What is the limit of a robot.txt file?

  1. #1
    Junior Member pihu147741's Avatar
    Join Date
    Jan 2018
    Posts
    27
    Post Thanks / Like

    What is the limit of a robot.txt file?



    Hello friends,

    What is the limit of a robot.txt file?

  2. #2
    Banned
    Join Date
    Jul 2014
    Location
    Forum
    Posts
    586
    Post Thanks / Like
    Google spiders crawl only 500 KB of the text file. So it is beneficial to limit your Robotx.txt file to only 500KB.

+ Post New Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
 Protected by : ZB BLOCK  &  StopForumSpam