+ Post New Thread
Results 1 to 4 of 4

Thread: How to disallow/ban Google bot to view certain webpage

  1. #1
    Fli
    Fli is online now
    Administrator Fli's Avatar
    Join Date
    Mar 2013
    Posts
    2,471

    How to disallow/ban Google bot to view certain webpage



    How to disallow Google bot to view a webpage?

    Here is one idea:
    Code:
    <?
    if(strpos($_SERVER['HTTP_USER_AGENT'],'google') !== false ) { header('HTTP/1.0 404 Not Found'); exit(); }
    if(strpos(gethostbyaddr(getenv("REMOTE_ADDR")),'google') !== false ) { header('HTTP/1.0 404 Not Found'); exit(); }
    ?>
    If user agent contains "google", then header 404 (not found) is sent and script is stopped.
    If IP address host contains "google", then same thing happens.

    ---------
    Similar topic, how to hide part of webpage from bots: https://internetlifeforum.com/html-c...ike-googlebot/

  2. #2
    Junior Member ELyon01's Avatar
    Join Date
    Jun 2017
    Posts
    5


    Is this useful / helpfull? Yes | No
    Hey there! thanks for that code. Helped a lot.

  3. #3
    Member nesir's Avatar
    Join Date
    Sep 2017
    Location
    South Africa
    Posts
    87


    Is this useful / helpfull? Yes | No
    Two way you can use you rbt.txt file , or you can use yout htaccess file on the root of your server ht acces would look like this :

    RewriteEngine On RewriteCond %{HTTP_USER_AGENT} (googlebot|bingbot|Baiduspider) [NC] RewriteRule .* - [R=403,L] or

    disallow google bor in rbt text file. Htacess is a better option as you can redirect google and also rbt.text makes google think you are hiding pbm networks.

    Also you can use the noffollow , noindex tags for specific pages.

    I should tell you that google visits pages under anonymous user agent strings from time to time , to see what you are up to , u can stop it from being indexed but it will get crawled eventually.

  4. #4
    Member nesir's Avatar
    Join Date
    Sep 2017
    Location
    South Africa
    Posts
    87


    Is this useful / helpfull? Yes | No
    Quote Originally Posted by RH-Calvin View Post
    You can also use robots.txt to allow and disallow certain webpage on your website as required.

    Yes but rbt.txt is passable by anyone who wants to , i know backlink bots can bypass it easily and often do. Google also does not obey the rbt txt . Easy way to to test this , set your rules in rbt text and htacess then fetch the page as google and see if google fetch it succesful. if youve disallowed google properly should come up with a 403 error. Also googlebots has a separate crawler for mobile so youd need to block that too. Google also indexes information from bing index so youd need to block bing aswell.

+ Post New Thread

Similar Threads

  1. How much earn per youtube view?
    By shreeharisg in forum Website Monetization
    Replies: 6
    Last Post: 07-23-2019, 04:59 AM
  2. How to Disallow links in google..?
    By ruhikhan in forum SEO - Search Engine Optimisation
    Replies: 2
    Last Post: 07-17-2019, 09:29 AM
  3. How many type of website View..?
    By pihu147741 in forum Introductions
    Replies: 0
    Last Post: 03-13-2018, 06:37 AM
  4. Replies: 0
    Last Post: 09-29-2015, 01:01 PM
  5. Replies: 0
    Last Post: 01-18-2015, 03:24 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
 Protected by : ZB BLOCK  &  StopForumSpam