How to disallow/ban Google bot to view certain webpage

[Fli]

How to disallow Google bot to view a webpage?

Here is one idea:

<?
if(strpos($_SERVER['HTTP_USER_AGENT'],'google') !== false ) { header('HTTP/1.0 404 Not Found'); exit(); }
if(strpos(gethostbyaddr(getenv("REMOTE_ADDR")),'google') !== false ) { header('HTTP/1.0 404 Not Found'); exit(); }
?>

If user agent contains "google", then header 404 (not found) is sent and script is stopped.
If IP address host contains "google", then same thing happens.

---------
Similar topic, how to hide part of webpage from bots: https://internetlifeforum.com/html-css-forum/1743-how-hide-link-other-part-webpage-bots-like-googlebot/

[ELyon01]

Hey there! thanks for that code. Helped a lot.

[nesir]

Two way you can use you rbt.txt file , or you can use yout htaccess file on the root of your server ht acces would look like this :

RewriteEngine On RewriteCond %{HTTP_USER_AGENT} (googlebot|bingbot|Baiduspider) [NC] RewriteRule .* - [R=403,L] or

disallow google bor in rbt text file. Htacess is a better option as you can redirect google and also rbt.text makes google think you are hiding pbm networks.

Also you can use the noffollow , noindex tags for specific pages.

I should tell you that google visits pages under anonymous user agent strings from time to time , to see what you are up to , u can stop it from being indexed but it will get crawled eventually.

[nesir]

You can also use robots.txt to allow and disallow certain webpage on your website as required.

Yes but rbt.txt is passable by anyone who wants to , i know backlink bots can bypass it easily and often do. Google also does not obey the rbt txt . Easy way to to test this , set your rules in rbt text and htacess then fetch the page as google and see if google fetch it succesful. if youve disallowed google properly should come up with a 403 error. Also googlebots has a separate crawler for mobile so youd need to block that too. Google also indexes information from bing index so youd need to block bing aswell.

[virtueltime]

an other way is to add <meta name="googlebot" content="noindex" /> as suggested here developers.google.com/search/reference/robots_meta_tag

[vijaykhatri96]

an other way is to add <meta name="googlebot" content="noindex" /> as suggested here developers.google.com/search/reference/robots_meta_tag

I totally agree with this option but if you want to do that with the help of on-page coding then you can add a tag like <meta name="googlebot" content="noindex" /> That will block the request of crawling this page.

[Leasedlayer]

Here are a few examples

Robots.txt file URL: www.example.com/robots.txt
Blocking all web crawlers from all content
User-agent: * Disallow: /

Blocking a specific web crawler from a specific folder
User-agent: Googlebot Disallow: /example-subfolder/

[kumkum]

You can disallow Googlebot in robots.txt file. And you can also use this file and mention any other bots.
You can get more details in robotx.txt file here.