Which is better Robots.txt or Meta Robots Tag?

[Sonamkumari]

Hello Friends,

Which is better Robots.txt or Meta Robots Tag?

[Dina44]

Using a meta robots tag also ensures that your link equity is not being lost, with the use of the ‘follow’ command.

Robots.txt files are best for disallowing a whole section of a site, such as a category whereas a meta tag is more efficient at disallowing single files and pages. You could choose to use both a meta robots tag and a robots.txt file as neither has authority over the other, but “noindex” always has authority over “index” requests.

[Barbara]

You can use robots.txt rules for general things, like disallow whole sections of your site. If you say Disallow: /family then all links starting with /family are not indexed by a crawler.

Meta tag can be used to disallow a single page. Pages disallowed by meta tags do not affect sub pages in the page hierarchy. If you have meta disallow tag on /work, it does not prevent a crawler from accessing /work/my-publications if there is a link to it on an allowed page.