What Is A Noindex Tag?

Seobility

Date First Published: 16th October 2022

Topic: Web Design & Development

Subtopic: SEO

Article Type: Computer Terms & Definitions

Difficulty: Medium

Difficulty Level: 5/10

CONTENTS

Blocking Specific Bots

Learn more about what a noindex tag is in this article.

A noindex tag is a meta tag used to instruct search engine bots not to index a specified page. It is located in the head of the HTML document and prevents the URL from being included in the search results. Indexing on a website can be controlled on a page-by-page basis and it only applies to one specific URL. Web crawlers have to crawl pages in order to see the tag.

The types of pages that website owners often do not want to show up in the search results include:

Error pages
Success message pages
Checkout pages
Webpages that are in development
Private webpages
Webpages with sensitive information.

Website oenwers would not want people to find them in the search results and there is no purpose of those pages showing up there. An example of the noindex tag can be seen below:

<html>

<head>

<title>Don't index this page</title>

</head>

</html>

Noindex tags can be manually written in a HTML document or when using a CMS, such as WordPress, they can be added by clicking on an option in a dropdown list in SEO tools, such as Yoast SEO, reducing the chances of human errors. If the page a website owner is setting to noindex is already in the search results, it might take some time for the page to disappear from the search results as search engine bots will have to recrawl the page to see the noindex tag.

Note:

In order for the noindex instruction to be effective, the page must not be blocked by the robots.txt file. If the web crawler is blocked by the robots.txt file or cannot access the page, the crawler won't see the noindex instruction and the page can still appear in search results if other pages link to it.

Blocking Specific Bots

If a website owner wants to block specific bots from indexing pages, they can change the text on the 'meta name' property to the correct name of the bot. Googlebot and Bingbot are examples of bots. For example, to block Google from indexing a page, the website owner can add the following tag:

Use this to link to this article:

HTML: <a href="https://computerhelp4all.com/articles/what-is-a-noindex-tag.html">Computerhelp4all</a>

Plain link: https://computerhelp4all.com/articles/what-is-a-noindex-tag.html