What is noindex in a web page?
Noindex is a value used in the robots meta tag of a page's HTML code. It tells search engines like Google, Bing, and Yahoo not to include that URL in their search index. When Google encounters a noindex tag, it treats it as a directive and will not show that page in its results.
The counterpart of noindex is index, which explicitly allows indexing. That said, you rarely need to use it, since search engines interpret the absence of any tag as permission to crawl and index the content.
It is also worth clarifying something that confuses many people: a noindexed page is not deleted and does not become inaccessible. Users can still reach it via a direct URL or through internal links. It simply will not appear in search engine results pages (SERPs).
Why the noindex directive matters for SEO
The noindex tag gives you precise control over what gets included in a search engine's index. This is more powerful than it sounds. Not every page on your site deserves to rank, and forcing search engines to evaluate low-quality or duplicate pages can hurt your overall performance. Using noindex strategically has two main benefits:
- Better indexing quality: you keep only your most relevant, valuable pages in the index, which can improve how search engines perceive your site overall.
- Crawl budget savings: search engines allocate a limited amount of resources to crawling each site. By preventing bots from indexing low-value pages, you help them focus on the pages that actually matter. This is especially important for large sites with thousands of URLs.
For these reasons, noindex remains one of the most frequently used tools in any technical SEO workflow.
How to implement the noindex tag
There are two ways to implement noindex: via a meta tag in the page's HTML, or via an HTTP response header. Both produce the same result, so choose the method that fits your setup best.
Meta tag
To prevent search engines from indexing a page, add the following tag inside the <head> section of your HTML:
<meta name="robots" content="noindex">You can also target a specific search engine bot instead of all bots. For example:
<meta name="googlebot" content="noindex">
<meta name="bingbot" content="noindex">HTTP response header
Instead of a meta tag, you can use an X-Robots-Tag HTTP response header. This method is particularly useful for non-HTML resources such as PDFs, images, and videos, where you cannot place a meta tag in the source code. An example HTTP response with a noindex header looks like this:
X-Robots-Tag: noindex
Both methods are recognized by Google and other major search engines. Use the one that is easier to manage given your site's technology stack.
Combining noindex with other directives
Noindex does not have to be used alone. You can combine it with other directives to get more granular control over how bots interact with a page.
noindex, follow
This combination tells search engines not to index the page, but still allows them to follow the links on it. This means any link equity or signals can still pass through to other pages on your site.
<meta name="robots" content="noindex, follow">Use this when a page should not appear in search results but contains links to other important pages you want bots to discover and evaluate.
noindex, nofollow
This combination instructs search engines to neither index the page nor follow any of its links. It is a stronger directive that effectively tells bots to ignore the page entirely.
<meta name="robots" content="noindex, nofollow">This is appropriate for pages like internal admin areas, login pages, or any page where you do not want any SEO signal passing through at all. As a shortcut, the value none is equivalent to "noindex, nofollow" and can be used in both meta tags and X-Robots-Tag headers.
Noindex vs. disallow in robots.txt
This is one of the most misunderstood areas in technical SEO, and it is critical to get right.
Disallow is a directive used in your robots.txt file. It tells search engine bots not to crawl a URL. Noindex, on the other hand, allows the bot to access and crawl the page but instructs it not to index it. These two directives work at different stages of the process:
- Disallow: the bot is blocked before it even reaches the page.
- Noindex: the bot visits the page, reads the tag, and decides not to include it in the index.
Here is the critical issue: if you block a page with disallow in robots.txt, Google cannot crawl it, which means it also cannot read the noindex tag. As a result, the page may still remain indexed (or re-indexed later) because Google never receives the noindex instruction.
Using both directives simultaneously is counterproductive and can lead to unexpected results. The right approach is to use one or the other, depending on your goal:
- Use disallow if you want to prevent bots from accessing the page entirely (for example, staging environments or private areas).
- Use noindex if you want the page to remain crawlable but simply not appear in search results.
Google does not support noindex in robots.txt
Another common mistake is trying to use a noindex directive directly inside a robots.txt file. Google does not support noindex as a valid directive in robots.txt. If you add it there, Google will ignore it entirely. Noindex must always be implemented via a meta tag in the page's HTML or via an X-Robots-Tag HTTP header. There is no workaround for this.
When to use the noindex tag: Common page types
Knowing how noindex works is only half the picture. Knowing when to apply it is what makes the difference in practice. Below are the most common types of pages that typically benefit from a noindex directive.
- Thank you pages and purchase confirmation pages: these appear after a form submission or checkout. They have no search value and should not be indexed.
- Internal search results pages: paginated or filtered results generated by on-site search tools create near-duplicate content and add no value to the index.
- Login and registration pages: these are functional pages for users, not content pages for searchers. There is no benefit to having them appear in Google.
- Duplicate content pages: if multiple URLs display the same or very similar content (for example, due to URL parameters, session IDs, or printer-friendly versions), use noindex on the duplicates to consolidate indexing signals.
- Pages with syndicated or copyrighted content: if you are republishing content from another source and cannot claim originality, noindexing the page helps avoid potential duplicate content issues.
- Thin or low-quality pages: pages with very little original content, like auto-generated tag archive pages or empty category pages, can dilute your site's overall quality in Google's eyes.
- Admin and staging pages: these should never appear in public search results. Consider whether disallow or noindex is more appropriate depending on whether the page is publicly accessible.
- Privacy policy and legal pages: some sites noindex these to keep the index focused on content pages, though this is optional and depends on your strategy.
How to verify that noindex is working correctly
After implementing noindex, it is good practice to confirm that search engines have received and processed the directive. There are a few reliable ways to do this.
Google Search Console URL Inspection tool
This is the most accurate method. Enter the URL in the Search Console URL Inspection tool and check the coverage status. If the page has been successfully noindexed, Google will show a message indicating the page is "excluded" and will specify the noindex tag as the reason.
The site: Search operator
You can also check whether a page still appears in Google's index by searching site:yourdomain.com/your-page-url directly in Google. If the page no longer shows up in the results, the noindex directive has been processed. Keep in mind that this method is less precise than Search Console and may not always reflect the most current state of the index.
Viewing page source or HTTP headers
Before verifying with Google, always confirm the tag is correctly present in the page itself. Use your browser's "View page source" option and search for the meta robots tag, or use a browser extension or developer tool to inspect the HTTP response headers for the X-Robots-Tag value.
Summary: Key points to remember
- Noindex prevents a page from appearing in search results but does not delete it or make it inaccessible.
- It can be implemented via a meta tag in HTML or an X-Robots-Tag HTTP header.
- Combine noindex with follow to allow link signals to pass through, or with nofollow to block both indexing and link following.
- Noindex and disallow serve different purposes. Using them together can prevent Google from reading the noindex instruction.
- Google does not support noindex inside robots.txt files. Always use a meta tag or HTTP header instead.
- Apply noindex to pages that add no search value: thank you pages, confirmation pages, login pages, duplicate content, and internal search results.
- Use Google Search Console's URL Inspection tool to verify the directive has been correctly processed.