What Is a Noindex Tag? Keeping Pages Out of Google

Jul 25, 2026 · SEO Glossary

The most expensive single line of HTML I've ever encountered in an audit: a staging site's noindex meta tag, copied to production during a relaunch and left there for seven weeks. Rankings built over eight years evaporated in about three, because the tag was doing exactly what it's designed to do — to every page on the site. The client's traffic chart looked like a cliff dive. The fix took thirty seconds. The recovery took five months.

What is a noindex tag? It's a directive — <meta name="robots" content="noindex"> in a page's head — that tells search engines: crawl this page if you like, but do not show it in search results. Unlike most SEO signals, which are hints and suggestions, noindex is a command Google obeys. That obedience is precisely what makes it both useful and dangerous.

What Is a Noindex Tag, Mechanically?

Two delivery methods, same effect. The meta tag version lives in the HTML head and covers that page. The HTTP header version — X-Robots-Tag: noindex — is set by the server and works on anything, including PDFs and images that have no HTML to put a tag in. Directives can be combined, e.g. content="noindex, follow", though as we'll see, the "follow" part has an expiry date.

The critical mechanic people miss: a crawler must fetch the page to see the directive. Noindex works at the moment of crawling. Hold that thought, because it's the setup for the classic blunder.

Noindex vs. Robots.txt: The Mix-Up That Breaks Both

These two get confused constantly, and combining them wrong produces the opposite of what anyone intended:

• Robots.txt says "don't crawl this." It controls fetching, not indexing — a URL blocked in robots.txt can still be indexed if enough links point at it, appearing in results as a bare URL with no description.
• Noindex says "don't index this." It requires crawling to be discovered.

Put both on the same URL and you've built a paradox: robots.txt prevents Google from fetching the page, so Google never sees the noindex, so the page can sit in the index indefinitely as an ugly, description-less result. If you want a page out of the index, it must remain crawlable until it's deindexed. Sequence matters: noindex first, verify it's gone, then block crawling if you still want to.

When Noindex Is the Right Call

Legitimate use cases, from the audit trenches:

1. Thin utility pages: login screens, cart and checkout steps, "thank you" pages, internal search results — Google specifically recommends noindexing on-site search results, which can otherwise spawn infinite low-value URLs.
2. Faceted and filtered duplicates that canonical tags aren't handling cleanly.
3. Staging and development environments — though password protection is the grown-up solution; noindex on staging is exactly the tag that gets copied to production. Ask my former client.
4. Tag and archive sprawl on blogs: date archives and hundred-post tag pages that compete with real content.
5. Genuinely private-ish content you can't put behind auth but don't want searchable. (Noindex hides from search, not from anyone with the URL — it's a curtain, not a lock.)

What noindex is not for: pages you merely think are underperforming. A page that ranks poorly but earns a few long-tail visits is a net positive; deindexing it converts a small asset into nothing.

The Long-Term Wrinkle: Noindex Eventually Swallows Your Links

A subtlety worth knowing: Google has confirmed that pages kept noindexed long-term get crawled less and less, and eventually the links on them stop being followed — effectively, long-term noindex, follow decays into noindex, nofollow. So if a noindexed page is a hub that links to content you do want ranked, that internal link equity route will quietly close. Make sure indexed pages have paths to your important content that don't run through noindexed intermediaries.

How Sites Noindex Themselves by Accident

The greatest hits, all found in real audits:

• The staging copy-paste. Covered above. Relaunches and migrations are when it strikes.
• The CMS checkbox. WordPress's "Discourage search engines from indexing this site" is one click, applies sitewide, and looks identical whether checked by intention or elbow.
• SEO plugin defaults that noindex categories, tags, or custom post types — sometimes correctly, sometimes taking out page types that were earning traffic.
• Template-level tags in a theme's head file, applying to every page built on that template. Sometimes only in the mobile template, which is the one that counts under mobile-first indexing.
• The X-Robots-Tag header set at server or CDN level — invisible in the HTML, missed by anyone who only views source.

The common thread: accidental noindex is silent. Pages keep working, users notice nothing, and traffic decays on a lag as Google recrawls. By the time someone asks "why is organic down 40%?", the tag has been live for weeks. This is exactly the class of problem scheduled site monitoring exists for — a crawler that checks your pages' robots directives on a schedule flags the change within a day, while it's still a one-page problem instead of a quarterly-review problem. It's also a standard line item in any competent audit report: every noindexed URL, listed, so a human can confirm each one is intentional.

Noindex does exactly what you tell it, immediately, sitewide if that's where you put it. Treat it with the respect you'd give any tool that obeys without asking questions.