What Is Indexability, and What Quietly Blocks It?

Jul 24, 2026 · SEO Glossary

The most expensive single character I've ever seen in SEO was a checkbox. A client redesigned their site, the developers worked on a staging server with "Discourage search engines" enabled — standard practice — and shipped to production with the setting still on. Every page got a noindex tag. Traffic bled out over six weeks before anyone connected the redesign to the decline. The site was fully crawlable the whole time; it just wasn't indexable.

What is indexability? It's a page's ability to be added to a search engine's index — the database of pages eligible to appear in results. It's distinct from crawlability, and the distinction is where teams get burned: crawlable means Googlebot can fetch the page; indexable means Google is permitted and willing to store it. A page can be crawled daily and never indexed, and a blocked-from-crawling page can even linger in the index as a bare URL. Two gates, two separate sets of controls.

What is indexability determined by?

Whether a fetched page makes it into the index comes down to a stack of signals, checked roughly in this order:

1. The noindex directive. A <meta name="robots" content="noindex"> tag or X-Robots-Tag HTTP header is an explicit instruction Google always honors. This is the right tool for pages you genuinely want out — internal search results, thank-you pages, thin archives — and the wrong tag to ship sitewide by accident.
2. The HTTP status code. Only pages returning a 200 status get indexed. Redirects pass their signals to the destination; 404s and 500s get dropped. A "soft 404" — an empty results page returning 200 — gets classified as an error anyway and excluded.
3. The canonical resolution. If your page declares (or Google infers) that another URL is the canonical version, your page steps aside and the canonical gets indexed instead. Misfired canonicals are a top cause of mysterious deindexing — I've seen a template bug canonicalize 3,000 product pages to the homepage.
4. Quality judgment. The gate nobody controls directly. Google fetches the page, evaluates it, and may simply decline — that's the "Crawled — currently not indexed" status in Search Console. No error, no directive, just a judgment that the page doesn't clear the bar right now. Thin content, near-duplicates, and weak internal linking are the usual suspects.

Robots.txt deserves its own clarification because it's chronically misused: blocking a URL in robots.txt prevents crawling, not indexing. Google can still index a robots-blocked URL from links alone, showing it with no description. Worse, if you add noindex to a page and block it in robots.txt, Google can never fetch the page to see the noindex — the two directives sabotage each other. To deindex a page, it must be crawlable.

How indexability problems actually arrive

These issues almost never come from someone deliberately editing a meta tag. They ship as side effects:

• Staging settings that survive deployment — my opening story, and I've watched some version of it happen to at least five clients.
• SEO plugin misconfiguration. One toggle in a WordPress SEO plugin can noindex whole content types. Site owners change themes or plugins and inherit settings nobody reviews.
• JavaScript rendering gaps. The raw HTML is fine, but client-side JavaScript injects a noindex, or the meaningful content only exists after rendering that times out or errors. The page looks perfect in a browser and empty to the indexer.
• Faceted navigation floods. Filter combinations generate tens of thousands of near-duplicate URLs; Google burns its crawl capacity on them and downgrades its interest in the site's real pages. Nothing is technically "blocked," but indexation coverage rots anyway.

Checking a page, then checking the site

For a single URL, Search Console's URL Inspection tool is the ground truth: it shows whether the page is indexed, what Google chose as canonical, and any blocking directive it found. The Page indexing report aggregates the same data sitewide, sorted by exclusion reason — worth reading monthly even when nothing seems wrong.

But Search Console reports what Google already encountered, on Google's schedule. The staging-noindex disaster ran six weeks precisely because nobody was watching for the change. The preventive version is a crawler you control: a scheduled site audit checks every page's robots directives, status codes, and canonicals on your schedule and flags the day a noindex appears where it shouldn't — turning a six-week bleed into a same-day fix. Once a page is confirmed indexable, whether it actually gets picked up is the separate story of indexing itself.

The strategic version of the question

Mature sites don't aim for "index everything" — they aim for "index exactly the right things." Every large site has pages that should be noindexed: parameter variants, internal search results, tag archives with one post, checkout steps. Letting those flood the index dilutes the site's perceived quality, since Google's assessment averages across what it indexes. The practical goal is a deliberate map: revenue and content pages indexable and clean; utility and duplicate pages consciously excluded; and a monitoring habit that notices when reality drifts from the map. Indexability isn't a launch-day checkbox — it's a property that decays without supervision.