Site SEO AI Auditpor Internet Solutions

Noindex by Accident: Find Pages Hidden From Google

17 de noviembre de 20268 min de lecturaSEO técnico
Noindex by Accident: Find Pages Hidden From Google

Short answer: A noindex instruction tells search engines to keep a page out of results. Site SEO AI Audit lists every page that carries one as a warning, “Pages are hidden from search (noindex)”, and shows an Indexable column in the page list. Check the list against the pages you want to rank. For private or utility pages, noindex is fine. For important pages it is a silent traffic killer: remove it at its source, whether a page setting, a template, a plugin or a server header, and run a new check.

Most SEO problems reduce visibility. A stray noindex removes it completely. The page loads, looks perfect and works for every visitor, yet search engines have been asked to ignore it, and they obey. It often happens during a redesign or a migration, when a development setting survives into production, or when a plugin option is changed for one page and applied to a whole group. Because nothing looks wrong, the problem can last for months. The audit catches it in one pass, because it reads the instruction on every page. This article shows how to read the findings and trace each noindex to its cause.

What noindex does

Noindex is an instruction that says: you may read this page, but do not include it in your index. It can live in a meta tag in the head of the page or in an HTTP response header called X-Robots-Tag. Search engines respect it, and they drop the page from results once they see it. The instruction is not a mistake in itself. It is the right tool for pages that should exist but not be found: thank-you pages, internal search results, login and account pages, test pages and some archives.

It is easily confused with a different tool, the robots.txt disallow, which stops crawlers from fetching a page at all. The two work differently and have different side effects, as the article on noindex versus robots disallow explains. The audit reports both: blocks in robots.txt as their own issues, and noindex as the warning discussed here.

What the report shows

The issue “Pages are hidden from search (noindex)” sits in the Crawl and index area. It is a warning, and its card says that these pages will never appear in search results, which is fine for private pages and harmful for important ones. The fix is to remove noindex from pages that should be found. On WordPress, the card gives the steps: open the page, find the box of your SEO plugin, go to the advanced settings and allow search engines to show this page.

The page list gives you the other half of the picture. Its Indexable column marks pages that cannot be indexed, because of a noindex or because the canonical points to another address. Use the search box to look at one section at a time, and scan for pages you care about. The key figures also show how many pages are indexable, which is a quick measure: if you expected 400 and see 120, something is hiding the rest.

A related protection is built in. If robots.txt blocks the whole site, or if no page is indexable, the score is capped at 20, and the report shows the critical issue “robots.txt blocks the whole website” when that is the cause.

Intended or accidental?

Go through the list and sort pages into two groups. Intended noindex belongs on pages that visitors need but search does not: account pages, carts, checkouts, thank-you pages, internal search results, filtered duplicates and drafts. Accidental noindex shows up on pages that you want people to find: services, products, categories, articles and landing pages.

Type of page Noindex?
Home page, services, products, categories No: remove it
Articles and guides No: remove it
Thank-you and confirmation pages Yes: intended
Cart, checkout, account, login Yes: intended
Internal search results Yes: usually intended
Thin tag or date archives Often yes, decide by value

A good test is the “would I be upset” question. If you would be upset to learn that a page cannot be found in Google, it should not carry noindex.

Where an accidental noindex hides

The instruction can come from many places, so tracing the source is the main task. Work from the broad to the narrow:

To find the source on a sample page, view the source and search for “robots”. Check the response headers with your browser’s developer tools or an online header checker. If the tag is in the source, look at the plugin and theme. If the instruction appears only in the header, look at the server and the CDN. The guide to the X-Robots-Tag header explains how it works.

Fix it at the source

  1. Pick three pages from the list that should be indexable, and trace the instruction to its origin.
  2. Change the setting that produces it: the site-wide option, the page type setting, the single page or the header rule.
  3. Clear caches, including your CDN, and verify in the source and the headers that noindex is gone.
  4. Check that the sitemap lists these pages and that their canonicals point to themselves, so that the signals agree.
  5. Run a new check. Starter, Pro and Agency include re-checks. The affected list should contain only intended pages.
  6. Ask search engines to recrawl key pages through Search Console, and watch the indexing report.

Be aware that fixing noindex does not bring a page back instantly. Search engines must recrawl it, and that takes days or weeks. The article on removing a page from Google describes the reverse process, which also shows how long changes take.

A worked example

An online course provider with 150 pages sees a score of 52 and a warning that 38 pages are hidden from search. Reading the list, the owner recognises 12 intended pages: login, account, thank-you and cart. The other 26 include the main course pages and the pricing page. Searching the source of a course page shows the robots meta tag with noindex, and the SEO plugin settings reveal that the content type used for courses was set to “do not show in results” during development, when the courses were still drafts. One checkbox fixes 26 pages. The owner clears caches, runs a re-check that shows 64 indexable pages instead of 38, and requests indexing for the three most important course pages in Search Console. Impressions begin to return within two weeks.

Make signals agree

Noindex rarely travels alone. Pages that carry it are often listed in the sitemap, which is a contradiction: you invite search engines to a page and tell them to ignore it. The audit reports that as “Sitemap lists pages that do not work”. Clean the sitemap, as described in the article on sitemap problems. Also watch canonicals: a page with a canonical to another address is not indexable either, and a canonical to a broken page triggers a critical issue.

A quick monthly habit

You do not need a complex process to avoid this problem. Once a month, after your regular check, look at two figures: the number of indexable pages and the length of the noindex list. A sudden change in either is worth a look before it turns into lost traffic. Keep a short list of the pages that are intentionally noindex, so that anyone on the team can tell the difference between a deliberate choice and a leftover. When someone asks why a page is not in Google, this list and the report answer the question in a minute.

Prevent it from coming back

Add noindex to your launch and release checklists. Before a launch, check the site-wide setting and robots.txt. After releases, theme updates and plugin changes, run a check and read the number of indexable pages. If you use Pro, e-mail alerts warn you when a check shows a lower score, and comparison of audits over time shows which issues are new. To see how your own site looks to a crawler, create a free account and run your check, and choose a plan on the pricing page to unlock the lists.

Related reading

The bottom line

A noindex on the wrong page silently removes it from search. Sort the audit’s list into intended and accidental, trace each accidental case to its setting, template or header, fix it at the source and confirm with a new check.

FAQ

Is noindex always a mistake?

No. It is right for private or utility pages such as thank-you pages, carts and account areas. The report shows it as a warning so you can check the list.

Where can a noindex come from?

A meta tag set by a plugin, theme or page setting, a site-wide option, or an X-Robots-Tag header from the server or CDN.

How do I see which pages are indexable?

The page list has an Indexable column, and the key figures show the number of indexable pages.

What if the whole site is blocked?

The report shows a critical issue and caps the score at 20. Remove the block in robots.txt or the site-wide setting first.

How long until a page returns to Google?

Search engines must recrawl it, which can take days or weeks. Ask for a recrawl in Search Console for key pages.

#Indexing#robots.txt#SEO audit#Technical SEO
Audita tu propio sitio web.¿Google ve su sitio web?
Inicia tu auditoría
Internet Solutions

Más de nuestro equipo

Creadas por Internet Solutions. Prueba nuestros otros productos: cada uno te ahorra tiempo de una forma distinta.

internet-solutions.net ↗