When a Page Vanishes From the Index

A client messaged me on a Sunday to say their best page had been deindexed. It had been their top earner for two years and it was gone from search entirely. They wanted to know who to appeal to.

The page had a noindex tag on it. It had been put there by a plugin update eleven days earlier, and nobody had touched the page itself, so nothing in their content workflow flagged it. The whole thing took four minutes to find and about thirty seconds to fix.

That’s the useful lesson, and it’s why this is worth going through as an ordered checklist rather than a list of possible causes. A page disappearing feels dramatic, and most of the time it’s mechanical, but only if you check things in the right sequence. Checking in the wrong order is how people spend a fortnight on a problem that was a checkbox.

First, establish that it’s actually gone

Start here, because a surprising share of these turn out to be a page that dropped in position rather than one that left the index.

Search for a distinctive phrase from the page in quotes, along with your domain restricted using the site operator. If the page comes back, it’s indexed. What you have is a ranking problem, which is a completely different investigation, and one where the rest of this checklist will waste your time.

If that returns nothing, use the URL Inspection tool in Search Console on the exact address. That tells you what the search engine currently holds, and it’s the only source worth trusting here. A rank tracker showing “position not found” tells you about one query, on one day, from one location.

One thing to watch: inspect the precise URL, with the same protocol, the same subdomain, and the same trailing slash. I’ve watched two people conclude a page was deindexed when they were inspecting a slightly different address to the one that actually exists.

Reading what the inspection tool tells you

The verdict it gives back is more specific than people treat it, and the wording is doing real work.

“URL is on Google” means indexed. Done. If you’re seeing that and the page still isn’t showing up for anything, your problem is ranking, and you can close this checklist.

“URL is not on Google” is the one that sends you down the rest of this list, but read the reason underneath it rather than stopping at the headline. “Excluded by noindex tag” names your cause outright. “Crawled - currently not indexed” means it was fetched and a decision was made not to keep it, which points at the quality question near the end of this article rather than at anything mechanical. “Discovered - currently not indexed” means it hasn’t even been fetched yet, which is a different problem again, and usually about internal linking or crawl volume.

“Alternate page with proper canonical tag” is the one worth memorising, because it looks alarming and it means the search engine found a canonical pointing elsewhere and honoured it. That’s step four below, and the tool has just handed you the answer for free.

Second, read the page’s own instructions

If it’s genuinely not indexed, the next question is whether the page is telling the search engine to stay away. This is the single most common cause I see, and it takes two minutes.

Fetch the raw HTML rather than looking at the rendered page in your browser, because a view-source check catches things a visual check never will. Look for a robots meta tag with noindex in it. Check the HTTP response headers as well, because an X-Robots-Tag header does the same job and is invisible in the page source.

Then check robots.txt for a rule that covers the path. A blocked page usually stays in the index for a while and then drops out, which produces exactly this delayed, mysterious disappearance.

These get switched on by accident constantly. A plugin update. A staging environment pushed live with its settings intact. A developer adding a rule for one directory that turns out to match a great deal more than anybody intended, which is the version I see most and the one that produces the largest mess, because it takes out a whole section at once rather than a single page somebody would have noticed. It’s rarely anybody being careless. It’s almost always the answer.

Third, check what the server actually returns

The page might not be reachable in the way you think it is.

Request the URL and look at the status code that comes back, not the page that renders. A 404 or a 410 means it’s genuinely gone as far as anything crawling is concerned. A 301 or 302 means it’s pointing somewhere else, and you want to know where, because redirect chains get created by well-meaning tidy-ups.

Soft 404s are the sneaky version of this. The server returns a 200 saying everything is fine, but the page is empty or shows a not-found message in the template. From the outside that looks like a page with nothing on it, and pages with nothing on them get dropped. Search Console flags these under its own label.

And confirm the page loads without a login. Content behind a session cookie is invisible to a crawler even though it looks perfectly normal to you, sitting there logged in.

Fourth, look for a canonical pointing elsewhere

This one is quieter than the others, and it’s the cause people miss most often.

If the page carries a canonical tag naming a different URL, you’re telling the search engine that some other page is the version worth keeping. Respected, that means your page drops out and the other one stays. It doesn’t look like an error anywhere in your CMS, and nothing warns you.

It happens through templates that hardcode a canonical, through bad pagination, and through migrations where it kept pointing at the old domain. Inspect the tag on the actual page and confirm it names itself.

Fifth, ask whether it’s still linked to

A page that nothing points at gets crawled less and can eventually fall out, especially if it wasn’t earning much attention anyway.

Check whether it’s still in your sitemap. Check whether any internal link still reaches it, because a navigation rebuild or a category restructure can orphan a page while leaving it perfectly functional for anyone with the address. If a page is reachable only by typing the URL, it’s in a weak position.

This rarely explains a sudden disappearance on its own. It explains a slow fade, and it’s often a contributing factor sitting behind one of the causes above.

Sixth, the ones that aren’t your fault

If all of that comes back clean, now consider the external explanations, and only now.

A manual action would be sitting in Search Console under Manual Actions, described plainly. That report exists precisely so you aren’t left guessing, and if it’s empty, you haven’t had one.

A legal removal request can take a page out of results in a particular country while leaving it indexed elsewhere. That shows up as a page gone for you and present for a colleague abroad. Worth testing before you assume.

And there’s the possibility that the page was judged not worth keeping. Thin pages, near-duplicates of something else on the site, and pages built to catch a query without saying anything get dropped, and it isn’t a penalty. It’s a decision that the page wasn’t adding anything. That one doesn’t announce itself, and you diagnose it by honestly reading the page.

The country-version trap

There’s one more cause that catches sites running more than one version of the same page, and it doesn’t look like any of the above.

If you publish a page for the UK and a near-identical one for Australia, and the hreflang annotations between them are wrong or missing, the search engine may treat the two as duplicates and keep whichever it prefers. Your Australian page then looks deindexed. It isn’t, quite. It’s been folded into the other one.

The tell is that the inspection tool names the other country’s URL as the canonical, and that your traffic didn’t so much vanish as move. I had this on a site with British and Irish versions whose pages differed by a currency symbol and a phone number, and honestly, that read was fair. Two pages that differ by nine words are one page.

The fix is either to make the versions genuinely different, with local content that justifies a separate page, or to accept the consolidation and stop maintaining two. What doesn’t work is annotating harder. Hreflang tells the search engine which version to show which audience, and it doesn’t create a difference that isn’t there.

What I’d stop doing

A few reflexes make this worse, and they’re all common.

Resubmitting the URL repeatedly in Search Console does nothing beyond the first request. It can’t override an instruction the page itself is giving, and no amount of asking changes that, because the request only ever means “come and look again” rather than “please ignore what you find when you get here.” If the page says noindex, requesting indexing a dozen times queues up a dozen confirmations of the noindex. That’s it. That’s the whole outcome.

Rewriting the content before you’ve checked the mechanical causes wastes the work, because you’ll republish the same problem with different words on it.

And building links at a page that’s telling the search engine to ignore it achieves nothing at all. I’ve seen a few hundred pounds spent on placements pointing at a page with an X-Robots-Tag header on it.

How long it should take to come back

Once you’ve fixed the actual cause, recovery is usually days rather than weeks for a page that has history and internal links pointing at it. Request indexing once, then leave it.

If it was a robots block, expect it to take a bit longer, because the crawler has to come back, discover it’s allowed in now, and fetch the page properly before anything changes.

I won’t give you a specific number of days, because it varies with how often your site gets crawled and I’d be making it up. What I can say is that if two weeks have passed with a clean fix and nothing has moved, the cause you found probably wasn’t the only one.

Keeping it from happening again

Most of these are preventable with one habit, which is monitoring the mechanical signals rather than the traffic.

Pick your twenty or thirty pages that actually matter, and check weekly that each returns a 200, carries no noindex, and has a self-referencing canonical. That’s a small script or a cheap monitoring tool, and it catches a plugin update the day it happens, not eleven days later.

Watch the indexed page count in Search Console as a number over time, because a sudden step down is visible there long before it’s obvious in traffic. And treat any deployment that touches templates, robots rules, or the CMS as something to spot-check afterwards, since that’s where these come from.

The honest limit

This checklist finds the mechanical causes, and mechanical causes are most of them. It won’t tell you why a page slid from third to fortieth, which is the far more common and far harder problem.

It also won’t help much with a page that was never really good enough. If a page disappears and every check comes back clean, the answer is sometimes that the page didn’t have a reason to exist, and no amount of technical work fixes that.

So when a page goes missing

Confirm it’s actually deindexed rather than ranking badly. Read the raw HTML and the response headers for noindex. Check that robots.txt covers nothing it shouldn’t. Verify the status code and rule out a soft 404. Check that the canonical names the page itself. Confirm something still links to it. Only then look at manual actions and the external causes.

Nine times out of ten you’ll stop at step two, which is the point of doing them in that order.

For more checklists and technical SEO breakdowns like this one, head over to The SEO Desk.