How to spot orphan pages before they waste crawl time

If you want to know how to find orphan pages on a website, start by comparing three things you already control. First, a crawl of pages your internal links can actually reach. Second, the URLs you publish in XML sitemaps. Third, the URLs Google has seen through Google Search Console. Any URL that appears in your known lists but not in your internal crawl is a likely orphan.

That does not automatically make it a mistake. Some URLs are intentionally hard to discover, excluded from indexing, canonicalised elsewhere, or kept only for paid campaigns, testing, downloads, or legacy redirects. The useful job is not just finding them, but deciding which ones should be linked, merged, redirected, removed from sitemaps, or left alone.

What counts as an orphan page and why it matters

An orphan page is a URL on your site that exists, but has no internal links pointing to it from crawlable pages on the same site. In plain terms, if a standard site crawl starts at your homepage and follows links, it cannot reach that page.

That is different from a page that simply gets little traffic. A low traffic page may still be properly linked in your navigation, category structure, or related content. It is also different from a page that is excluded from Google’s index for a known reason, such as a duplicate with a chosen canonical, a noindex page, or a redirected URL. Google’s own guidance is to investigate why a URL is excluded rather than assuming every excluded page is a problem Google Page indexing report.

Why orphan pages matter is mostly practical.

First, they are easy to overlook in routine editing. If nobody can click to them from the rest of the site, they often miss internal review, content updates, and quality checks.

Second, discovery becomes less reliable. Search engines can find URLs through sitemaps, external links, redirects, feeds, or historical knowledge, but internal linking remains one of the clearest ways to show that a page belongs in the site structure Google SEO Starter Guide.

Third, they can waste crawl attention. That point is often overstated. A few orphan URLs on a modest site are not usually a crisis. The issue grows when old campaign pages, faceted URLs, expired products, or duplicate content accumulate in large numbers. Then Googlebot may keep revisiting pages that no longer help users, while important pages rely on thinner internal signals.

There is also a user side to this. If a page is meant to help people, but the only route to it is a search result or a direct link, your site structure is probably not doing its job. Clear navigation and understandable linking help both discovery and usability W3C accessibility design tips.

Which URL lists do you need first?

Before comparing anything, gather the broadest sensible set of URLs you know about. One list is never enough.

We usually start with these:

1. A crawl list from a crawler
2. A sitemap list
3. URLs from Google Search Console
4. Optional extras from analytics, server logs, CMS exports, and backlink tools

1. Crawl the site

Run a crawl with a desktop or cloud crawler that follows internal links. Start from the canonical host and preferred protocol, so you are crawling the same version you expect users and search engines to use.

Export at least:

• All HTML URLs discovered
• Inlinks or internal link counts
• Status codes
• Canonical targets
• Indexability directives if your crawler reports them

This is your reachable set. It tells you what a bot can find by following links from the starting point.

2. Export your XML sitemap URLs

Download the XML sitemap or sitemap index and extract all listed URLs. If you maintain multiple sitemaps, combine them into one master list, then deduplicate.

This matters because sitemaps often contain pages that are not well linked, recently published, outdated, or accidentally left behind. A sitemap is not proof that a page should rank or even be indexed, but it is a strong clue that someone on the site intended the URL to matter.

3. Export URL evidence from Google Search Console

From Google Search Console, collect URLs from the indexing reports and any relevant performance export.

Useful places include:

• Page indexing
• Sitemaps
• Performance, Pages
• URL Inspection for spot checks

Google Search Console does not always hand you a complete list in one neat export, so you may need to combine several exports. If your site is large, work in sections.

What you are looking for are URLs that Google knows about through discovery or prior crawling, even if your own current crawl cannot reach them.

If you want a wider picture of what Search Console can and cannot tell you, our guide to Search Console evidence and its limits is useful before you draw conclusions from a single report.

4. Add optional URL sources if you have them

These extra sources often reveal the oddest orphans:

• CMS exports of published pages
• Product feeds
• Historic migration lists
• Analytics landing pages
• Server log files
• Backlink reports
• Paid campaign destination URLs

Server logs are especially useful on larger sites because they show what crawlers actually requested. That can include Googlebot, and sometimes other bots such as OAI-SearchBot or GPTBot, depending on your robots rules and whether those pages have been discovered elsewhere. OpenAI documents its crawler controls separately from search access, which is worth knowing if you are comparing visits from different bots OpenAI crawler documentation.

A simple method to find likely orphan pages

Once you have your exports, the comparison is straightforward. You do not need an enterprise platform to do the first pass. A spreadsheet is often enough.

Step 1, normalise the URLs

Before matching lists, clean them so that equivalent URLs are treated consistently.

Check:

• http vs https
• www vs non-www
• trailing slash vs no trailing slash
• uppercase vs lowercase where your server treats them differently
• URL parameters
• fragments, which should usually be ignored
• obvious duplicate encodings

If your site uses canonicals heavily, keep both the crawled URL and the canonical target in separate columns. Do not merge them too early or you will hide useful issues.

Step 2, create a master URL list

Combine the crawl export, sitemap URLs, and Search Console URLs into one sheet or table. Deduplicate exact matches after normalisation.

Add columns for:

• Seen in crawl, yes or no
• Seen in sitemap, yes or no
• Seen in Search Console, yes or no
• Status code
• Canonical target
• Meta robots or x-robots-tag
• Internal inlink count from crawl
• Notes

Step 3, filter for URLs not found in the crawl

Now filter for URLs that:

• appear in sitemap or Search Console or another trusted source
• but do not appear in the crawler’s discovered URL set

These are your likely orphan pages.

You can also catch softer cases by filtering for URLs with zero or extremely low internal inlinks, especially if they are only linked from XML sitemaps, HTML sitemaps, or utility pages no one uses.

Step 4, separate non-HTML and technical URLs

Not every orphan candidate is a page you care about in the same way.

Split out:

• PDFs and documents
• image URLs
• feed URLs
• tracking URLs
• parameter variants
• redirected URLs
• error pages

Keep the focus on HTML pages first, unless your documents are important search landing pages in their own right.

Step 5, test a sample manually

Open a representative sample in the browser and in your crawler’s inlink view.

Ask:

• Can a user reach this from the menu, category pages, filters, related links, breadcrumbs, or on-site search?
• Is the page only accessible from an old email, ad, or external backlink?
• Does the URL return a live 200 status?
• Does the canonical point elsewhere?

This is where spreadsheet theory meets the actual site.

If internal linking is unfamiliar territory, our guide to internal linking for people who edit websites explains what to look for without turning it into jargon.

How to check whether an orphan page is actually a problem

Finding a likely orphan is only the start. The next question is whether the page should be discoverable through normal site navigation and internal links.

Check the page’s purpose

Be explicit about what the page is for.

Common cases include:

• a core service or product page
• a blog article
• a campaign landing page
• a thank-you page
• a legal page
• a seasonal page
• a retired product
• a duplicate created by filtering or tracking

A core page with no internal links is usually a structural problem. A thank-you page with no internal links may be entirely intentional.

Check canonical status

If the page has a rel=canonical to another URL, ask whether it is effectively a duplicate or alternate version.

Examples:

• print versions
• parameter variants
• duplicate product sort pages
• old CMS path variations

If the canonical is valid and intentional, the orphan may not need fixing with new links. It may need tidying elsewhere, such as removal from the XML sitemap.

Check redirects

If the URL redirects, it is not a page you need to link to directly. What matters is whether:

• the redirect destination is correct
• old internal links still point to the redirected URL
• the redirected URL is still listed in the sitemap or other feeds

Orphaned old URLs often show up after migrations because they remain in Search Console history or external links long after the site structure changed.

Check noindex and robots handling

If the page is noindex, ask why.

A noindex page may still be useful to users and worth linking internally, or it may be deliberately excluded. The point is not to treat noindex as inherently wrong. Google’s own documentation is clear that excluded pages need interpretation, not blanket correction Google Page indexing report.

Also check whether the page is blocked in robots.txt. If it is blocked, your crawler may not have fetched it depending on settings, which can complicate orphan checks.

Check duplication and thin variants

Some orphan pages are leftovers from:

• tag archives
• filtered category combinations
• duplicate location pages
• expired product variants
• old pagination patterns

If the content is substantially duplicated elsewhere, adding links may make the site messier rather than better. Consolidation is often the better fix.

Check whether it should be discoverable at all

This is the most important judgement.

A page should usually be discoverable if it is meant to:

• serve as a search landing page
• support a conversion path
• answer a recurring customer question
• represent a current product, service, category, or article

A page may reasonably stay undiscoverable if it is:

• a post-conversion thank-you page
• a testing or preview URL
• a temporary campaign page with a short life
• a utility page for a specific audience
• an archived asset retained for compliance or record purposes

The answer is often editorial, not purely technical.

What to do with each orphan page you find

Once you know why the orphan exists, the action usually falls into one of five buckets.

Add internal links to pages that should be found

If the page is important, link to it from places that make sense.

Good candidates include:

• navigation and sub-navigation
• relevant category or hub pages
• related articles
• service overview pages
• breadcrumbs
• footer links, if genuinely useful
• contextual links within existing content

Do not add links everywhere just to increase counts. Add them where a user would expect them and where the destination helps complete the task. Google’s guidance remains simple here, useful content and descriptive links help discovery Google SEO Starter Guide.

Merge or consolidate duplicate content

If the orphan is one of several near-identical pages, choose the best destination and consolidate.

That may mean:

• moving unique content into the stronger page
• redirecting the weaker URL
• updating canonicals
• removing old URLs from sitemaps
• replacing internal references to the retired version

This is common after site redesigns, CMS changes, or well-meant content expansion that created overlap.

Redirect obsolete URLs

If the page is no longer needed, use a proper server-side redirect to the nearest relevant replacement.

Do not redirect everything to the homepage by default. That often creates a poor user experience and weakens the logic of the clean-up.

After redirecting:

• update internal links to point straight to the final destination
• remove the old URL from XML sitemaps
• check that canonical tags and hreflang references, if any, no longer mention it

Remove URLs from sitemaps when they are not meant for discovery

A surprising number of orphan checks end with a sitemap fix rather than a linking fix.

Remove URLs from XML sitemaps if they are:

• redirected
• canonicalised elsewhere
• noindex by design
• expired and retired
• utility pages not intended as search entry points

That keeps your sitemap closer to a list of URLs you actually want search engines to consider.

Leave some pages intentionally excluded

Not every orphan needs rescuing.

If a page is intentionally isolated, document that decision. Keep a simple note in your content inventory or technical changelog so the next audit does not treat it as an unexplained defect.

This is especially helpful for:

• campaign pages
• gated assets
• internal-use resources
• archived notices
• conversion confirmation pages

The value is in making the status deliberate rather than accidental.

A practical way to stay on top of this is to build orphan checks into a broader review cycle. Our practical website audit you can finish and trust gives you a sensible order for checks so orphan pages are not investigated in isolation.

Orphan pages are rarely the biggest issue on a site, but they are often a revealing one. They show where publishing, migration, search reporting, and site structure have drifted apart. If you gather a crawl list, a sitemap list, and what Google Search Console knows, you can usually find the likely orphans quickly. The harder and more valuable part is deciding which ones deserve stronger internal links, which should be consolidated, and which should stay out of the way.

That is the evidence-led way to approach the question of how to find orphan pages on a website. Not by assuming every unseen URL is harmful, but by checking what the page is for, how it is discovered, and whether your own site structure supports its purpose.