Juan Luis MoraSEO, Google Ads e IA
ES
Hablemos

Why does Search Console show 404s your crawler can't find?

Juan Luis Mora · Updated 18 November 2026

You crawl the whole site. No broken links.
You open Search Console and there they are: 404s.

It isn't a tool bug. Those 404s aren't born inside your site.

Where do they come from?

On an institutional portal with tens of thousands of pages, most of them came from old PDFs.

PDFs that were still indexed.
Linked from sites outside the portal.
With URLs of document versions that were no longer valid.

Google still had them. Other sites still linked to them. And the portal answered 404.

Why doesn't a crawl find them?

A crawler starts on your site and follows your links. If none of your links point to that old version, it never sees it.
Search Console does, because Google already knew the URL or finds it in external links.

Logs don't make it easy either. Google has just documented how to spot Search visits in your logs with the Referer header: it only carries the domain, no path, and it can be empty because of privacy settings, a new tab or a redirect Google Search Central.
You see the 404. You don't always see where it came from.

So, for this kind of 404: Search Console first.

How I found them

Mostly with Search Console reports. That's where the old PDFs returning 404 showed up.
Plus Screaming Frog audits to compare them with what the site itself links to.

Two tools. That's it.

How it was fixed

Not with one redirect per URL. There are too many versions.

The portal's tech team set the server to send every version of each PDF to its new canonical URL.
One rule, not thousands of patches.

And one more: if a site embeds one of those PDFs in an iframe, it can no longer serve it inside its own page. It always takes the user to the portal, to the canonical.
The traffic from those links comes home.

That PDF will become HTML in the future.

If you have to do this, Google gives you two options:

  • A permanent server-side redirect (301 or 308). It's the strongest signal that the target should be canonical Google, redirects.
  • An HTTP Link header with rel="canonical". It's how you set a canonical on a PDF, which has no head Google, duplicate URLs.

What's hard about a public-sector portal?

It's fascinating.
But many recommendations can't be followed, because they need broad consensus.

Watch out these days

Google's November 2026 core update started on 10 November and may take up to two weeks. Google asks you not to draw conclusions from Search Console until it's done Search Status Dashboard.
Old PDF 404s don't depend on it. You can clean them up now.

How many of your Search Console 404s also show up in your own crawl?


Was this useful? If you want me to look at your site, here is how I work SEO in Spain or get in touch.

¿Lo miro en tu web?

Le echo un vistazo a los datos y te digo qué tocar primero.

Escríbeme un correoPedir presupuesto SEO

Juan Luis Mora · Ingeniero informático y consultor SEO freelance. SEO en español para España y el mundo.

Escribo para IAs desde Google Bard.

Black Friday y Navidad

Web con IA + Google Ads

2.000 €

Para Black Friday quedan34 días 2 h 58 min

Te hago una web como esta, lista para Google y para las IAs, y la campaña de Google Ads que la acompaña.

Quiero mi web con IA
Hablemos