Standard vs crawler documentation

RFC 9110 Definitions vs Google's Documentation

The IETF defines what each status code means in RFC 9110 (HTTP Semantics). Google documents separately how its crawlers and indexing system respond to those codes:

HTTP 404 Not FoundRFC 9110 §15.5.5

Not found; permanence unstated

RFC 9110: "The 404 (Not Found) status code indicates that the origin server did not find a current representation for the target resource or is not willing to disclose that one exists. A 404 status code does not indicate whether this lack of representation is temporary or permanent; the 410 (Gone) status code is preferred over 404 if the origin server knows, presumably through some configurable means, that the condition is likely to be permanent."

What Google documents:

"All 4xx errors, except 429, are treated the same." Google doesn't use the content of 4xx URLs and removes previously indexed URLs from the index. For 404 specifically: "Newly encountered 404 pages aren't processed," and "The crawling frequency gradually decreases."

HTTP 410 GoneRFC 9110 §15.5.11

Gone; likely permanent

RFC 9110: "The 410 (Gone) status code indicates that access to the target resource is no longer available at the origin server and that this condition is likely to be permanent. If the origin server does not know, or has no facility to determine, whether or not the condition is permanent, the status code 404 (Not Found) ought to be used instead."

What Google documents:

Google lists 410 among the 4xx codes that are "treated the same." Its documentation does not describe faster removal or different crawl treatment for 410 compared with 404.

Crawling & removal

Crawling & Removal: What Google Documents, and What It Doesn't

Many SEO guides give specific de-indexing timelines for 404 and 410. We could not find those timelines in Google's current documentation, and ErrorCodeDecoder has not measured them. Here is what the cited Google pages actually say.

Documented by Google

  • "Google doesn't use the content from URLs that return 4xx status codes."
  • "All 4xx errors, except 429, are treated the same."
  • For Google Search, the indexing pipeline removes the URL from the index if it was previously indexed.
  • "The crawling frequency gradually decreases."
  • 4xx codes other than 429 do not affect Google's crawl rate for the site; 429 and 5xx errors make Google's crawlers temporarily slow down.
  • Search Console help: "404 responses are not necessarily a problem, if the page has been removed without any replacement," and "Googlebot will probably continue to try this URL for some period of time."

Not found in Google's documentation

  • Any specific number of hours, days, or weeks before a 404 or 410 URL leaves the index.
  • Faster removal for 410 than for 404.
  • A ranking "penalty" for 404s or for soft 404s.
  • A fixed number of re-crawls before removal.

Removal speed you see in Search Console will depend on how often Google crawls each URL. Treat third-party timelines as anecdotes unless they are measured on your own site.

Common pitfall

The Soft 404 Problem

A soft 404 is a page that tells visitors the content is missing while the server returns 200 OK. Google's status-code documentation says that if the content suggests an error, such as "an empty page or an error message," Search Console will show a soft 404 error.

Pages that can look like errors to Google

Examples of a 200 OK response whose content reads as an error or as empty:
• "Sorry, this product is no longer available"
• An empty search results page ("0 items found")
• A blank or near-empty template with only header and footer navigation

Why it matters and how to fix it

1. Mixed signal

The status code says the page exists while the content says it doesn't. Search Console help describes a soft 404 as returning "a user-friendly 'not found' message but not a 404 HTTP response code."

2. What Google recommends

"We recommend returning a 404 response code for truly 'not found' pages." If a page has moved, Search Console help says to "use a 301 redirect to the new location." If the page is real, add enough content to show it is not an error page.

3. Not a documented penalty

The cited Google pages describe soft 404s as an indexing and reporting issue. They do not describe a site-wide ranking penalty.

Implementation

Decision Matrix: When to 301, 410, or 404

A practical guide for removed URLs. Under RFC 9110 the choice between 404 and 410 depends on whether you know the removal is permanent, and Google handles both the same way:

Scenario / Resource ConditionRecommended StatusRationaleServer Configuration Snippet
Page moved, or a direct replacement exists301 Moved PermanentlySearch Console help: "If your page has moved, use a 301 redirect to the new location." Google says its systems use a 301 "as a strong signal that the redirect target should be processed."return 301 /product-new/;
Intentionally and permanently removed, with no replacement410 Gone (or 404)Matches RFC 9110's meaning of 410 (removal likely permanent). For Google Search, handled the same as 404.return 410;
Mistyped URL, random bot probe, or you don't know whether the removal is permanent404 Not FoundRFC 9110: 404 "ought to be used" when the server does not know whether the condition is permanent.return 404;
Redirecting all deleted pages to the homepageAvoidGoogle's site-move guide: this "can confuse users and might be treated as a soft 404 error."Do not redirect everything to /

Server Configuration Examples

NGINX Configuration (/etc/nginx/conf.d/):# Return 410 for permanently removed catalog
location ~ ^/discontinued-sku-[0-9]+/ {
  return 410 "Resource Permanently Removed";
}
Apache .htaccess Configuration:# Return 410 Gone for retired endpoints
Redirect 410 /catalog/discontinued-item
RewriteRule ^old-promo/.*$ - [G,L]

Frequently Asked Questions

HTTP 404 vs 410 SEO FAQs

Does Google treat HTTP 404 and HTTP 410 identically?

Google's documentation says it does: "All 4xx errors, except 429, are treated the same." Google doesn't use the content from URLs that return 4xx status codes, removes previously indexed URLs from the index, and crawls them less often over time ("The crawling frequency gradually decreases"). The documentation gives no timeframe and does not say that 410 URLs are removed faster. RFC 9110 still gives the two codes different meanings: 410 says the removal is likely permanent, while 404 does not say whether it is temporary or permanent.

Will having a large number of 404 errors hurt my overall site ranking?

Google's Search Console help says "404 responses are not necessarily a problem, if the page has been removed without any replacement," and recommends fixing only 404 errors that you link to yourself or list in a sitemap. Google's status-code documentation covers crawling and indexing and does not describe a site-wide ranking penalty for 404s. Fix broken internal links, keep sitemaps current, and use a 301 redirect when a page has moved.

Should I use 410 instead of 404 for discontinued products?

Either is valid for Google Search, which treats them the same. RFC 9110 says 410 fits when the server knows the condition is likely to be permanent, and that 404 ought to be used if the server does not know. It also says "it is not necessary to mark all permanently unavailable resources as 'gone' or to keep the mark for any length of time." If a direct replacement product exists, a 301 redirect to it is usually more useful to visitors.

Should I redirect all broken 404 URLs to my homepage with a 301 redirect?

Generally no. Google's site-move guide says: "Don't redirect many old URLs to one irrelevant single URL destination, such as the home page of the new site. This can confuse users and might be treated as a soft 404 error." Use a 301 only when a real replacement page exists, and return 404 or 410 otherwise.

How does Google identify a Soft 404 and how can I fix it in Google Search Console?

Google's Search Console help describes a soft 404 as a page that "returns a user-friendly 'not found' message but not a 404 HTTP response code." Google's status-code documentation says that when the content suggests an error, such as an empty page or an error message, Search Console shows a soft 404 error. To fix it, return a real 404 (or 410) for pages that are gone, use a 301 redirect when there is a replacement, or, if the page is real, add content that makes clear it is not an error page.

Sources

Primary Sources & Standards

Quotes on this page come from the documents below (source checked October 11, 2026). Google can change its documentation at any time; check the live pages before relying on a detail.

Spotted an outdated RFC reference or search engine guideline revision? Report a correction ↗

Related HTTP & SEO Tools