---
title: "Pages returning HTTP errors - Serpwise"
description: "404, 410, 451, 500, 503 each tell a crawler something different. Pick the right code, find the patterns, fix the source - don't paper over with redirects."
url: "https://serpwise.ai/learn/pages-with-http-errors/"
source: "https://serpwise.ai/learn/pages-with-http-errors/"
---
[← Back to Learn](/learn/)

Crawling & Indexing Critical

# Pages returning HTTP errors

404, 410, 451, 500, 503 each tell a crawler something different. Pick the right code, find the patterns, fix the source - don't paper over with redirects.

HTTP status codes are the only language crawlers actually trust. A page that returns the wrong code at the wrong time can disappear from the index for weeks. A page that returns the right code recovers in a single crawl.

This guide covers the codes that matter for SEO, what each signals to Googlebot, and how to find the patterns behind error clusters before they pile up.

## The codes that matter

| Code  | Meaning                       | What Google does                                     |
|-------|-------------------------------|------------------------------------------------------|
| `200` | OK                            | Index normally                                       |
| `301` | Moved permanently             | Pass signals to target, eventually drop old URL      |
| `302` | Found (temporary)             | Keep old URL indexed, do not pass signals strongly   |
| `304` | Not modified                  | Reuse cached copy, no re-fetch needed                |
| `404` | Not found                     | Drop from index after a few retries                  |
| `410` | Gone                          | Drop from index faster than 404                      |
| `451` | Unavailable for legal reasons | Drop and mark; respect the legal signal              |
| `500` | Server error                  | Keep page, retry - but extended 5xx tanks crawl rate |
| `503` | Service unavailable           | Treat as temporary if `Retry-After` is set           |

Two practical rules:

1. **If a page is gone forever and you don’t have a replacement, use `410`.** It tells Google “stop checking” faster than `404`.
2. **If your origin is briefly down, return `503` with `Retry-After`.** Persistent `500`s convince Google the site is broken and slow the crawl. `503` is the polite signal.

## How to detect it

The signal is in your access logs, filtered to crawler user-agents.

```
# Top error URLs hit by Googlebot in the last 7 days
awk '$9 ~ /^[45]/ && $0 ~ /Googlebot/ { print $9, $7 }' access.log \
  | sort | uniq -c | sort -rn | head -30
```

For a richer view, use the **Pages** report in Google Search Console:

- “Not found (404)” - actual 404s Google encountered
- “Server error (5xx)” - sustained 5xx
- “Soft 404” - your origin returned `200` but the page is empty or says “not found” in the body. Soft 404s are worse than real 404s because they waste crawl on dead pages.

## The fix

### Universal - pick the right code

```
HTTP/1.1 410 Gone
Content-Type: text/html
```

Returned for URLs that are gone for good (discontinued products, deleted user profiles).

```
HTTP/1.1 503 Service Unavailable
Retry-After: 3600
```

Returned during planned maintenance. Tells Google to come back in an hour rather than re-checking aggressively.

```
HTTP/1.1 301 Moved Permanently
Location: https://example.com/new-url
```

Returned when there’s a true replacement URL. Don’t redirect everything to the homepage - that’s interpreted as a soft 404.

### WordPress

WordPress returns `200` for deleted pages by default (a “Page not found” template with a `200` status). That’s a soft 404. Fix with the `wp_die` template override or via a plugin like Yoast (Tools → Crawl optimization → 404 handling).

For permanent removals - discontinued products, deleted authors - use a redirect manager to issue a `410`, not a `301` to the homepage.

### Shopify

Deleted products redirect to a 404 template that returns the right status. The problem is variants: deleting variant URLs that were previously crawled leaves them in Google’s index with no replacement. Use 301 redirects from the deleted variants to the parent product (Online Store → Navigation → URL redirects).

### Nginx

```
# Permanently gone - use 410
location = /products/discontinued-widget {
  return 410;
}

# Maintenance window - use 503 with Retry-After
location / {
  return 503;
  add_header Retry-After 3600 always;
}
```

### Apache

```
# Permanently gone
Redirect 410 /products/discontinued-widget

# Maintenance window
RewriteEngine On
RewriteCond %{REQUEST_URI} !=/maintenance.html
RewriteRule ^.*$ /maintenance.html [R=503,L]
Header always set Retry-After "3600"
```

### Next.js (App Router)

```
// app/products/[slug]/page.tsx
import { notFound } from "next/navigation";

export default async function Page({ params }: { params: { slug: string } }) {
  const product = await getProduct(params.slug);
  if (!product) notFound(); // Next emits 404 with the proper status
  return <Product product={product} />;
}
```

For `410 Gone`, set the status in a route handler:

```
// app/products/discontinued-widget/route.ts
export function GET() {
  return new Response("This product has been discontinued.", { status: 410 });
}
```

## Pitfalls

**Don’t redirect every 404 to the homepage.** Google treats large numbers of redirects-to-root as soft 404s. The signal is “this page didn’t exist, and you have nothing to replace it with” - return `410` instead.

**Don’t rely on `meta robots noindex` to remove a page.** It works, but only after Google re-crawls. For permanent removals, a `410` is faster and more decisive.

**Don’t `301` discontinued products to a similar product without thinking.** If the user intent doesn’t match, you’ll get a soft 404 anyway. Either redirect to a real replacement (same product, new SKU) or return `410`.

**`500` errors are the silent killer.** A flaky origin returning intermittent 500s teaches Google to slow the crawl. Fix the root cause; don’t paper over it with retries at the CDN.

## Fix at the edge with Serpwise

Most teams discover an HTTP error pattern after damage is done - a deploy that 500’d for 90 minutes during a Googlebot surge, a CDN misconfig returning 404 for a section of the site, a CMS update that broke every URL containing a slash.

Serpwise can return the right status code for any URL pattern at the edge, immediately, without an origin deploy. Convert a cluster of 404s to 410. Issue a 503 with `Retry-After` during a maintenance window. Redirect a section of URLs based on a CSV import. The fix is live in seconds; the long-term cleanup lands in your stack on its own schedule.

[See pricing](/pricing) or run a [free AI visibility audit](/audit).

From diagnosis to deployment

## Find the issue. Ship the fix.

Use Learn to understand the problem, then run Serpwise against your own site to see what can be approved and deployed.

[Run free audit](/audit/) [Book a demo](/demo/)