Googlebot eventually executes JavaScript, but with a delay (sometimes hours, sometimes days) and a limited execution budget. Every other crawler — Bingbot, GPTBot, OAI-SearchBot, ClaudeBot, Claude-User, PerplexityBot, CCBot, Google-Extended for the AI Overviews pipeline — either doesn’t execute JS at all or does so unreliably. If your content depends on client-side rendering, the AI bots see an empty shell and the slow lane of Googlebot indexing your real content with significant lag.
Why it matters
Five patterns that silently break crawling, in rough order of frequency:
- Content loaded after hydration. The initial HTML response is
<div id="root"></div>plus a script tag. To a non-rendering crawler, that’s the entire page. - Hydration errors. A mismatch between server and client renders causes React/Vue/Svelte to throw out the server HTML and re-render client-side. The first render Googlebot reads might disappear when hydration fails.
- Content loaded on user interaction. Tabs, accordions, modals, “load more” buttons that fetch content on click. Crawlers don’t click.
- Infinite scroll without paginated fallbacks. Crawlers don’t scroll. The content past the initial viewport is invisible.
- SPA routing without server responses. A client-side router that returns 200 for every URL — including ones that should be 404 — creates soft-404 patterns that confuse the index.
This is the single most common AI visibility failure mode in 2026. Most modern AI bots have explicitly stated they don’t render JavaScript.
How to detect it
The single most useful test: fetch with curl and look at what comes back:
curl -s -A "Mozilla/5.0 (compatible; ClaudeBot/1.0; +claudebot@anthropic.com)" \
https://example.com/some/page | wc -wA page with substantial content should return hundreds of words. A 50-word response is almost certainly a CSR shell.
For comparison, fetch the same URL with a rendering crawler (Googlebot Smartphone):
# Use Search Console → URL Inspection → Test Live URL → View Tested Page → HTMLThe “Tested page” HTML in Search Console shows exactly what Googlebot rendered. If the rendered HTML contains your content but curl does not, you have CSR content. That content is invisible to every AI crawler.
For pinpointing hydration errors, the browser console will log them on every page load — search for “Hydration failed” or “Text content did not match.”
The fix
Universal — server-render the critical content
The principle: anything that must be indexed must be in the initial HTML response. Hydration is fine for interactivity; it’s not fine as the sole source of content.
<!-- Bad: empty shell, content rendered client-side -->
<div id="app"></div>
<script src="/app.js"></script>
<!-- Good: content in the response, hydrated for interactivity -->
<main>
<h1>Knee-length wool overcoat</h1>
<p>Tailored from Italian merino…</p>
<ul class="specs">
<li>100% wool</li>
<li>Made in Portugal</li>
</ul>
</main>
<script src="/app.js"></script>The four common server-rendering strategies:
- SSR (Server-Side Rendering) — render the page per request on the server. Next.js, Remix, Nuxt, SvelteKit, Astro do this.
- SSG (Static Site Generation) — render at build time, serve static HTML. Fastest, only works for content that doesn’t change per visitor.
- ISR (Incremental Static Regeneration) — SSG with revalidation. Best of both for content-heavy sites.
- Dynamic rendering — serve SSR to bots, CSR to humans. Google supported this as a workaround but discourages it now. Acceptable for AI bots that can’t render JS.
Fix hydration errors
Hydration errors usually come from:
Date.now()orMath.random()rendered server-side, then re-rendered with a different value client-side.- Conditional rendering on
typeof window !== "undefined"— server renders one tree, client renders another. - Locale-dependent formatting (
toLocaleDateString()) with no fixed locale.
Lock to a deterministic value or use suppressHydrationWarning (React) / <ClientOnly> (Nuxt/Vue) wrappers for genuinely client-only content. Don’t suppress errors; fix the mismatch.
Don’t gate content on user interaction
<!-- Bad: tab content fetched on click -->
<button onclick="fetchTab('specs')">Specs</button>
<div id="tab-content"></div>
<!-- Good: tab content in DOM, hidden until interacted with -->
<button aria-controls="specs">Specs</button>
<div id="specs">…full specs content…</div>CSS or JS can hide and show; it must not fetch on demand for content that needs to be indexed.
Paginated fallbacks for infinite scroll
<!-- Real navigable URLs the crawler can follow -->
<a href="/blog?page=2" rel="next">Page 2</a>
<a href="/blog?page=3">Page 3</a>Each paginated URL must return its own server-rendered HTML. The infinite scroll layer is a UX enhancement on top.
Return real 404s from your SPA router
For client-side routers, this means a server fallback that returns 404 for unknown routes. Returning 200 with a “not found” page in the body is a soft 404 — Google flags it, and AI bots index empty content.
// Next.js App Router — use notFound() for invalid routes
import { notFound } from "next/navigation";
export default async function Page({ params }) {
const post = await db.query.posts.findFirst({ where: { slug: params.slug } });
if (!post) notFound(); // returns real 404
return <Article post={post} />;
}WordPress
WP is server-rendered by default. JS rendering failures usually come from theme/plugin patterns that load critical content via REST API on the client (e.g., a product grid powered by a JS widget). Audit by viewing source — if the content isn’t in the HTML response, it’s not indexable for non-rendering bots.
Next.js / Remix / Nuxt / SvelteKit / Astro
Default to server rendering. Use "use client" / <script client:only> only for components that genuinely need it — sliders, charts, interactive forms. The content of the page should land in the server response.
Common pitfalls
- Trusting Googlebot’s rendering will save you. It usually does — eventually, with delay. But AI bots won’t, and the delay matters for time-sensitive content.
- “Dynamic rendering” as a permanent solution. Maintaining two render paths is technical debt. Use it as a transition strategy while migrating to SSR.
- Lazy-loading critical content via
IntersectionObserver. Crawlers don’t always trigger intersection events reliably. Server-render anything above the fold. - Cookie banners or paywalls blocking content for crawlers. If the bot sees a wall instead of the article, the article isn’t indexed. Either let bots through, or use Google’s allowed flexible sampling patterns.
FragmentURLs (/page#section). Everything after#is invisible to crawlers. Use real URL paths.