Most AI crawlers used by GPTBot, PerplexityBot, ClaudeBot, and similar bots are generally less capable at rendering JavaScript than Googlebot, and several reportedly do little to no JS execution at all, relying mostly on the raw server-delivered HTML. If your React, Next.js, or other JavaScript-heavy site relies on client-side rendering for its core content, there's a real risk these crawlers see a mostly empty page. Server-side rendering or static generation for anything you want indexed and cited is the safest approach in 2026.
I'll be direct about the limits of what's confirmed here: none of the major AI companies publish a detailed, continuously updated technical spec of exactly how their crawlers render pages, and behavior can change without notice. What follows is based on widely reported patterns from the SEO and web-dev community, documented crawler behavior where available, and general best practice, not a guarantee of any specific crawler's current behavior. Always verify against your own site (see the testing section below).
Why This Matters More for Startups and Edtech Sites
A huge share of startup and edtech product sites are built on React, Next.js, Vue, or similar frameworks, often because the same team building the product is building the marketing site, and it's faster to reuse the stack. That's a reasonable engineering trade-off, but it creates a specific SEO/AEO risk: if your blog, docs, or landing pages render content client-side (i.e., the initial HTML response is a near-empty shell that JavaScript fills in after load), any crawler that doesn't execute JS will index or cite essentially nothing from that page.
This isn't a hypothetical edge case. It's one of the most common technical issues I find auditing marketing sites for growth-stage startups, the content looks complete in a browser, but the raw HTML response is a skeleton.
What's Generally Understood About Each Crawler
Googlebot has supported JavaScript rendering for years, using a rendering pipeline built on a recent Chromium engine. It's the most JS-capable crawler in the search space by a wide margin, though even Googlebot's rendering happens on a delay (it fetches, queues for rendering, then indexes) and isn't infinitely patient with slow-loading or render-blocking scripts.
GPTBot (OpenAI) is generally reported to have limited or no JavaScript rendering capability, it's largely understood to fetch raw HTML rather than execute a full browser rendering pipeline, though OpenAI has not published a detailed technical spec confirming exact behavior, and this could change over time.
ClaudeBot (Anthropic) similarly is not documented as running a full JS rendering pipeline; treat it as likely HTML-only unless you've specifically tested and confirmed otherwise for your site.
PerplexityBot and CCBot (Common Crawl, which many AI systems train on) are generally understood to behave similarly, fetching raw HTML without executing client-side JavaScript, though again, this is based on community testing and reporting rather than official published specs.
The pattern across nearly all of them: assume no JS rendering unless you've tested and proven otherwise. Googlebot is the outlier that can handle it, not the norm.
The Practical Fix: Don't Rely on Client-Side Rendering for Critical Content
Server-side rendering (SSR) or static site generation (SSG) should be the default for any page you want crawled, indexed, and cited, blog posts, docs, product/pricing pages, case studies. Frameworks like Next.js support this natively (getServerSideProps / generateStaticParams / the App Router's server components), so this is often a configuration decision more than a rewrite.
Specifically:
- Static generation (SSG) is ideal for content that doesn't change per-user: blog posts, marketing pages, documentation. Pre-render at build time so every crawler, JS-capable or not, gets full HTML on first fetch.
- Server-side rendering (SSR) works for content that needs to be dynamic per-request but still needs to be crawlable, e.g., a personalized landing page that still needs an SEO-friendly fallback state.
- Client-side rendering (CSR) should be reserved for genuinely interactive, logged-in, or app-like experiences that don't need to be indexed or cited at all, dashboards, in-product tools, anything behind auth.
The Fallback: Pre-rendering / Dynamic Rendering Services
If a full SSR/SSG migration isn't feasible right now (common with legacy CSR apps where marketing content is entangled with the product app), a pre-rendering service is a reasonable interim fix. Tools like Prerender.io, or a self-hosted headless-browser middleware, detect bot user agents and serve them a pre-rendered, fully-loaded HTML snapshot while regular users still get the normal client-rendered experience.
This is a patch, not a permanent architecture, it adds infrastructure complexity and another thing that can silently break, but it's a legitimate stopgap while you plan a proper SSR/SSG migration for your highest-value pages.
How to Test What a Crawler Actually Sees
Don't guess, verify directly. Three practical methods:
1. Fetch the raw HTML with a bot user-agent
Use curl (or a similar tool) to fetch your page while spoofing a known crawler's user-agent string, and inspect what comes back before any JavaScript runs:
curl -A "GPTBot" https://yoursite.com/your-pageCompare the output to what you see when you view-source in a browser (which also shows raw HTML, pre-JS). If your critical content, headline, body copy, key data, isn't present in that raw response, a non-JS-rendering crawler won't see it either.
2. Compare "View Source" vs "Inspect Element" in your browser
"View Source" (Ctrl/Cmd+U) shows the original server response, before JavaScript executes. "Inspect Element" / DevTools shows the DOM after JavaScript has run. If your important content only appears in the Inspect view and not in View Source, you have a client-side-rendering-dependent page.
3. Use Google Search Console's URL Inspection tool
This shows you what Googlebot's rendering pipeline sees after executing JavaScript, alongside the raw fetched HTML. It's specific to Google, but it's a useful sanity check for whether rendering is working at all, and a reasonable proxy for "if even Googlebot struggles here, everyone else will too."
4. Check server logs for crawler visits
Filter your server or CDN logs for known bot user-agent strings (GPTBot, ClaudeBot, PerplexityBot, CCBot, Google-Extended) to confirm they're actually reaching your pages in the first place, a rendering problem doesn't matter if the crawler isn't visiting at all, which is a separate robots.txt/crawlability issue worth checking alongside this.
A Quick Checklist
- [ ] Identify every page type you want AI-crawled and cited (blog, docs, pricing, case studies, key landing pages)
- [ ] Confirm those page types are server-rendered or statically generated, not purely client-rendered
- [ ] curl-test your top 10 pages with at least one non-Google bot user-agent and check the raw HTML for your core content
- [ ] Compare View Source vs Inspect Element on a sample of pages
- [ ] Check server/CDN logs to confirm crawler visits are actually happening
- [ ] If full SSR/SSG isn't feasible short-term, evaluate a pre-rendering service as a stopgap
- [ ] Re-test after any major frontend framework upgrade or migration, rendering behavior can regress silently
FAQ
Does GPTBot render JavaScript? It's generally reported and widely assumed within the SEO community that GPTBot does not reliably execute JavaScript, relying instead on raw HTML. OpenAI hasn't published a detailed technical confirmation, so test your own site directly rather than relying solely on this assumption.
Is Googlebot the only crawler I need to worry about for JS rendering? No, Googlebot is generally the most capable at JS rendering among major crawlers, which means optimizing only for Google can leave you invisible to AI-specific crawlers like GPTBot, ClaudeBot, and PerplexityBot that appear to have weaker or no rendering support.
Will switching to SSR hurt my site's performance or user experience? Not inherently, modern frameworks like Next.js are designed for SSR/SSG without sacrificing interactivity, since JavaScript still hydrates the page for users after the initial HTML loads. The initial HTML is what crawlers see; the hydrated, interactive version is what users experience.
How often should I re-test crawler rendering? After any significant frontend deployment, framework upgrade, or CMS migration, and as a routine check every few months, since crawler capabilities and your own site's rendering setup can both change.
Is a pre-rendering service a permanent solution? Treat it as a bridge, not a destination. It adds a dependency and another point of failure; migrating critical content to true SSR/SSG is the more durable long-term fix.
Auditing whether a client's React or Next.js marketing site is actually visible to AI crawlers, not just to Googlebot, has become a standard part of the technical SEO/AEO work I do with edtech and startup teams, since it's one of the most common and most fixable gaps I find.