A 41-Day Test Just Showed AI Crawlers Can't See Your JavaScript Links
A 41-Day Test Just Showed AI Crawlers Can't See Your JavaScript Links
A controlled 41-day experiment split one site's internal links in half: half hardcoded in HTML, half injected by JavaScript. Googlebot was the only crawler that executed the JavaScript at all, and it still only reached about 2% of those links. GPTBot, ClaudeBot, Bingbot, and every other major bot found exactly zero.
That's not a rendering delay. That's a permanent blind spot.
If you've spent this year refreshing old posts, adding internal links, and updating pages to earn AI citations, this is worth stopping to check.
What the Experiment Actually Found
The test ran a logging layer that recorded every bot request across roughly 2,400 pages for 41 days. Half the site's internal links lived in plain HTML. The other half only appeared after JavaScript ran in the browser.
Google's crawler stack was the sole exception that attempted to render JavaScript and follow what it found there. Even Googlebot only reached about 2% of the JavaScript-linked pages during that window.
GPTBot, ClaudeBot, Bingbot, and crawlers from Meta and Amazon found none of them. Not slowly. Not eventually. Zero.
When the site owner converted those same links to plain HTML, the picture flipped fast. GPTBot discovered roughly 250 new pages quickly. Bingbot found about 212.
Why This Isn't a Google Story
It's tempting to read this as one more JavaScript SEO footnote, the kind of thing that's been true for a decade and mostly stopped mattering once Googlebot got better at rendering.
That read misses what's different here. Googlebot did evolve to handle JavaScript-heavy pages reasonably well, which is exactly why a site can still look fine in traditional search results.
AI crawlers didn't follow the same path. A separate analysis of over 500 million crawler fetches found that none of the major AI bots, including OpenAI's and Anthropic's, execute JavaScript at all. They fetch the raw HTML response and move on.
They'll download a JavaScript file if your page links to one. They just never run it.
Why This Should Change How You Prioritize Refreshes
A refresh is invisible if the bot never sees it
Most refresh work focuses on what's on the page: updated data, a rewritten section, a stronger answer to the reader's question. All of that assumes something is actually crawling the page.
If the refreshed content, or the internal link that points to it, only exists after JavaScript runs, an AI crawler never gets that far. The improvement is real. It's also invisible to roughly the entire AI search ecosystem.
Internal linking is the part people forget to check
Content teams usually know if their CMS renders article bodies client-side. Fewer check whether the links connecting those articles are also JavaScript-dependent.
A related-posts widget, a "read next" module, or a JavaScript-built navigation menu can quietly wall off an entire section of a site from every AI crawler at once.
Old pages carry the most risk
Pages published years ago on an older template are the most likely to have JavaScript-dependent navigation baked in, simply because that's how the site was built at the time.
Those are often the exact pages sitting in a refresh queue right now, getting new content poured into a shell that AI crawlers can't reach in the first place.
How to Check Your Own Site
Read the raw HTML, not the rendered page
View your page's source through a plain HTTP request instead of a browser render. In a terminal, curl the URL and search the response for the links and content you expect to see.
If they're missing from that raw response, they only exist after JavaScript executes. That's precisely what GPTBot and ClaudeBot never do.
Test a few page types, not just the homepage
Check an article page, a category or tag page, and whatever internal linking module connects them. Each template can render differently, so one clean result doesn't clear the rest of the site.
Fix navigation before you fix copy
If you find JavaScript-dependent links, server-side rendering is the most direct fix, since it delivers the HTML and links from the first response. Prerendering or dynamic rendering can work as a fallback where a full rebuild isn't realistic right now.
Either way, fix the discovery path before investing more hours in the content behind it. A great refresh on an unreachable page doesn't do anything yet.
FAQ
Does this only affect AI search, or does it hurt regular Google rankings too? Mostly AI search. Googlebot still renders JavaScript reasonably well for traditional ranking purposes, even though this test found its coverage of JS-only links was far from complete. AI crawlers are the ones with essentially no JavaScript rendering at all.
How do I know if my site has this problem?
Pull the raw HTML response for a few key pages with a tool like curl and check whether your main content and internal links appear in it. If they only show up after the page renders in a browser, an AI crawler won't see them.
Is server-side rendering the only real fix? It's the most complete one. Prerendering and dynamic rendering are workable fallbacks that serve a fully rendered HTML snapshot to bots while keeping the JavaScript experience for regular visitors.
Should I stop refreshing content until this is fixed? No, but sequence the work. Confirm your key templates deliver content and links in raw HTML first, then keep refreshing. Otherwise you're improving pages that a large share of AI crawlers can't reach anyway.
Find Out Which of Your Pages Are Actually Reachable
Refresh priority is usually based on what's decaying in Search Console. This test is a reminder to also ask whether a page is discoverable in the first place. SEORefresher tracks which pages are losing visibility so you know where to focus once the technical basics are covered. Start finding out at seorefresher.com.