Do AI crawlers run JavaScript? Check the HTML they receive

You cannot count on an AI crawler to run your page’s scripts, so the answer you want quoted belongs in the HTML your server sends. Google documents that Googlebot renders JavaScript. The crawler pages from OpenAI and Perplexity do not say whether their agents do. In a December 2024 study of specific sites, Vercel and MERJ observed named OpenAI and Perplexity agents fetching pages without rendering JavaScript. That is a dated third-party observation on those sites, not vendor policy, and it does not establish how those agents behave today. You do not need to settle the question: a page that carries its answer in the first response is readable whether a crawler renders or not.

The distinction that matters is between the raw HTML your server sends and the finished page your browser paints. A browser runs scripts, sends stored cookies and fetches more text when a visitor clicks. A plain crawler fetch may read the initial response and stop there. So judge a page by what your server returns, not by how it looks on your screen.

What does JavaScript rendering mean?

Page text reaches a visitor in one of two ways. With server-side rendering, the server writes the article and its answers into the HTML response for each request. A static site does the same work once, at build time, so its HTML files also arrive with the text in place. Either way, the words are in the first response.

With client-side rendering, the first response often holds little more than navigation, an empty content container and <script> tags. The browser downloads and runs those scripts, which fetch the article or its data and insert the text into the live DOM. Turning that empty shell into a readable page inside the browser is client-side JavaScript rendering. A human visitor sees a finished page even when the first response held almost no body text.

Many production pages mix the two. The main article may arrive in the first response while an FAQ answer, a specification table or a price loads only after a script runs or a visitor clicks a tab. So the question is never about the page as a whole. It is about the exact sentence you want quoted: is it in the HTML returned for the URL, or does it appear only after a script or a click?

Which crawlers run JavaScript?

Two kinds of evidence answer two different questions. Vendor documentation, re-read for this article on 25 September 2026, tells you what a company says about its crawler. The Vercel and MERJ measurement, published in December 2024, tells you what named agents did on the sites it covered. Silence in a vendor document is not a statement that its crawler never runs JavaScript.

Agent	Vendor documentation	Vercel and MERJ observation (December 2024)
Googlebot (Google Search)	Google documents rendering with an evergreen Chromium renderer and indexing from the rendered HTML	Vercel noted that Gemini uses Google’s crawling infrastructure
OAI-SearchBot, ChatGPT-User, GPTBot (OpenAI)	The crawler overview and publisher FAQ do not say whether they execute JavaScript	Fetched pages without rendering JavaScript on the sites tested
PerplexityBot (Perplexity)	The crawler page does not say whether it executes JavaScript	Fetched pages without rendering JavaScript on the sites tested

Google’s JavaScript SEO guide splits processing into three phases: crawling, rendering and indexing. A page can wait in the rendering queue “for a few seconds, but it can take longer than that,” and Google uses the rendered HTML to index it. Google’s separate AI features page states that a page must be indexed and eligible to be shown with a snippet in Google Search to serve as a supporting link in AI Overviews or AI Mode. Meeting those conditions makes a page eligible; it does not guarantee a link.

The same JavaScript guide still recommends server-side or pre-rendering and notes that “not all bots can run JavaScript.” OpenAI’s crawler overview and Perplexity’s crawler page list their user agents and point to published IP ranges. Neither mentions JavaScript execution, and OpenAI’s publisher FAQ does not either.

Vercel’s study, published with MERJ in December 2024, analyzed crawler traffic across nextjs.org, Vercel’s network and two job-board sites. On those sites it observed OAI-SearchBot, ChatGPT-User, GPTBot and PerplexityBot fetching pages without rendering JavaScript. It also noted that ChatGPT and Claude crawlers downloaded JavaScript files without executing them. Treat these findings as a dated third-party measurement on specific sites, not as vendor policy.

Both kinds of evidence point to the same design rule: put public, important answers in the HTML returned for their URL, so a plain fetch receives them without a rendering step.

Check what your server sends

Pick a page that matters and the one sentence on it that answers a buyer’s or reader’s question. Then compare the first server response with the finished browser view in three ways:

1. View the page source. In Chrome, Edge or Firefox, right-click the page, choose View Page Source (Ctrl+U on Windows and Linux), and search for the sentence. View Page Source shows the raw HTML returned for that browser request. The Inspect panel shows the live DOM after scripts may have changed it, so it cannot tell you what arrived in the first response.
2. Switch JavaScript off. Open the page in a private window so stored cookies do not change the response. In Chrome DevTools, open the Command Menu with Ctrl+Shift+P (Cmd+Shift+P on macOS), type javascript, run Disable JavaScript and reload the page. This approximates a visit without scripts; it does not reproduce a crawler’s network or user agent.
3. Fetch the URL with curl, a command-line tool that saves the server response without running any of the page’s scripts:

curl -sSL --compressed -D headers.txt -o page.html 'https://example.com/your-page/'
grep -c -F 'the exact sentence you want quoted' page.html

The first command follows redirects (-L), asks for a compressed response and unpacks it (--compressed), writes the headers of every response, redirects included, to headers.txt, and saves the final body to page.html. The second counts the lines in page.html that contain your exact string. The curl manual explains each flag.

A 0 is a lead, not a verdict. It means that exact string never appears whole on a single line of page.html; it does not prove the words are absent. An HTML entity such as &, a curly quote, a line break or an inline tag (<strong>, <a>) inside the sentence will break a full-sentence match. Search page.html for a short, distinctive phrase of two or three words and read the markup around it before you mark the answer absent. Then open headers.txt: an error status, or a redirect to a login screen or the homepage, changes what the test measured.

How do you read the result?

What you find	What it establishes	What to do
The sentence sits in a normal paragraph in page.html	A plain fetch of this URL receives the answer as body text	Record the result and test the next page
It appears only inside a <script> block or JSON payload	The words arrive, but as script data rather than body text	Put the core answer in standard body HTML; do not rely on a script payload
It appears only after a click on a tab or “read more,” and the Network panel shows a new request returning it	The click fetches the answer after the page has loaded	Put the answer in the initial HTML and let the tab only show or hide it
It appears in the browser shortly after load, but is missing from page.html	A client-side script fetched or built the text after the first response	Move the core text to server-side rendering or build-time generation
page.html holds only a cookie wall or consent screen	This fetch received the gate, not the article	If the page is meant to be public, send the article text in the initial HTML and lay the consent layer over it
page.html is a login screen, an error page or the homepage	Your test never reached the intended page	Check the URL, redirect rules and access controls before you debug rendering

A tab can hide text or fetch it

Two tabs or accordions can look identical in a browser and behave differently on the wire. In the first pattern, all panel text is already in the initial HTML; a click only toggles CSS visibility or a hidden attribute. In the second, the panel stays empty until a click sends a network request that fetches the answer and inserts it into the DOM. A crawler that reads the initial HTML and never clicks receives the first tab’s text, not the second’s.

So write the ticket about the network behavior, not the widget: “The answer to X is absent from the initial HTML for this URL; clicking the FAQ tab triggers request Y to load it.” The DevTools Network panel shows you request Y. That a page uses tabs or accordions tells you nothing until you have seen what the server sent.

A worked example (hypothetical)

Suppose a software company in Utrecht has a pricing page with an FAQ that answers a common buyer question: “Is our data stored in the EU?” The answer sits inside a tab. In a regular browser window, the page looks complete.

In this hypothetical check, the team runs the curl and grep commands against the pricing URL, and grep -c returns 0. The DevTools Network panel shows why: clicking the FAQ tab sends a request to /api/faq?page=pricing, which returns the answer as JSON. The initial HTML in page.html holds only the tab button and an empty container.

A developer changes the page so the server renders the FAQ answers into the HTML response, and the tab script only hides or reveals text that is already in the DOM. When the team reruns curl, grep -c returns 1, and the sentence sits inside a standard <p> tag. They confirm the result by reloading the page in a private window with JavaScript disabled.

What the fix proves: this URL now delivers the answer in its initial HTML, with no JavaScript execution required. What it does not prove: that ChatGPT will cite the page. That takes a separate, logged test in ChatGPT Search, which the citation-readiness audit sets out.

How do you fix the page?

• Render core answers on the server or at build time. Vercel’s advice after its December 2024 study makes the same split: render critical content on the server and keep client-side rendering for extras such as view counters, chat widgets and live feeds.
• Let tabs reveal text that is already in the HTML. Keep FAQ and specification text in the initial server response, and use a script or a native <details> element only to show and hide it.
• Keep the name next to the claim. Put the source, company or author in the same HTML as the answer, so a fetcher receives both together. How to get cited by ChatGPT covers that step: a name the machine can attach.
• Do not treat llms.txt as a substitute for server-rendered HTML. A separate text file does not put a missing answer into the HTML returned for your page URLs, and Google states that its AI features require no special AI text files. What is llms.txt? explains what the file is for.

What this test cannot tell you

A curl request from your laptop is not a fetch from OpenAI’s, Perplexity’s or Google’s infrastructure. Network location, request headers, CDN caches and firewall rules can all change what your server returns to a real bot. If a crawler is blocked at the edge or disallowed in robots.txt, server-side rendering will not let it in. The three OpenAI agents in the table also do different jobs: OAI-SearchBot for search in ChatGPT, GPTBot for model training and ChatGPT-User for pages a person asks ChatGPT to open. GPTBot vs OAI-SearchBot vs ChatGPT-User explains how their permissions differ.

Readable is not cited. A pass means the response you fetched carries the answer as plain HTML. It does not mean the page is indexed, snippet-eligible or chosen as a citation. A fail today does not prove an answer engine has never seen your claim, for example through a mention on another site or an earlier crawl. Log your HTML results separately from your citation checks.

Try this today

1. Pick three public pages you want cited and copy one core answer sentence from each.
2. View each page’s source or run the curl and grep commands above. If the exact match returns 0, search page.html for a short, distinctive phrase from that sentence.
3. Log one of four outcomes per URL: present in body HTML, present only in script or JSON data, split by inline markup, or absent. Note the HTTP status and any redirect from headers.txt next to it.
4. For each absent answer, open the DevTools Network panel, find the click or script request that loads the text, and give the developer the page URL, the missing sentence and the request URL.

Close the ticket when a fresh curl fetch of the public URL returns the answer in body HTML. Whether answer engines then cite the page belongs in a separate audit.

Cite this:Do AI crawlers run JavaScript? Check the HTML they receive.Len P. van der Hof. https://lenvanderhof.com/en/blog/javascript-rendering-kills-citations/ · Published 22 October 2026.