Is your website readable by ChatGPT? A JavaScript check for South African sites

ChatGPT can read your website only if your words are in the HTML your server sends. If your text appears only after JavaScript runs in a browser, the crawlers behind ChatGPT may see an almost empty page, and an assistant cannot quote what it cannot read.

Many modern sites send the browser a near-empty HTML file plus a bundle of JavaScript that draws the page. Visitors never notice. A crawler that does not run that JavaScript sees only the shell. This post explains which crawlers are affected, how to check your own site in five minutes, and what we found when we checked ours.

In short

• OpenAI, Anthropic and Perplexity do not document that their crawlers run JavaScript, and crawler data published by Vercel and MERJ in December 2024 found that none of the major AI crawlers did.
• Google is different: it renders JavaScript before indexing. Even so, Google's own documentation recommends server-side rendering or prerendering because "not all bots can run JavaScript".
• You can see what a crawler sees with one curl command and the crawler's user-agent string.
• The fix is server-side rendering or prerendering, so the HTML already contains the words.
• Our own homepage showed crawlers 66 words before we prerendered it, and about 2,090 words after (October 2026).

Why can't some AI crawlers see JavaScript content?

A browser does two jobs. It downloads the HTML, then it runs the JavaScript, which on many sites adds most of the text. A crawler that only downloads the HTML stops after the first job.

OpenAI's crawler documentation explains what each of its user agents is for, but says nothing about running JavaScript. Anthropic's and Perplexity's crawler pages are the same. The best public measurement is "The rise of the AI crawler", published by Vercel and MERJ on 17 December 2024. It found that "none of the major AI crawlers currently render JavaScript". ChatGPT's and Claude's crawlers did download JavaScript files (11.50% and 23.84% of their requests) but did not execute them. That study is almost two years old, so test your own site rather than rely on it.

Google works differently. It processes JavaScript pages in three phases, crawling, rendering and indexing, and renders with an up-to-date version of Chromium. That is why a JavaScript-only site can rank in Google and still be close to invisible to ChatGPT. Google's documentation still adds a warning: server-side or pre-rendering "is still a great idea because it makes your website faster for users and crawlers, and not all bots can run JavaScript."

Which crawlers does ChatGPT use?

OpenAI documents three user agents that matter to a marketing team:

User agent	What OpenAI's documentation says it does
OAI-SearchBot	Surfaces websites in ChatGPT's search features. Sites that block it "will not be shown in ChatGPT search answers"
GPTBot	Crawls content that may be used to train OpenAI's models
ChatGPT-User	Visits a page when a user asks ChatGPT a question. Not used to crawl the web automatically

If OAI-SearchBot cannot read your page, ChatGPT's search answers have nothing of yours to quote. Which crawlers you allow is a separate decision, covered in robots.txt for AI crawlers.

How to check whether ChatGPT can read your website

You need curl, the command-line tool most developers already have, and five minutes.

1. Pick a sentence you can see on the page in your browser. Choose one from the main content, not the menu or footer.
2. Fetch the page as GPTBot and search the raw HTML for that sentence, using the first command below. A result of 0 means the sentence is not in the HTML: a crawler that does not run JavaScript cannot read it.
3. Check the status code with the second command. 200 is good. A 403, or a "checking your browser" page, usually means a firewall or CDN is blocking bots. Perplexity's documentation tells site owners with a web application firewall to allow its crawlers explicitly, and Google's AI features page lists making sure crawling is allowed "by any CDN or hosting infrastructure".
4. Compare titles and canonicals. Fetch two or three different pages and look at the <title> and <link rel="canonical"> in each. If every URL returns the homepage's title and canonical, crawlers see one page, not a site.
5. Look at the page with JavaScript off. In Chrome DevTools, open the Command Menu (Control+Shift+P, or Command+Shift+P on a Mac), type "javascript" and choose Disable JavaScript, then reload. What is left is roughly what a non-rendering crawler gets. JavaScript stays off in that tab while DevTools is open.

Step 2, with a placeholder address to replace with your own page:

curl -s -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \
  https://www.example.com/your-page | grep -c "a sentence from your page"

Step 3:

curl -s -o /dev/null -w "%{http_code}\n" \
  -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \
  https://www.example.com/your-page

The user-agent string is the example OpenAI publishes, and OpenAI notes the version number may change. Most servers send every visitor the same HTML, so this test shows what any non-rendering crawler gets, including those from Anthropic and Perplexity.

What we found on our own site

In October 2026 we ran this check on CitedAI's own site, citedai.co.za. It was a single-page app: the server sent the same small HTML shell for every address, and JavaScript drew the page. A crawler that does not run JavaScript saw 66 words on the homepage. Every public address also returned the homepage's title and canonical, so to a crawler the whole site looked like one thin page.

We switched the public pages to prerendering. At build time, each public page is rendered to complete HTML with its own title, description and canonical.

What a non-rendering crawler receives	Before: single-page app	After: prerendered (checked 11 October 2026)
Words on the homepage	66	about 2,090
Title and canonical	The homepage's, on every address	Each public page has its own
Login page	The 66-word shell	Still the 66-word shell, marked noindex

The app behind the login stays a single-page app on purpose. Nobody needs a login screen in search results or AI answers.

How do you fix a website ChatGPT can't read?

Choose the lightest option your stack supports. Google recommends "server-side rendering, static rendering, or hydration", and describes dynamic rendering (detecting bots and serving them a separate prerendered version) as "a workaround and not a long-term solution".

Approach	How it works	Best for
Static rendering (prerendering)	Each page is rendered to full HTML once, when the site is built	Marketing pages, blog posts, product pages that change when you publish
Server-side rendering	The server builds full HTML for every request	Pages that change often, such as live prices or stock levels
Hydration	The browser receives server-rendered HTML first, then JavaScript makes it interactive	Pages that need both readable text and rich interactivity

Ask your developer or agency which of these your framework supports, then rerun the curl test after the change. The sentence you picked should now appear in the raw HTML.

The same rule applies to schema markup. JSON-LD that JavaScript adds after the page loads is invisible to a crawler that does not run JavaScript; structured data for AI search covers what is worth adding. And readable HTML is a precondition, not a guarantee: whether an assistant quotes you depends on what the page says, which is the subject of how to write content AI assistants quote.

Does this matter for Google's AI Overviews?

Less directly. Google says a page must be indexed and eligible to appear in Google Search with a snippet to be shown in AI Overviews, and that there are no additional technical requirements. Google renders JavaScript, so a JavaScript-only page can qualify. But Google queues pages for rendering as well as crawling, and its own guidance still favours server-side rendering or prerendering. Fixing your pages for ChatGPT also makes them easier for Google to process.

For the wider picture of how assistants choose what to cite, start with generative engine optimisation for South African brands.