How to measure GEO without fooling yourself

To measure GEO (generative engine optimisation) honestly, put a fixed set of real buyer questions to several AI engines, more than once, on a regular schedule, and count how often your brand is named. Never read a percentage from fewer than 30 answers. AI answers change between engines, between runs and between days, so a single screenshot of ChatGPT naming your brand proves very little.

Most teams start the way they once checked rankings: type a question, read the answer, take a screenshot. That works for a results page, which is fairly stable from one hour to the next. It does not work for AI answers, and the reason is built into how the engines work.

In short

• AI answers are not deterministic. OpenAI and Anthropic both say the same input can produce a different output.
• Measure each engine separately. They are different models, and each decides for itself whether to search the web.
• Keep the question set fixed between runs, or you will measure your questions changing, not your visibility.
• Do not report a percentage on fewer than 30 answers. Below that, one answer moves the figure by more than three points.
• Measure on a fixed cadence, usually monthly, and before and after any change you want to judge.

Why does the same question get different AI answers?

Three kinds of variation sit on top of each other.

Between runs. Anthropic's documentation says that even with temperature set to 0 (the setting for the least randomness), "the results will not be fully deterministic and identical inputs may produce different outputs across API calls." OpenAI's guide to reproducible outputs says its APIs "are non-deterministic by default." Ask the same question twice and you can get a different list of brands.

Between engines. OpenAI's GPT models, Anthropic's Claude and Google's Gemini are different models. Each also decides whether to search the web before it answers. OpenAI's documentation says the model "can choose to search the web or not based on the content of the input prompt". Anthropic says Claude "determines when to search based on the prompt". Google says Gemini "determines if a Google Search can improve the answer." An answer written from memory and one written from a live search can name different brands.

Between days. Answers built from search depend on what the search returns that day. Google says its AI Mode and AI Overviews "may use different models and techniques, so the set of responses and links they show will vary." Our own analysis of 96 Google South Africa searches on 11 October 2026 found an AI Overview on 92 of them. That is one day, on desktop. It cannot tell you which of those 92 will show an Overview next week.

The conclusion is simple. One answer is an anecdote. A measurement is many answers, collected the same way every time.

How to measure GEO: the method

1. Fix the question set. Write down the real buyer questions you will track, using the steps in how to build a GEO question set. Keep the wording identical every run. Add new questions as a separate group so they do not distort the trend.
2. Choose the engines. Pick the assistants your buyers use and score each one separately. An average of ChatGPT and Gemini hides the fact that you may be strong on one and missing from the other.
3. Ask each question more than once. Repeats on the same engine show how stable an answer is. A brand named in three runs out of three is in a different position from one named in one run out of three.
4. Keep the raw answers. Store the full text, the brands named in order, the sources cited, the engine, the model version and the date. When a figure moves, the raw answers are how you find out why.
5. Count four things. Presence (named at all), first position (named first), share of voice (your share of all brand mentions) and cited sources. GEO metrics for your marketing dashboard explains each one.
6. Split branded from unbranded. A question that names you, such as "Is Meridian Bank good for small businesses?", will almost always mention you. (Meridian Bank is a fictional example.) The unbranded version, "Which bank is best for a small business in South Africa?", is where visibility is won or lost. Our guide to branded vs unbranded questions covers the split.

Why is 30 answers the floor?

A percentage is only as good as the number of answers under it. With 10 answers, each answer is worth 10 percentage points: one different reply turns 40% into 50%. With 30 answers, one answer moves the figure by 3.3 points. That is why no percentage should rest on fewer than 30 answers.

Thirty is a floor, not a target. The table shows how much one answer moves a figure, and the usual approximate 95% margin of error for a percentage near 50%, where uncertainty is widest. These margins are arithmetic, not measured data.

Answers behind the figure	One answer moves it by	Approximate margin of error (95%)
10	10 points	±31 points
30	3.3 points	±18 points
100	1 point	±10 points
300	0.3 points	±6 points
1,000	0.1 points	±3 points

Three consequences follow.

• Every slice needs its own 30. A figure for one engine, one product line or one province needs 30 answers of its own before you quote it.
• Comparisons need more. Comparing two months combines the uncertainty of both. With 300 answers in each month, a change near 50% has to exceed about 8 points before you can be confident it is real. A 3-point rise on 300 answers is most likely noise.
• Repeats are not independent. Answers to the same question tend to agree with each other, so 300 answers from 30 questions carry less information than 300 answers from 300 questions. Treat the margins above as the best case.

How often should you measure GEO?

Monthly suits most South African brands. It is frequent enough to catch a competitor's move and slow enough that each run has enough answers to compare. Measure more often in three situations:

• Around a change you want to judge. Run before you publish a new page or start a PR push, then again once the page is indexed. Same questions, same engines.
• During a launch or a crisis, when the questions buyers ask are changing quickly.
• After a new model version. Answers drawn from what a model learned in training change only when a new version is released. Recording the model version with every answer lets you see when a shift on one engine lines up with a release.

Daily measurement mostly tracks noise, unless each day has hundreds of answers behind it.

What mistakes make GEO measurement misleading?

• Screenshots as evidence. One answer, from one engine, on one day.
• Rewording questions between runs. "Best" and "most affordable" are different questions with different answers. Version the set when you must change it.
• Testing in your own account. Your history and settings are not your buyer's. Use clean sessions or the official APIs, and keep the setup the same each run.
• Blending engines into one number. It hides where you are missing.
• Percentages without a base. Every figure should carry the number of answers behind it.
• Counting branded questions as wins. Report them separately from unbranded ones.

Measured this way, GEO becomes a number you can manage: the same questions, the same engines, enough answers, on a schedule. For what to do once you have that number, start with our guide to generative engine optimisation.