When someone asks an AI assistant about your company, the answer comes from one of two places. Either the assistant runs a web search right then and builds the answer from what it finds, or it answers from what the model learned in training, which can be a year or more out of date.
Two years ago search was an add-on. Today ChatGPT, Gemini and Claude all decide for themselves when to look something up, Perplexity searches on every question, and Google's AI Overviews, built on its search index, reach more than 2.5 billion people a month.
We went through the research on how often these assistants search, and for which questions. Then we started testing it in our own audits. This post covers both, and the changes we made to our method because of them.
What ChatGPT searches for
Most of the testing has been on ChatGPT, so we start there. The numbers come from small tests run by SEO and analytics firms, so treat them as rough.
- Questions about a specific company, its pricing or what it does triggered a search close to 100% of the time in one small test (RankControl, October 2026).
- Buying questions, like "best X for Y" and comparisons, triggered a search between 58% and 87% of the time across vendor tests (Graphite, cloro, Grow and Convert). A broader set of industry prompts came in at 31% (Nectiv, October 2025).
- General knowledge questions ("what is X," "how does X work") almost never triggered a search. cloro measured 0.9%.
A buyer early in their research often starts with a general question about the category. If those questions behave like other general questions, ChatGPT answers them from training data, and the brands it names are the ones the model already knew about.
What Claude, Gemini and Google search for
The other assistants have been measured less. The data that exists shows the same thing: how often they search depends on the question.
Claude. LLM Pulse asked the Claude app 400 buying questions 18 times each at the start of November. Claude's Sonnet model searched in 56.6% of answers and its Haiku model in 61.0%. By category the spread was much wider. Claude searched every time on wireless earbuds, robot vacuums and smartphones, and 1.4% of the time on skincare and language apps. Two tests by Profound landed far apart for the same reason: 36.6% on a mix of recommendation and explainer prompts, and 93% on a different set.
Gemini. cloro tracks how often each assistant's answers carry at least one live source, using its customers' prompts, which lean commercial. In October, 68.2% of the Gemini app's answers did, against 97.6% for ChatGPT and 99.9% for Perplexity. Gemini was lowest on all three of cloro's readings, and its monthly rate this year has swung between 27% and 70%.
Google's AI Overviews. An Overview is always built from search, so the number to look at is how often one appears. Trackers put that at roughly 43% to 48% of Google queries this summer (Similarweb and Semrush), up from 13% to 20% a year before. Researchers at Washington University in St. Louis found an Overview on 64.7% of trending queries phrased as questions and 9.5% of the rest. In cloro's data, an Overview appeared on 87.5% of brand queries and 97.6% of product queries.
These are vendor tests and one academic study, each with its own prompts, so the numbers can't be compared directly. What they share is that on every assistant, some questions about your category get a searched answer and some get an answer from memory. From the outside you cannot tell which one a buyer got.
ChatGPT is searching less overall
Semrush clickstream data put ChatGPT's overall search rate at about 46% of queries worldwide in late 2024 and 34.5% in February 2026. That second figure is US-only, and we have seen it only in secondhand summaries, so treat the comparison loosely.
The likely reason is speed. ChatGPT's fast default mode answers more from memory. In a June test by Semrush and Kevin Indig, the same model searched about 4.6 times as often when set to think harder, and the two settings shared only about a quarter of the sites they cited. Two buyers asking the same question can get answers built from different sites, depending on which mode they are in.
API answers and app answers do not match
Most AI visibility tracking runs questions through the assistants' developer APIs, because that is cheap and repeatable. A Surfer study from August ran 1,000 prompts both ways across five AI engines. Only 21% to 32% of the brands named in the API answers were also named in the consumer apps. ChatGPT's API named 13.8 brands per answer and the app named 7.9. Each prompt ran once each way, so part of that gap is ordinary randomness.
One run of a question is also a weak measure. Rand Fishkin and Gumshoe found under a 1 in 100 chance of an assistant returning the same list of brands twice. A brand's position in a single list changes almost every time. The share of runs it appears in is steady enough to report.
What we are seeing in our own runs
This month we started running our audit questions both ways, with search on and with search off, several times each. Here is what we have seen so far.
- On broad category questions, we have seen a brand named more often from memory than when the assistant searched. On questions about what that brand wants to be known for now, it was missing from both.
- From memory, one model kept describing a company by the single region where it started. With search on, it mostly described the company as it is today.
- Answering from memory, a model confused a chief executive with someone at another company. When it searched, it got the name right every time.
- When a major model shipped this month, we reran one set of questions the next day. On questions where the earlier model almost never named the brand from memory, the new one named it most of the time. The searched answers stayed the same.
- Asked the same question three times, the assistants agreed with themselves on most questions and split on a few. A single run would have reported those few as settled.
Our old method could not show any of this.
Why we are changing our method
Until now, our audits asked each question once per assistant, and whether search was on depended on the platform. That is a common setup, and it blends the searched answer and the memory answer into one number. Our runs show the two can differ on the same question, and that they change for different reasons.
We now measure three things:
- What buyers see. We run the questions in the consumer apps, in their default mode, and log on every run whether the assistant searched and what it cited.
- What the model remembers. We run the same questions with search turned off. Buyers get this answer on the questions that never trigger a search, and it changes only when a new model ships.
- How often, across several runs. Each question runs several times in each mode. We report how often a brand is named, with a range, in place of where it ranked in a single answer.
The gap between the searched answer and the memory answer tells you what kind of work is needed.
If a brand shows up when the assistant searches and is missing from memory, it is missing on the general questions buyers ask first. That gets fixed slowly, because memory only updates with a new model.
If the model remembers a brand well and search no longer backs that up, competitors are winning the searched answers today. That can be worked on this quarter.
If the two describe the brand differently, the model learned an older version of the company. It will keep saying so until the next release, however good this quarter's coverage is.
Search results change weekly. Memory changes with each model.
SISTRIX tracked more than 80,000 prompts every week for 17 weeks. For the typical prompt, no single source stayed in ChatGPT's searched answer for the whole period.
Most of that churn reverses. Ahrefs found that about three quarters of citations that drop out come back within 20 days.
Some of it sticks. In August, right after OpenAI swapped ChatGPT's default model and changed how it searches, Reddit's share of ChatGPT's citations fell from 3.83% to 0.52% in about a week, by Promptwatch's count. In our skincare study, sources that had led the category in the spring did not appear at all. A brand reviewing its AI presence once a quarter could have gone three months without knowing.
Memory changes on a different schedule. When Anthropic released Claude Fable 5.2 on November 9, what Claude says about a brand from memory changed that day, for every brand, whether or not anything about the brand had changed.
So we are adding a monthly program alongside the quarterly one:
- A set of each client's most important questions runs on a schedule, and again whenever a major model ships.
- Once a month you get a short read: what moved, whether it matters, and one thing to do about it. Some months that read will say nothing changed that needs you.
- When something breaks between reads, you hear about it within days, with our view of what it means.
- The full report stays quarterly. That is where we read both answers in full against your business and comms objectives.
We are not building a dashboard. Most of what moves week to week reverses on its own, so the monthly read reports the changes that stuck and what to do about them.
One related change from this week. On November 10 OpenAI opened Businesses in ChatGPT, in beta. An eligible company can claim its profile, correct its facts, and see how often ChatGPT mentions it and how many clicks follow. If you can claim yours, do. It is a useful signal between measurements. It covers one assistant and counts mentions, so we use it alongside the measures above.
If you want to see how this looks for your brand, a one-page snapshot is free. Every snapshot from here on uses the new method. Here is a sample, built on a made-up meal kit brand.
