ChatGPT's Three-Layer Retrieval Stack: How It Finds, Reads, and Cites Your Pages

Quick answer: When ChatGPT mentions your page, the version it has in hand often is not the page itself. In the default mode on a free account, Instant, it may have only the title and a roughly 200-character excerpt—about a few dozen English words [1].

That happens because ChatGPT does not gather web information in one step. It uses three layers: one to discover a page, one to retain a copy, and one to open the page live. Each layer stores a different amount of information, refreshes on a different schedule, and sees a different version of the page. The route it chooses is driven largely by cost.

This article explains those layers from the outside in: first through a familiar analogy, then through the rules governing each layer, and finally through the actions that remain useful after a specific retrieval mechanism changes.

---

A simple analogy for ChatGPT's three retrieval layers

Imagine a reporter on deadline who needs to verify a fact. The reporter has three options, ordered from cheapest to most expensive:

1. Check an index card. The card contains one title and a one-sentence summary. It takes seconds to read, but that single line is all the reporter knows about the source.
2. Open an old saved copy. The complete document is already on the reporter's computer. It contains the full text, but it may no longer be current.
3. Call the source directly. The reporter gets the newest and most complete information, but the call takes more time and costs more effort. It is reserved for stories that justify the expense.

ChatGPT's three-layer retrieval stack is the machine equivalent:

The three layers differ along three dimensions: completeness (one line versus the full page), freshness (live versus saved), and cost (near-zero marginal cost versus paid retrieval and processing time).

---

How were these retrieval layers measured?

The evidence source matters because it determines how confidently each mechanism can be described.

The main evidence comes from a July 2026 packet-capture study by French SEO consultancy RESONEO. Its browser extension recorded the raw data stream ChatGPT sent to the front end, including fields the interface did not display. The study covered more than 1,200 ChatGPT answers, about 88,000 search results, and 26,900 distinct pages. The team also published canary pages on its own domain and kept complete server logs to compare ChatGPT's internal data with what OpenAI's robots actually fetched [1]. Search Engine Journal independently reported on the same study and published additional measurements of how the stored snippets were constructed [2].

The central limitation is important: these mechanisms are reverse-engineering observations. OpenAI has not published them as product specifications. OpenAI publicly refers only to “third-party search providers,” and its crawler documentation does not describe the internal index architecture [2][6]. The internal source field used by the researchers disappeared from ChatGPT's data stream around July 21. After its removal, pipeline identification relied on formatting signatures [1].

Independent observations support the broad direction. In April 2026, Chris Green tested 1,000 prompts up to 10 times each and captured 9,946 completed search runs. Suganthan Mohanadasan separately examined ChatGPT network traffic. Both found the same set of internal source labels, and OpenAI's in-house index accounted for 88.1% of the primary search sources in Green's dataset [4]. Oncrawl's Jérôme Salomon had independently demonstrated the existence of a cache layer through an API parameter in December 2025 [3].

Multiple observers therefore agree on the architecture's broad direction. Their exact percentages should still be read within the boundaries of each sample.

---

Layer one: A discovery index with a title and 200 characters

The discovery layer appears to be OpenAI's own index. RESONEO tested whether Bing or Google supplied it three ways: only 1.5% of URLs from this layer appeared in Bing's top 20 for the same fan-out queries; none of its snippets matched Bing's snippets; and Bing capped titles at 75 characters while 24% of titles in the observed OpenAI index exceeded that length. The longest captured title was 289 characters [1].

This layer gave the model only three items: the URL, the complete untruncated title, and a snippet of about 200 characters.

Those 200 characters are not your meta description

This is the easiest part of the system to misread. A marketer might assume the model receives the page's carefully written meta description. In the observed discovery layer, the snippet came from the beginning of the visible page content instead, and the meta description did not participate [2].

The snippet was anchored on the H1 and visible text immediately surrounding it. Among 463 pages with an H1, 387 snippets included that heading, or 83.6%. The median H1 was 51 characters long, leaving roughly 150 characters for the text that followed [2].

Template elements frequently consumed that remaining space:

One in seven sampled pages had no H1 markup at all. In those cases, the snippet began with whichever subheading the template provided [2]. RESONEO also recorded an extreme case where the snippet contained 100% table-of-contents text and 0% article content [1].

The snippet does not change with the user's question

The same URL returned the same frozen snippet whether the user asked about pricing, history, or side effects. The excerpt was cut at indexing time and remained in service until the next crawl. The researchers considered this a likely temporary mechanism that could improve quickly [1].

By modern search standards, that is a crude design. Query-independent snippets are a pattern traditional web search largely moved beyond decades ago. Until the implementation changes, however, that frozen excerpt is part of the evidence on which a page is judged.

Can smaller sites enter the discovery index?

Yes, based on the observed sample. This matters because, as later sections show, OpenAI's own index supplies most of the retrieval set in free Think mode. If the index were restricted to licensed publishers, other sites would have little opportunity to surface there.

The study compared licensed and non-licensed publishers within the same layer. A content agreement with OpenAI did not change the format, length, or freshness with which a stored page was served. Hundreds of publishers without agreements appeared in the same format as licensed partners [2].

There is one boundary: licensed articles can reach OpenAI through a feed that bypasses crawling. A licensing agreement may therefore change how content arrives, even though the study did not find a difference in how content was stored and served after arrival [2].

---

Layer two: A reading cache that may hold a three-month-old copy

ChatGPT appears to retain a complete copy of pages it has fetched, convert the HTML to Markdown, index the result by URL, and share that copy across users and subscription tiers [1]. This cache layer had already been independently demonstrated in December 2025, when Oncrawl's Jérôme Salomon identified a Web Search API parameter that exposed the existence of an internal cached index [3].

Cross-user sharing has a practical consequence. If a paid user caused ChatGPT to fetch one of your pages last week, a free user asking about the same page today can receive last week's cached copy. Your server may record no new request.

Cache freshness depends on how often users ask about the page

The observed cache followed a stale-while-revalidate pattern. A stored copy was treated as fresh for about 30 minutes. During that window, users received the cached version. After 30 minutes, a user still received the stale copy immediately while a background request refreshed the page for a later user [1].

That makes demand the controlling refresh signal: how often ChatGPT users ask about the page. Popular pages remain comparatively fresh while rarely requested pages can age indefinitely. Salomon documented copies still being served more than 90 days after they were fetched and found no visible eviction ceiling [1].

Familiar cache and indexing directives did not control this layer

The observed cache ignored Cache-Control: no-store and noindex [1].

OpenAI's official crawler documentation helps explain part of that behavior. It defines two different agents: OAI-SearchBot surfaces sites in ChatGPT search results and is the agent site owners should manage through robots.txt; ChatGPT-User visits pages in response to user actions, and OpenAI states that robots.txt rules “may not apply” because those actions are initiated by a user [6].

Part of the cache can therefore be populated through a route that OpenAI does not treat as an ordinary automatic crawl. The distinction follows the product's division of crawler roles.

Some page elements disappear during HTML-to-Markdown conversion

The conversion process made its own choices, including one with a direct technical SEO consequence:

• Scripts, iframes, and JSON-LD were removed, so structured data did not reach the model through this path [1]
• Image alt text survived
• Text hidden with CSS was still extracted, allowing the robot to read content human visitors could not see [1]

The last behavior suggests that human-visible and machine-visible versions of a page were not easy for this path to distinguish at the time of the study. The researchers described OpenAI's cloaking detection as underdeveloped. The finding describes the parser's measured behavior and offers no support for serving different claims to bots.

Site owners can test this cache layer

Salomon found an external_web_access parameter in OpenAI's Web Search API. With the parameter set to false, the model can answer only from previously cached material. With it set to true, the system can fetch current web content. The documented default in the cited test was true [3].

That exposes a practical check. Ask the model to summarize one of your URLs with external_web_access: false. A successful summary suggests the page exists in the cache; an access failure suggests it does not. Repeat the request with true, then inspect server logs for a ChatGPT-User request to determine whether a live fetch occurred [3].

A separate API field reportedly returned the fetch date for a stored copy. For a site owner's own URL, that date can reveal when a ChatGPT user most recently caused the page to be read. The researchers characterized it as a low-cost exposure signal that few teams were monitoring [1].

---

Layer three: The few pages opened live were cited at a much higher rate

The third layer is a live page open: ChatGPT fetches and reads the page during the current conversation. In the observed sample, this layer was rare and disproportionately valuable.

Across the corpus, 61,332 URLs entered the source sidebar, 5,032 became the lead source behind a citation, and only 759 pages were actually opened. Every observed live open occurred in Thinking mode [1].

That is a roughly tenfold difference. Entering the live-open set was therefore the highest-value position in the measured retrieval chain.

The free-user pattern moved in the other direction: 93% of observed Instant answers opened no page at all [1]. A Thinking conversation retrieved roughly 100 results across about 28 domains, compared with approximately 11 results for an Instant conversation [1].

Two hard technical limits

A page had to clear two non-negotiable conditions to be opened and read successfully.

OpenAI's crawlers did not execute JavaScript. Client-rendered content was therefore absent from the version they could read. Independent large-scale evidence supports this result: Vercel and MERJ observed that major AI crawlers did not render JavaScript, explicitly including OpenAI's OAI-SearchBot, ChatGPT-User, and GPTBot. The crawlers downloaded JavaScript files—11.50% of ChatGPT crawler requests in that dataset—but did not execute them [10].

Mohanadasan recorded a corresponding source-selection behavior. ChatGPT appeared to prefer official pricing pages, then fell back to third-party sources when the official price was hidden behind JavaScript or otherwise difficult to parse [4].

The observed page-size limit was exactly 4 MB. A page exceeding that limit was not truncated. The entire fetch was rejected with HTTP 400, leaving the model with no page content [1]. In this layer, an oversized page effectively did not exist.

---

Why three layers exist: Retrieval routing is an economic decision

The preceding behavior becomes easier to understand when viewed through cost.

Why would OpenAI maintain both its own index and purchased Google-scraped results? Cost.

Instant mode has to answer within seconds and usually serves a non-paying user. It therefore queries information OpenAI already controls: its in-house index. The answer can be built from titles and 200-character snippets without opening a page. In Thinking mode, the user pays and accepts a longer wait, so more expensive tools can run: purchased Google-scraped rankings and live page opens performed by ChatGPT-User [1].

Actions that cost OpenAI money, such as paying retrieval providers, or time, such as opening URLs and calling live APIs, are reserved for the modes where users pay and wait.

The impact depends on the size of the free tier. OpenAI reports more than 900 million weekly active users, more than 50 million consumer subscribers, and more than 9 million paying business users; the cited page does not display a publication date [7]. Because all three numbers are lower bounds, an exact share cannot be calculated. The combined paying groups—roughly 59 million using the stated floors—still represent a small part of a 900-million-plus weekly audience.

That means the title-and-snippet route can shape a large majority of ChatGPT interactions.

---

The same question can reach two different webs

A second routing split sits above the three layers and explains why ChatGPT visibility is difficult to measure.

On a free account, selecting Think more than doubled the retrieval pool in the observed comparison: URLs rose from 15.1 to 35.3 per conversation, while distinct domains increased from 9.8 to 16.3. That volume was close to paid Thinking at medium effort [1].

The two modes did not search the same web. In free Think, 74.7% of results came from OpenAI's in-house index and only 3.1% came from purchased Google web results. Paid Thinking was nearly the mirror image: 75.3% came from Google-scraped results and 24.7% from the in-house index [1][2].

For brands, the implication is concrete. Google ranking remains important for paid Thinking, but it does not guarantee entry into free Think's retrieval set, where OpenAI's own index is the primary gatekeeper.

Two measurement traps follow.

Do not use the API to predict the ChatGPT product. Researchers ran the same prompts across four ChatGPT product regimes and four API models, then compared the brands mentioned using Jaccard similarity. API variants overlapped with one another at 0.32–0.39 and ChatGPT regimes at 0.25–0.37, while product-to-API similarity fell to 0.23–0.27 [1]. The API can help probe what a model family knows about a brand. It is a poor predictor of what the ChatGPT product will answer.

Repeated runs can switch the source pipeline. In Green's repeated-prompt test, 11.6% of prompts changed their primary search source across runs. When that happened, URL overlap fell from 0.273 to 0.149 and domain overlap from 0.265 to 0.155—declines of roughly 45% and 42% [4].

---

Three counterintuitive findings from the retrieval data

Retrieval and citation are separate markets

The system pulled arXiv more than 2,600 times in this corpus and ultimately cited it only 10 times. Reddit followed a similar pattern [1]. ChatGPT appeared to use preprints and forums during reasoning, then present readers with ordinary web pages.

Mohanadasan's independent observation supplied one possible mechanism. ChatGPT frequently fetched both Reddit and YouTube, but cited Reddit and not YouTube in the sampled commercial queries. The difference was text availability: Reddit exposed usable text, while YouTube search results often supplied metadata without a complete transcript [4].

URLs with no snippet were cited more often

In Instant mode, after Reddit, YouTube, and arXiv were excluded, URLs without a snippet were cited 14.9% of the time, compared with 8.2% for URLs with a snippet [1]. The model cited pages it knew less about at a higher observed rate. The study recorded the pattern but did not explain it.

Some cited URLs did not match any search result

Some cited links matched no recorded search result at all. The researchers hypothesized that the model produced them from parametric memory—information encoded during training. These links were usually bare roots of well-known domains and still carried utm_source=chatgpt.com, making them look as though they came from a search [1].

The hypothesis deserves attention because a related phenomenon has been measured independently in academic work. A 2026 conference paper audited citation behavior in Google AI Overviews for YMYL queries and found that the citation difference it studied was driven primarily by non-retrieved citations: cited documents did not all appear in the observed retrieval set [8].

The system boundary matters. That paper studied Google AI Overviews. It cannot prove the ChatGPT hypothesis, but it shows that citations without a corresponding observed retrieval result are not unique to one reverse-engineering study.

UTM reporting misses pages ChatGPT read

Analytics teams may already filter for utm_source=chatgpt.com. That parameter tracks outbound links clicked by users. Yet in the observed Thinking-mode behavior, citations for pages ChatGPT opened itself did not carry that parameter [1].

The pages with the 74% citation rate—the set ChatGPT actually read—therefore left no UTM trace for the read itself. A UTM report counts clicks and misses machine reads. To observe reading activity, inspect server logs for the ChatGPT-User user agent, which OpenAI's documentation identifies as the agent used for certain user-triggered page visits [6].

---

What should site owners do about the three layers?

Prioritize actions by how likely they are to survive the next retrieval update.

Make foundational changes that help across all three layers

These actions improve the page for human readers and multiple retrieval systems. Their value does not depend on one temporary implementation detail.

Change how ChatGPT visibility is measured

• Track ChatGPT-User in server logs alongside UTM traffic. UTM parameters record clicks; logs can reveal machine reads [6].
• Use external_web_access: false to spot-check whether priority pages exist in the cache [3].
• Report free Instant, free Think, and paid Thinking separately. A combined “ChatGPT visibility” score hides the differences between them.
• Repeat the same prompt set. A single run is not sufficient to establish a trend [4].

Do not rebuild a site around one retrieval quirk

The researchers emphasized how quickly the system changes. An internal field disappeared overnight, shopping providers were anonymized during the same week, and Google Shopping tokens moved from readable to encrypted within four days. A specific mechanism can vanish before a site redesign built around it is deployed [1].

The site: operator provides a documented example. ChatGPT uses this operator to restrict a search to one domain, and its use moved sharply across several model versions.

The denominators differ. Writesonic compared model versions and effort tiers, Promptwatch measured all tracked fan-out queries, and RESONEO separated product modes. Their percentages cannot be subtracted from one another. The denominator difference also helps explain why measurements taken around the same period can be far apart: free Instant used site: in only 3.5% of observed queries while paid high effort used it in more than half, so any platform-wide average depends on the mix of product modes [1].

The comparable signal is directional: this mechanism was sharply reduced and then sharply restored. Promptwatch also found that average searches per answer rose from 1.08 to 1.83 on August 8, suggesting that domain-scoped searches were added on top of broader searches while the broader searches continued [5].

The evidence grade should remain visible. Writesonic tested 50 prompts from one account, with one run per configuration. The authors described the study as directional and warned that any single-run result should not be read as a decimal-level measurement [9]. The trend is useful. Treat the exact values as directional observations and avoid using them as a universal benchmark.

A mechanism that moves this far across a few months is a weak foundation for an expensive redesign.

Two investments are safer. First, models have knowledge cutoffs, so they need retrieval to bridge the gap between training data and current information. Retrieval will remain a dependency for AI assistants. Second, rudimentary indexes produce rudimentary answers that users notice, creating pressure for every engine to improve. The 200-character snippet may disappear; the need to publish the clearest and best-supported answer will remain [1].

---

Where Innflows fits

The operational problem created by the three-layer stack is that the same brand faces different source corpora in different modes. A screenshot from one answer does not reveal that variation.

Innflows supports periodic monitoring built around simulated user questions across platforms including ChatGPT, Gemini, Google AI Overviews, Google AI Mode, DeepSeek, Qwen, and Grok. Its monitoring, source-credibility, and website-structure layers can be used to establish per-platform baselines, then check whether pages carrying important claims remain crawlable and consistent.

The boundary matters. GEO can improve the probability that useful, accessible, well-supported content enters retrieval and citation. It cannot guarantee that a specific mode will cite a particular page during a fixed period, and it cannot determine whether OpenAI will change one of these retrieval layers next month.

---

Where does the evidence stop?

The mechanism details come from reverse engineering. OpenAI has not published the composition of its internal index, cache policy, or page-size limit. The internal field used by the researchers was removed, and subsequent pipeline identification relied on formatting signatures [1].

The primary evidence comes from one organization's packet-capture study. RESONEO sells SEO consulting services and distributes the browser extension used to collect the data at no charge [2]. The three-layer model, cache behavior, 4 MB limit, and open-versus-retrieved citation rates still depend mainly on this study. Pipeline composition and the cache's existence have independent support; many implementation details do not.

Percentages vary by sample. OpenAI's in-house index accounted for 88.1% of Green's primary search sources, while purchased pipelines carried more weight in Mohanadasan's sample, particularly for commercial, shopping, finance, weather, and local queries [4]. Account type, country, and query category can all change the result.

Changing the snippet has not been shown to increase citations. The studies measured how pages were stored and served. They did not test whether changing the text after an H1 increased the probability of citation [2].

The parametric-memory explanation remains a hypothesis. The researchers explicitly marked it as unresolved and invited challenges [1]. Non-retrieved citations observed in Google AI Overviews are an independent finding in another system and cannot serve as proof about ChatGPT [8].

The conclusions have a timestamp. Most observations were made between July and mid-August 2026. The August 8 retrieval shift demonstrates that this kind of implementation detail can have a shelf life measured in weeks.

---

FAQ

Does a ChatGPT citation mean ChatGPT read my page?

Not necessarily. The study separated retrieval, opening, and citation. Across the corpus, 61,332 URLs entered the source sidebar, while only 759 were opened, all in Thinking mode. Opened pages were ultimately cited 74% of the time, compared with 7% for pages that were retrieved but never opened [1]. A citation can therefore be based on little more than a title and a 200-character snippet.

Do free and paid ChatGPT users search the same web?

No. In the observed comparison, 74.7% of free Think results came from OpenAI's in-house index and 3.1% from purchased Google web results. Paid Thinking drew 75.3% from Google-scraped results and 24.7% from the in-house index [1]. Two users can ask the same question, retrieve a similar number of sources, and still receive answers assembled from substantially different corpora.

Does my meta description help ChatGPT?

It depends on the layer. OpenAI's observed in-house index did not use the meta description; it used the H1 and visible text immediately following it. The purchased Google pipeline behaved more like Google and used the meta description in roughly one out of three observed results [2]. Keep writing a useful meta description, but do not expect it to control the 200-character excerpt in OpenAI's own index.

Will noindex or no-store prevent ChatGPT from caching my page?

The observed cache ignored both directives [1]. OpenAI's official documentation partly explains the distinction: ChatGPT-User performs user-triggered visits, so robots.txt rules “may not apply,” while OAI-SearchBot controls whether content can surface in ChatGPT search answers [6]. To manage search visibility, configure OAI-SearchBot in robots.txt.

How can I tell whether my page is in ChatGPT's cache?

Use OpenAI's Web Search API with external_web_access set to false and ask the model to summarize the URL. A successful summary suggests the page is cached; an access failure suggests it is not. Repeat the request with the parameter set to true, then inspect server logs for ChatGPT-User to determine whether a live fetch occurred [3].

Does my structured data reach the model?

Not through the observed cache-conversion path. The HTML-to-Markdown process removed scripts, iframes, and JSON-LD [1]. Schema still serves traditional search and other systems, but a critical fact that exists only in JSON-LD will not reach the model through this particular path.

---

Bottom line

ChatGPT does not follow a single “read the page, then cite it” process. It uses three layers: a discovery index that may provide only a title and about 200 characters, a full-page cached copy that can be months old, and a live page open observed only in paid Thinking mode. Each layer sees a different amount of your page at a different level of freshness.

Cost largely determines which route runs. Free Instant must answer within seconds using information OpenAI already controls, and 93% of observed Instant answers opened no page [1]. OpenAI's reported floors of more than 50 million consumer subscribers and more than 9 million paying business users still represent a small share of its 900-million-plus weekly audience [7].

The implementation details can change quickly; one retrieval behavior changed again on August 8 [5]. The durable investments are the ones that work in every layer: a self-contained title, a first sentence after the H1 that states the core answer, key facts available without JavaScript, and a page below the observed 4 MB limit.

You cannot control which layer ChatGPT uses to read your page. You can make sure every layer receives the same clear answer.

---

References

[1] - Inside ChatGPT's retrieval stack: The index, cache, and pages it actually reads — Search Engine Land (RESONEO study), August 2026

[2] - ChatGPT's Search Index Serves Small Sites Too, Data Shows — Search Engine Journal, August 2026

[3] - OpenAI is quietly building a hidden cached index for ChatGPT Search — LLMrefs (based on Jérôme Salomon's finding), December 2025

[4] - ChatGPT citations change when hidden search pipelines switch — Search Engine Land (independent tests by Chris Green and Suganthan Mohanadasan), April 2026

[5] - ChatGPT Search Now Uses the site: Operator at Scale — Promptwatch, August 2026

[6] - Overview of OpenAI Crawlers — OpenAI documentation

[7] - Scaling AI for everyone — OpenAI (page does not display a publication date)

[8] - Auditing Citation Behavior in AI-Generated Search Summaries: A Framework and a Case Study of Google AI Overviews — PMLR Volume 318, 39th Canadian Conference on Artificial Intelligence, 2026

[9] - GPT-5.6 Sol Runs site: Queries on 71% of Its Searches. GPT-5.5 Did 11%. — Writesonic, 50 prompts from one account with one run per configuration, data collected April–July 2026

[10] - The rise of the AI crawler — Vercel and MERJ analysis of the Vercel network, December 2024