Same question, different answer: how stable ChatGPT, Gemini and Perplexity really are
Achtung.app asks every AI platform every tracked question twice: the exact same wording, under identical conditions, about one second apart. It is designed as a control measurement. As a side effect, it answers a question that has rarely been backed by data: how stable are the answers of ChatGPT, Gemini and Perplexity in the first place?
The result across 3,313 answer pairs from 1 June to 18 July 2026: on average, ChatGPT names 69 percent of the same brands as in its own answer from one second earlier. Perplexity reaches 49 percent, Gemini 41 percent.
Average overlap between two answers to the identical question
Jaccard overlap of the brands named in both answers, asked about one second apart. 1 June to 18 July 2026, 3,313 answer pairs.
The extremes make it even clearer. Only ChatGPT regularly returns exactly the same brand list twice, in 37 percent of pairs. Gemini tips the other way: almost one in four answer pairs shares not a single brand. Ask Gemini the same question twice and in 24 percent of cases you get two answers that flatly contradict each other.
The extremes: identical answer vs. no overlap at all
Share of answer pairs per platform, 1 June to 18 July 2026.
In percent of answer pairs. Identical: both answers name exactly the same brands. Disjoint: the two answers share no brand at all.
- ChatGPT is the most stable of the three platforms and has recently become more stable still: since late June, overlap sits at 70 to 74 percent, and the share of identical answers rose from roughly a third to over 40 percent.
- Perplexity has been remarkably consistent for twelve weeks, but at a low level: a stable core of brands persists while the rest of the list changes almost every time.
- Gemini oscillates between 35 and 53 percent overlap with no discernible direction. Identical answers are the exception here (10 percent).
The weekly view shows this is not a fluke: the order ChatGPT ahead of Perplexity ahead of Gemini holds in practically every single measurement week since May. The chart below is updated weekly with the latest measurements.
Answer stability by week
Average overlap (Jaccard) between two answers to the identical question, asked about one second apart. Weekly means per platform, updated weekly. Latest data point: W28/2026.
| Week | ChatGPT | Gemini | Perplexity |
|---|---|---|---|
| W19 | 67.0 % | 49.6 % | 49.0 % |
| W20 | 68.0 % | 52.9 % | 45.6 % |
| W21 | 65.1 % | 51.0 % | 57.0 % |
| W22 | 64.9 % | 42.9 % | 46.6 % |
| W23 | 65.9 % | 40.2 % | 48.7 % |
| W24 | 67.6 % | 35.2 % | 49.5 % |
| W25 | 66.8 % | 37.4 % | 46.9 % |
| W26 | 68.0 % | 45.3 % | 47.1 % |
| W27 | 72.8 % | 43.9 % | 49.0 % |
| W28 | 70.4 % | 43.0 % | 48.3 % |
Important context: the measurement runs with fixed temperature and a fixed seed, so the models' randomness is switched off. The differences arise because the platforms run a live web search per query and assemble the answer from scratch every time. An AI answer is a snapshot, not a database lookup.
For brands, this means: whether a brand appears in a single AI answer is decided anew with every query. AI visibility only becomes meaningful across repeated measurements. That is exactly why Achtung.app asks every tracked question multiple times and averages over weeks rather than over single answers.