Measured · The Brand Intelligence Gap ·
AI visibility scores changed with the testing setup
A controlled study of whether the way a question is set up changes how AI models score a brand. Across 864 paired observations, it did.
864 paired observations across four AI models, one brand. The two setups also differed in batching and answer format.
Can a brand's AI score be trusted?
Brands now pay for dashboards that report how often ChatGPT, Claude, Gemini and Perplexity recommend them. I wanted to know whether those scores can be trusted. In April 2026 I found at least 27 such tools, and none had published an independent check of its method. I did not test those tools directly.
I tested it the way I tested my doctorate
I designed a controlled study and ran the same questions through four AI models two ways: once with the brand named and a list of competitors to compare it against, the way a brand tracking tool might ask, and once without those additions. The study follows Design Science Research, the build and test method behind my doctorate.
The setup changed the score
Across 864 paired observations, the setup that named the brand got more mentions (99.5% against 91.4%), better rankings and lower sentiment. Same brand, different testing setup, different score. I wrote it up as a four part series and a whitepaper.
What this study cannot prove
One brand, Oura Ring, was tested. Questions repeated in clusters, some Gemini answers are missing in a non-random way, and the two setups also differed in batching and answer format. The results show the scores changed with the setup as a whole; they do not isolate the question wording.
Evidence
- Tested in a controlled study864paired observations across four AI models
- Study finding99.5% vs 91.4%mention rate with the brand named and competitors listed, against plain questions
- Market review, not tested directly27+tools identified in the April 2026 review, which found no published independent validation of their prompting methods