1. Define the decision before the metric
State the commercial question, the entity being measured and the action the result should support. A retailer shortlist, a product recommendation and a brand mention are different outcomes. Keep them in separate fields rather than merging them into an unexplained score.
2. Design and disclose the scenario set
Specify market, language, buyer role, situation and essential constraints. Keep spontaneous tests separate from prompts that already name the client. Record why each scenario matters and whether it comes from observed customer questions or an expert-designed test.
A hand-built prompt set is a controlled research instrument. It does not become an estimate of market share or real-user exposure simply because the number of responses is large.
3. Preserve the measurement conditions
Retain the exact prompt and response, time, model identifier, API or consumer interface, retrieval settings, requested output format and available configuration. Document session state where relevant. Platform and model updates can break comparisons; flag them explicitly.
- ↗Repeat observations and report denominators, failures and exclusions.
- ↗Version the prompt set and the classification rules.
- ↗Keep source links and enough context for a manual audit.
4. Use interpretable measures
For a defined test set, recommendation frequency can be expressed as qualifying responses recommending the entity divided by all valid responses in that set. Report the numerator and denominator alongside the percentage. If you report rank, explain how ties, unranked lists and missing entities are handled.
| Measure | Keep separate from |
|---|---|
| Recommendation frequency | Market share or probability of a real customer seeing a brand. |
| Answer stability | Factual accuracy or quality of the recommendation. |
| Data completeness | Guaranteed inclusion in an AI answer. |
| Referral sessions | All AI influence, including journeys without a tracked click. |
5. Test explanations before claiming impact
A before-and-after change can be consistent with an intervention without proving that the intervention caused it. Use repeated baselines, comparable controls where feasible and a change log. Review classification errors and uncertain cases manually. Report observed facts, possible explanations and proposed next tests as distinct statements.
This is a practical working approach, not a claim that a single universal method can measure every AI shopping experience.
Original documentation and published evidence. The working approach and priorities on this page are editorial recommendations, not platform guarantees. Reviewed 7 November 2026.
Hyper Agency · AI valkompass · 2 September 2026↗Hyper Agency · Brand Tracker↗