The published study
Hyper Agency’s study, published on 2 September 2026, reports 60,000 API calls across five models and three Swedish election compasses. Each question was repeated 100 times per model, with the available questionnaire answers constraining the response.
What the reported variation means
The report gives an average deviation of 13.2% from the most frequent answer. This describes answer variability in that experiment. It is not an error rate, a measure of factual accuracy or an estimate of variability in shopping recommendations.
Keep the domain boundary intact
The task involved political questionnaires and constrained answer choices. Model versions, wording and the testing period are part of that context. The study cannot establish what a population of shoppers sees, which product data causes a recommendation or how a consumer interface behaves today.
My practical takeaway
Build repetition into the research design instead of treating it as a late quality check. Preserve the conditions and investigate ambiguous results. In commerce, run a dedicated study around the buying task you actually need to understand; do not import a number from a different domain.
Original documentation and published evidence. The working approach and priorities on this page are editorial recommendations, not platform guarantees. Reviewed 7 November 2026.
Hyper Agency · AI valkompass · 2 September 2026↗