· Solveion · Perspectives · 6 min read
AI can find you a price, not a product
Every purchase we have made on an assistant’s recommendation has been mediocre, and the reason is structural. One of these jobs has a verifiable answer and the other depends entirely on trusting sources that are now being manufactured on purpose.
We have a small, unscientific, entirely consistent finding from the past year: every purchase we have made on the strength of an AI recommendation has been somewhere between disappointing and actively wrong. Not catastrophic, just mediocre in a way that a good human recommendation would not have been.
That is a small enough sample to ignore, except that thinking about why leads somewhere more interesting than the anecdote. There are two jobs hiding inside “help me buy something”, and they are not remotely the same problem.
Two jobs that look alike
The first is: I know what I want, find me the best price. This works. It works well, and it is going to keep working, because the answer has a property that matters enormously — it is verifiable, and cheaply. A price is a number on a page. If the model is wrong, checkout tells you immediately. The incentive to lie collapses at the point of sale, and the ground truth is public, structured and constantly refreshed.
The second is: I do not know what I want, tell me what is good. This is where it falls apart, and the reason is not that the models are stupid. It is that the ground truth does not exist in any form the model can reach. Whether something is good lives in the distributed, mostly unpublished experience of people who have used it for two years. What is published instead is a corpus that is now substantially manufactured.
The general principle is worth stating plainly, because it extends far beyond shopping: AI is reliable where verification is cheap and unreliable where the answer rests on trusting the source. That single sentence predicts most of where these tools succeed and fail.
The corpus is worse than “some fake reviews”
The usual framing is that reviews are unreliable, which understates it in three compounding ways.
The base rate is already grim. Independent analyses of Amazon bestsellers find roughly 43 per cent of top-selling products carry reviews flagged as inauthentic, and broader studies suggest 30 to 40 per cent of reviews on major platforms show at least one manipulation indicator. Fake reviews were estimated to cost consumers around $770 billion globally in 2025, and their number is growing about 12 per cent faster each year than genuine reviews. The FTC banned fake and AI-generated reviews in October 2024, with penalties over $51,000 per violation, and the volume has not gone down.
The manipulation is now aimed at the model, not at you. This is the part that changes the picture. Researchers at Harvard showed that appending an optimised “strategic text sequence” to a product page — generated with the same adversarial algorithms normally used to jailbreak models — substantially increases that product’s chance of being the assistant’s top recommendation. It is invisible to a human reading the page. And this is no longer theoretical: in February, Microsoft Security reported 31 companies across 14 industries manipulating AI recommendations through hidden prompts embedded in web pages. A 2026 paper extends it in the nastier direction, showing the same technique can suppress a competitor’s brand rather than promote your own.
The honest sources are leaving. Reviewers who actually test things are largely uncompensated for being read by a machine, and the rational response has been to block the crawlers or sit behind a paywall. So the pollution scales for free while the signal withdraws. Meanwhile generative engine optimisation has become a two-to-five billion dollar industry with ninety-plus platforms, reporting roughly forty per cent average visibility gains for clients. Enormous sums are being spent to produce content aimed at models. Approximately nothing is being spent to produce trustworthy content aimed at models.
That is not a temporary imbalance. It is the equilibrium you get when noise is cheap to manufacture and signal is expensive to produce, and nobody is paying for the expensive one.
Where we would push back on ourselves
Two things keep this from being a simple “do not use it”.
The counterfactual is not a perfect shopper. It is a human reading the same poisoned reviews, less systematically, with worse recall. Assistants are genuinely good at the adjacent job — what should I be looking for in this category, what are the failure modes, what questions should I ask — which is category knowledge rather than product ranking, and much harder to manipulate. Our own experience is that the advice is useful right up until it names a specific product.
And the market seems to be reaching the same conclusion. OpenAI paused Instant Checkout and shifted toward discovery and comparison; six months in, roughly thirty merchants had gone live, and a Walmart executive said conversion inside ChatGPT was about a third of their own site’s. Amazon’s Rufus, by contrast, is doing well at 300 million users and an estimated $12 billion of incremental sales — inside a catalogue Amazon controls, where prices and specifications are structured data. That split is exactly the one above.
The fix is boringly economic
Our reader’s suggestion is, we think, the correct one, and it is not utopian: the labs should pay independent reviewers.
The mechanism already exists. AI companies have signed substantial licensing deals with news organisations. Cloudflare has shipped pay-per-crawl so publishers can charge for machine access. The pipes for compensating a source of truth are built and operating — they have simply never been pointed at product reviews, because news carries legal and reputational risk in a way that headphone reviews do not.
A model provider that funded genuinely independent testing, and could say so, would have something none of its competitors could easily copy: a recommendation you have a reason to trust. In a market where every rival’s answer is quietly up for auction, that is a real moat rather than a marketing one.
What we would actually do this year
Use it for price and specification work, where being wrong is caught immediately. Use it for category education, which is what it is genuinely good at. Do the final choice yourself, from sources you chose deliberately, and be suspicious of any assistant that names a specific product with confidence.
And if you sell anything: your customers’ assistants are now a distribution channel, one that can be manipulated in both directions. Someone is already optimising for how your product is described to models. Whether that someone works for you or for a competitor is a question worth answering before it answers itself.