How LLMs Rank Products: A 91% Manipulation Success Rate and What It Means for Your Business

Researchers recently tested whether product descriptions can be optimized to manipulate how LLMs rank products in their recommendations. The answer: yes, with a 91.4% success rate for Top-5 placement. The CORE paper (Controlling Output Rankings in Generative Engines) tested this across four major LLMs — GPT-4o, Gemini-2.5, Claude-4, and Grok-3 — using 3,000 products in 15 Amazon categories. The implications for any business that wants to appear in AI-powered product recommendations are significant.

This article breaks down what the researchers found, what it means for your product pages, and where the ethical line sits between optimization and manipulation.

How LLMs Actually Rank Products

When you ask ChatGPT or Perplexity “What is the best noise-canceling headphone under $300?”, the LLM does not have an internal product database. Instead, it queries external search engines, retrieves a set of product pages, and then re-ranks those results based on the content it reads. The final recommendation list you see is the LLM’s interpretation of those retrieved pages.

The CORE paper revealed a critical finding: the initial order of results from external search engines heavily influences the LLM’s final output ranking. Products that appear higher in traditional search results start with a significant advantage. This means classical SEO is not dead — it is the foundation on which AI recommendations are built.

But the content on the product page itself also matters. The researchers demonstrated that by appending strategically designed content to product descriptions, they could override the retrieval order advantage and push lower-ranked products to the top.

Three Content Strategies That Influence LLM Rankings

The researchers tested three types of optimization content, all designed to sound natural and avoid detection:

1. String-Based Optimization

Adding specific text patterns to product descriptions that align with how LLMs evaluate relevance. These are keyword and phrase patterns that signal topical authority. Think of it as the AI equivalent of keyword optimization — but targeting LLM attention patterns rather than search engine crawlers.

2. Reasoning-Based Optimization

Adding logical arguments that explain why a product is the best choice. Statements like “This headphone outperforms competitors in its price range because of its 40-hour battery life, adaptive noise cancellation, and lossless audio codec support” give the LLM explicit reasoning to justify ranking the product higher. LLMs are trained on reasoning chains — content that pre-packages a rationale is easier for them to cite and recommend.

3. Review-Based Optimization

Incorporating customer reviews and social proof directly into the product description context. When the LLM sees “4.8 stars from 2,300 verified buyers” and specific review quotes, it treats this as evidence of product quality. The review-based approach was particularly effective because LLMs weight social proof as a trust signal when generating recommendations.

The Numbers: How Effective Is This?

These results were consistent across all four LLMs tested and across 15 product categories (from electronics to kitchen appliances to outdoor gear). The benchmark dataset, ProductBench, included 200 products per category with their top-10 recommendations collected from Amazon’s search interface.

The key insight: content quality and structure directly influence AI output rankings. This is not theoretical — it is measured, reproducible, and significant.

Why Classical Search Rankings Still Matter

One of the paper’s most important findings is often overlooked: the LLM’s final ranking is heavily seeded by the initial retrieval order from external search. If your product page ranks on page 3 of Google, it starts at a disadvantage in the LLM’s candidate set — even if your product description is excellent.

This means AI search optimization is not a replacement for traditional SEO. It is an additional layer. The retrieval stage (getting into the candidate set) still depends on classical signals: rankings, backlinks, domain authority. The generation stage (getting recommended) depends on content quality signals: reasoning, reviews, clarity.

As we found in our own research on AI search mechanisms: authority gets you into the candidate set; clarity gets you cited.

The Ethical Line: Optimization vs. Manipulation

The CORE paper demonstrates that LLM rankings can be manipulated. This raises an important question: should they be?

There is a clear distinction between making your content genuinely better and gaming the system with misleading content. Here is where we draw the line:

• Legitimate: Adding genuine reasoning about why your product solves a specific problem. Including authentic customer reviews. Clearly stating comparative advantages that are factually accurate.
• Questionable: Appending hidden or low-visibility text designed specifically to influence LLM scoring. Fabricating reviews or social proof. Making comparative claims without evidence.

Our position: do not manipulate. Be genuinely better. But understand what signals matter so you can improve your content along those dimensions. The three strategies from the paper — especially reasoning-based and review-based content — are powerful when applied honestly.

What to Check on Your Product Pages

Based on the CORE paper findings and our own research, here are the signals that matter most for AI product recommendations:

1. Reasoning-based content. Does your product page explain why a customer should choose this product? Not just features — but comparative advantages, use cases, and problem-solution framing.
2. Review quality and quantity. Are customer reviews present on the page with AggregateRating schema? Do you have 5+ reviews with specific details? Generic “great product!” reviews carry less weight than detailed experience reports.
3. Comparative advantages. Are your differentiators explicitly stated? LLMs need clear statements like “unlike X, our product offers Y” to generate reasoning for recommendations.
4. Classical SEO baseline. Is your product page ranking in the first 2-3 pages of traditional search? If not, you may not even enter the LLM’s candidate set.