Content Relevance Predicts AI Citations — Not SEO Score

How I Got Here

I built a 26-check AI Search Readiness Score. It evaluates schema markup, crawlability, content structure, trust signals, and offering completeness. Then I tested it against real citation data from Perplexity API — 485 domains, 30 queries, 90 LLM runs.

The result was a flat null: r = 0.009, p = 0.849. My score doesn’t predict citations. I published that finding and moved on to the harder question: what does predict them?

In the follow-up analysis, I tested three theories. One stood out: content relevance gating. Domains were cited 62× more often for queries in their own vertical (5.17%) than for irrelevant queries (0.08%).

That was a crude binary — same-topic vs. cross-topic. This study replaces it with continuous, quantitative measures of content–query similarity. The signal held up. It got stronger.

What I Tested

I measured content relevance two ways: BM25 (lexical matching — does the page literally contain the query’s words?) and embedding cosine similarity (semantic matching — does the page talk about the same concept?). For each domain–query pair, I computed both metrics against the crawled page content, taking the maximum score across all pages per domain.

How I Measured Content Relevance

I used two methods because they capture different things. BM25 catches exact keyword overlap. Embedding similarity catches semantic overlap — same concept, different words.

The key distinction: BM25 and embeddings are query-specific. They measure how well a page matches this particular query. My AI Readiness Score is query-agnostic — it measures the same structural properties regardless of what someone is searching for.

Finding 1: Content Relevance Predicts Citations. Score Does Not.

Both relevance measures showed strong, statistically significant associations with citation probability. My readiness score showed none.

This is not subtle. Content relevance correlates with citation at r = 0.27. My score correlates at 0.001. One predicts. The other doesn’t.

When I split BM25 scores into quintiles, the gradient is monotonic:

A 12× gradient from least to most relevant. Pages with the highest keyword overlap with a query are 12 times more likely to be cited in the answer.

Finding 2: It Works Within Topics Too

A reasonable objection: maybe BM25 is just capturing the topic-matching signal I already found. SaaS sites use SaaS keywords, so they match SaaS queries better. That’s trivially true and not interesting.

The real test is whether content relevance predicts citation within the same topic. If two SaaS companies both match the topic, does the one with higher BM25 get cited more often?

Yes. Within same-topic domain–query pairs, the point-biserial correlation is r_pb = 0.37 — stronger than the full-dataset correlation of 0.271. Content relevance doesn’t just separate verticals. It differentiates within a competitive set.

This is the finding I care about most. Among domains that are all topically appropriate, the ones whose content more closely matches the query get cited more. That’s actionable.

Finding 3: My Score Adds Nothing Beyond Content Relevance

I built four regression models, each adding variables incrementally. Domain Authority (DA) was included in all models as a control, since my first study identified it as the only significant structural predictor.

Adding my score to the model (M3 → M4) improves R² by 0.001. The coefficient isn’t significant (p = 0.14). Once you know how relevant a page’s content is to the query, knowing its structural optimization score tells you nothing additional about whether it will be cited.

The ROC Curve Makes It Clear

Cross-validated ROC AUC measures how well each model distinguishes between cited and non-cited domain–query pairs. Perfect = 1.0. Random guessing = 0.5.

My score plus DA alone: AUC 0.547. Barely above coin-flip. Content relevance: AUC 0.915. Adding my score to the relevance model actually decreases AUC (0.915 → 0.913). It introduces noise, not signal.

A permutation test confirmed it: the observed R² difference between the content relevance model and a score-only model is significant at p = 0.000 (1,000 permutations). Content relevance genuinely explains citation variance. My score does not.

What This Means

1. Write for the queries you want to be cited for

AI citation is query-specific. A page about “CRM for sales teams” won’t get cited for “best project management tool” no matter how clean its schema markup is. Identify the queries your customers actually ask. Create pages that directly answer them.

2. Keywords still matter in the AI era

BM25 — a lexical matching algorithm from the 1990s — predicts AI citations almost as well as modern neural embeddings (AUC 0.844 vs 0.910). The basics still work: use the actual words and phrases your audience searches for. Semantic meaning helps, but literal keyword coverage is the foundation.

3. Structural optimization is hygiene, not strategy

Schema markup, clean HTML, proper meta tags, AI crawler access — these are table stakes. They don’t cause citations. They remove barriers. Fix them once and move on.

4. Depth wins over breadth

Within a competitive topic, the pages with higher content relevance scores get cited more. Going deep on your niche — detailed comparisons, comprehensive guides, data-backed analyses — beats spreading thin across many topics with shallow content.

Limitations

• Text coverage: 438 of 441 domains had crawled content available (99.3%). Three domains excluded due to empty crawl results.
• Max-page assumption: I used the maximum relevance score across all crawled pages per domain. This assumes Perplexity retrieves the most relevant page, which may not always hold.
• Single LLM model: All citations come from Perplexity’s sonar-reasoning-pro. Results may differ for ChatGPT, Google AI Overviews, or other models with different retrieval pipelines.
• Not pre-registered: Unlike the first study, this analysis was exploratory. The content relevance hypothesis emerged from the prior study’s theory-testing phase.
• Embedding model mismatch: I used all-MiniLM-L6-v2 (384 dimensions), a general-purpose model. Perplexity likely uses a different, proprietary embedding model for retrieval.

Methodology