You Can Finally Measure Content Alignment. That’s The Dangerous Part via @sejournal, @DuaneForrester

We have always been approximating relevance. Every keyword list, every TF-IDF score, every editorial judgment about whether a page “covers the topic” has been an attempt to answer a single question: is this content about the thing the user is looking for? The tools changed. The question did not. What changed, meaningfully, is the resolution of the instrument. Keyword research approximated relevance through lexical overlap: If the words match, the topics probably align. Vector-based semantic analysis approximates it through meaning overlap: If the concepts are close in embedding space, the content is probably relevant regardless of whether the exact terms appear. That is a genuine, material upgrade, but it is not a move from guessing to knowing.

The reason that distinction matters is that a significant portion of the SEO and content strategy community is right now treating it as if it were. They are looking at alignment scores, cosine similarity outputs, and semantic proximity metrics and reading them as ground truth. A high score means aligned. A low score means not aligned. Optimize until the number goes up. And the number, because it is a number, feels like it has settled the question that keyword research always left open. It hasn’t. It has given you a higher-resolution version of the same approximation, and the higher resolution is exactly what makes it dangerous, because it removes the humility that low resolution used to enforce.

Precision Is Not Accuracy

Gerard Salton’s SMART system at Cornell introduced the vector space model for document retrieval in the 1960s. The core insight then was the same insight powering today’s embedding models: represent both the query and the document as vectors, measure the angle between them, and use that angle as a proxy for relevance. What has changed across 60 years is the sophistication of how those vectors are constructed. Salton used term frequency. Modern embedding models use transformer-derived representations that encode semantic relationships, contextual meaning, and conceptual proximity across hundreds or thousands of dimensions. The measurement got dramatically better. But the thing being measured, the angular distance between two vector representations, is still a proxy for a relationship that exists outside the math.

This is where the Netflix research team landed in their 2024 study on cosine similarity in embedding models. Steck, Ekanadham, and Kallus demonstrated that cosine similarity applied to learned embeddings can produce results that are, in their framing, arbitrary. The way an embedding model is trained, the regularization applied, the data it saw, all shape the geometry of the space in ways that make a raw cosine score unreliable as an absolute measure of semantic similarity. A high score in one embedding space is not equivalent to a high score in another. The score is real. The similarity it claims to represent may not be.

For practitioners optimizing content, the implication is direct. When you score your content’s alignment to a query using an embedding model, you are measuring semantic proximity inside that specific model’s representation of language. You are not measuring how Google’s retrieval infrastructure or OpenAI’s RAG pipeline or Perplexity’s index would evaluate the same relationship. Those systems use their own embedding models, their own retrieval architectures, and their own reranking layers. A score of 0.92 in your measurement space might correspond to strong retrieval in one system, weak retrieval in another, and irrelevance in a third.

What Kind Of Wrong Are You?

This is the axis that matters, and it is not the one most practitioners are thinking about. The question is not whether keyword research or vector alignment is the better method. The question is what kind of error each method produces, because the error type determines whether you can correct for it.

Keyword research, for all its limitations, produces a known unknown. You know you are approximating. You know that matching terms to a page does not guarantee topical coverage, does not guarantee user satisfaction, and does not guarantee that a search engine will judge the page as relevant. The imprecision is visible, and because it is visible, it keeps you honest. Practitioners who grew up in keyword-driven optimization learned to over-cover, to build supporting content, to triangulate intent from multiple angles, precisely because they understood the instrument was blunt. The bluntness was a feature. It forced humility.

Vector alignment scoring, by contrast, can produce an unknown unknown. The number is precise. It has decimal places. It can be tracked over time, graphed, compared across content assets, and optimized against. And that precision creates a psychological trap: it feels like the question has been answered. The content is 0.89 aligned to the query. That must mean something definitive. But what it actually means is that in one specific embedding space, using one specific model’s learned representation, the angular distance between two vectors falls within a certain range. The score says nothing about whether the production retrieval system that will actually serve your content uses a compatible embedding space, applies the same tokenization, or weights semantic similarity the same way during reranking.

The MTEB benchmark leaderboard illustrates this concretely. The performance spread across current embedding models is not small. A content asset that scores well against one model’s embedding space may score materially differently against another, not because the content changed but because the geometry of the space changed. And the embedding model your scoring tool uses is almost certainly not the one any given AI platform uses in production. There is no public registry of which model powers which system’s retrieval layer. You are measuring in a space that is representative of the general problem but not identical to the specific system where your content will be evaluated.

That is not an argument against measuring. It is an argument against reading the measurement as settled fact. The distinction between a directional signal and a definitive answer is the entire discipline.

The Instrument Got Better. The Old One Is Not Enough

None of this rescues keyword-only optimization as a sufficient strategy. It is not sufficient, and the reasons are structural, not sentimental.

LLMs and AI retrieval systems operate in semantic space, not lexical space. They process meaning, not strings. A page can score perfectly against a keyword target list while being semantically adrift from the actual intent the query represents, because keyword presence and semantic coverage are different things. Conversely, a page can use none of the target keywords and still be strongly aligned semantically, because it covers the same conceptual territory through different vocabulary. The paraphrase and synonym space that LLMs operate in is structurally invisible to a keyword-based evaluation. You cannot see what you cannot measure, and keyword tools cannot measure semantic proximity.

Consider a practical case. Keyword research correctly identifies “customer churn prevention strategies” as a high-value target. The content team builds a thorough, intent-appropriate piece around it. It covers the topic, uses the target terms naturally, and would pass any keyword audit without issue. But an alignment score reveals that the content’s semantic center of gravity sits closer to “measuring churn” than to “preventing churn,” because the piece leans heavy on diagnostic framing, identifying at-risk accounts, calculating churn rates, segmenting by behavior, and lighter on intervention framing, what to actually do once you have identified the problem. Both treatments are on-topic. Both satisfy the keyword target. But the semantic distance between the content and the query as a retrieval system represents it is larger than the keyword coverage suggests, and keyword research has no instrument to surface that drift. The alignment score does. Not because the keyword research failed, but because it was never built to see at that resolution.

This is not a criticism of people who focus on keyword research. Those practitioners are not wrong. They are working at the resolution the available instruments allow. Intuiting alignment between content and query intent is a real skill, and the best keyword strategists are doing something genuinely sophisticated: they are approximating semantic relevance through lexical indicators, using editorial judgment to bridge the gap the tools could not cross. The tools can now cross a version of that gap. The editorial judgment still matters, but the gap it has to bridge is different.

The danger is the practitioner who decides that because keyword research is no longer sufficient, vector alignment scoring is the complete replacement. That practitioner has traded one approximation for a better one while losing the awareness that it is still an approximation. They have upgraded the instrument and downgraded the literacy, which is a net loss.

The Discipline Is Knowing What The Number Is Not Telling You

Goodhart’s Law, the observation that when a measure becomes a target, it ceases to be a good measure, is not just an aphorism for economists. It is the exact failure waiting for any team that treats an alignment score as a target to optimize against rather than a signal to interpret. The moment the score becomes the goal, the content starts drifting toward the score’s geometry and away from the actual relevance it was supposed to approximate. You start writing for the embedding model instead of the reader and the retrieval system, and the embedding model you are writing for is not the one any production system uses.

The real discipline, the one that did not exist when practitioners were navigating by keyword intuition alone, is understanding what an alignment measurement is and is not telling you. It is telling you that in a given embedding space, your content’s vector representation is geometrically close to a query’s vector representation. That is useful. That is more information than keyword presence gives you. It is telling you something about semantic coverage that lexical analysis cannot. But it is not telling you whether the production system’s embedding space has the same geometry. It is not telling you how reranking will treat the result. It is not telling you whether the LLM’s generation layer will interpret your content as authoritative, complete, or worth citing. Alignment is a retrieval-adjacent signal. It says nothing about interpretation.

The practitioner who can hold those two realities, the signal is real and the signal is incomplete, is the one operating with genuine literacy about the systems they are trying to influence. The one who collapses them, who reads a high alignment score as confirmation that the content is “optimized,” is operating with a more sophisticated version of the same overconfidence that made people think a keyword density of 3% meant their page was relevant. The number got better. The mistake is the same.

Representative, Not Identical

The honest framing is not “right space versus wrong space.” That binary invites paralysis: If no measurement space is the production space, why measure at all? The best framing, in my opinion, is a spectrum of representativeness. Some measurement spaces are closer to what production systems use than others. Some embedding models share more architectural DNA with the models powering major AI platforms than others. Some scoring methodologies account for the gap between measurement and production better than others. The question is not whether your measurement is perfect. It never will be. The question is how representative your measurement space is of the systems you actually care about, and whether you are treating the score with appropriate directional respect rather than absolute faith.

This is the actual work. Not chasing a number. Not abandoning measurement because it is imperfect. Building enough literacy about how these systems work to know which signals to take seriously, which to discount, and which to combine with other indicators before making a content decision. That literacy was optional when the only instrument was keyword research, because the instrument was so obviously blunt that nobody mistook it for truth. It is not optional now. The instruments are precise enough to fool you, and the cost of being fooled is optimizing content for a geometry that does not represent the system where your brand needs to be visible.

I wrote about a related dimension of this problem in the vector index hygiene piece last year, focusing on how the quality and maintenance of the index itself shape retrieval outcomes. This article is the other side of that coin: not the index, but the measurement you use to evaluate whether your content belongs in it. And both connect to a larger question I will return to in future work, which is a gap most people aren’t talking about yet.

Start With What You Can See

If you are still running keyword research as your primary content alignment method, you are working with a blunt instrument in an environment that now demands more resolution. If you are running vector alignment scoring and reading the output as settled truth, you have the resolution but not the literacy to use it safely. Both are correctable. The path forward is not choosing one over the other. It is layering them, understanding what each can and cannot tell you, and building the organizational capacity to treat precise measurements as what they are: directional signals produced inside a specific space that may or may not represent the systems where your content competes.

The gut feeling was never the enemy. The illusion that you have moved past the need for judgment is.

For a broader look at how AI search visibility is reshaping the work of being found, “The Machine Layer” covers the structural shifts that make this kind of measurement literacy essential.

More Resources:

This post was originally published on Duane Forrester Decodes.

Featured Image: Luke Jade/Shutterstock; Paulo Bobita/Search Engine Journal