NLP & Semantic Search Vintage
Bill Slawski documented the evolution of Google's natural language processing capabilities through patent analysis, tracking the journey from simple keyword matching to deep semantic understanding. These patents reveal how Google understands the meaning of content, not just its words.
NLP Evolution in Search
Phrase-Based Indexing
One of the most important NLP patents Bill analyzed. This patent describes how Google moves beyond individual words to index phrases — meaningful multi-word units.
How Phrase-Based Indexing Works
Key Concepts
| Concept | Description | SEO Implication |
|---|---|---|
| Good phrases | Multi-word units that occur together predictably | Using natural phrases > isolated keywords |
| Related phrases | Phrases that co-occur across many documents | Including related phrases builds topical depth |
| Phrase-based spam detection | Pages with unusual phrase combinations get flagged | Content should have naturally co-occurring phrases |
| Phrase-based relevance | Documents matching more related phrases rank higher | Comprehensive coverage > keyword repetition |
What This Means for Content
A page about "Italian restaurants" should naturally contain related phrases like:
- "pasta dishes," "wine list," "dining experience"
- "chef," "menu," "reservations"
- "cuisine," "trattoria," "Italian food"
A page about "Italian restaurants" that is stuffed with the keyword but lacks related phrases looks unnatural to a phrase-based indexing system.
Source: Phrase-Based Indexing: Why SEOs Need to Understand It (2007)
Word Sense Disambiguation
Google must determine which meaning of a word is intended in both queries and documents.
The Disambiguation Process
Context Vectors (2016 Patent)
Google's Context Vectors patent provides the technical mechanism for disambiguation:
- Each word usage is encoded with its surrounding context as a vector
- Different meanings of the same word produce different vectors
- Query vectors are compared to document vectors to find meaning-level matches
- This enables matching across different surface forms with the same meaning
Source: Google Patents Context Vectors to Improve Search (2016)
Semantic Closeness and Sets
Bill analyzed patents on how Google uses HTML structures (lists, tables, headings) to understand semantic relationships.
Semantic Structures in HTML
Google analyzes these structures across millions of pages to learn:
- What items belong in the same category (semantic sets)
- How concepts relate to each other
- What terms share similar semantic roles
Source: 10 Most Important SEO Patents: Part 7 - Sets, Semantic Closeness, Segmentation, and Webtables (2012)
Opinion and Sentiment Analysis
Google patents describe systems for analyzing opinions in content, particularly in reviews and user-generated content.
Sentiment Analysis Applications
| Application | How It Works |
|---|---|
| Review quality scoring | Identifying substantive reviews vs. generic ones |
| Aspect-based sentiment | Understanding sentiment about specific features ("battery life is great but camera is poor") |
| Opinion spam detection | Identifying fake or incentivized reviews |
| Reputation signals | Aggregate sentiment as a quality/trust indicator |
| Product attribute extraction | Learning what matters to users from opinion text |
Review Innovation
Bill's analysis of Google's product review patents (2006) showed how Google could build a review system that:
- Aggregates reviews across multiple retailers
- Weights reviews by reviewer authority and review quality
- Extracts specific product attributes from review text
- Presents structured opinion data alongside search results
Source: Innovating Product Reviews at Google (2006)
Entities, Profiles, and Language Models
Bill's analysis of how Google combines entity recognition with language models to understand content at a deeper level:
The Integration
This integration means:
- Entities provide the "what" — what things is the content about?
- Language models provide the "how" — how well is the content written?
- User profiles provide the "who" — who is searching and what do they need?
Source: SEO is Undead Again: Profiles, Phrases, Entities, and Language Models (2010)
BERT and Neural Language Understanding
BERT (Bidirectional Encoder Representations from Transformers) represents the latest stage of NLP evolution in Google's patents. Bill analyzed how BERT enables:
- Bidirectional context understanding — Understanding words based on ALL surrounding words, not just left-to-right
- Passage-level ranking — Scoring individual passages within documents for relevance
- Question-answer matching — Semantic matching between questions and answer text
- Zero-shot understanding — Understanding content about topics not seen during training
Key Takeaways
- Phrases matter more than keywords — Google indexes meaningful phrase units and their co-occurrences. Write naturally with related phrases.
- Context determines meaning — The same word in different contexts is treated as different concepts. Make context clear.
- HTML structure teaches Google — Lists, tables, and headings provide semantic structure that Google uses to learn relationships.
- Sentiment is analyzed — Google can determine opinion polarity, identify spam reviews, and extract specific attribute sentiments.
- Entity + language model + profile = modern ranking — The combination of what the content is about, how well it is written, and who is searching creates the modern ranking experience.
- BERT enables meaning-level matching — Content does not need exact keyword matches to rank; semantic similarity is sufficient.