The SEO × LLM Era (Cross-Author Topic Synthesis)

When to open this: You are running a content site / want to capture AI Overviews traffic / are deciding whether to write llms.txt / want to adapt your site to be "LLM-friendly". This wiki places three schools of thought side by side for comparison: Google's official stance, Karpathy's argument for "rewriting infrastructure for Agents", and newtype's approach of "building an LLM-friendly corpus at the publishing layer".

One-sentence core:Google: how-not-who (evaluate quality, not the tool used) + crack down on scaled content abuse;Karpathy: robots.txt → llms.txt (rewrite web infrastructure for Agents);newtype: Instead of waiting for Google to hand out sugar, build your own LLM-friendly corpus layer.

0. Three perspectives on "SEO × LLM"

Perspective	Source	Position	One-liner
Google Official	GSC Blog: The evolution of AI search and Google's stance on AI content (2015–2026)	Regulator / Platform	"We focus on quality, not how content is produced."
Karpathy / Protocol camp	Karpathy · Software in the era of AI(Software 3.0) newtype miscellaneous practice: llms.txt protocol + CrewAI Obsidian project + n8n MCP workflow	Infrastructure rewriter	"In the Software 3.0 era, we must rewrite the web for Agents (robots.txt → llms.txt)"
newtype creator	Full evolution of the newtype knowledge system and Life OS	Third-party publisher	"I run my own wiki site, build LLM-friendly corpus, and bypass search engines"

1. Cross-author consensus (4 points)

AI-generated content will not be banned with a one-size-fits-all policy —— Google explicitly states "AI content is not a violation unless it is used to manipulate rankings"; Karpathy / newtype treat AI as a collaborator by default
Content quality + real value is the only long-term standard —— Google's E-E-A-T / Helpful Content; newtype's "continuously convert external information into your own material"
The importance of page structure for LLMs is rising —— Google has AI Overviews cite clearly structured pages; the llms.txt proposal is directly designed for this; newtype wiki uses clean Markdown / H1 / blockquote structure
Creators should proactively disclose AI involvement —— Disclose when readers would ask "How was this created?" (Google FAQ); newtype by default marks which passages are AI-collaborated in posts

2. Explicit disagreements (3 points — not smoothed over)

Point of disagreement	Google's position	Karpathy / llms.txt position	newtype practical position
Whether to write llms.txt	No official stance / no commitment to read it	"Web infrastructure should be rewritten" / it is a supporting component of Software 3.0	Have written it / but do not count on Google; it is mainly for OpenAI / Anthropic Bot to read
Crediting AI as an author	"probably not the best way"(2023/02 FAQ)	Does not discuss	Defaults to human as primary author / AI as collaborator
Whether to rely on search engine traffic	Implicit assumption	Karpathy: Agents read llms.txt directly and bypass search	newtype:Proactively bypass search, build owned AI traffic (Skill citations / direct wiki links)

3. Detailed breakdown of Google's official position

GSC Blog: The evolution of AI search and Google's stance on AI content (2015–2026) + GSC Blog: Evolution of quality policies and E-E-A-T (Webmaster Guidelines → Search Essentials):

3.1 Core proposition: Quality comes before "who/what wrote it"

"Our focus on the quality of content, rather than how content is produced, is a useful guide that has helped us deliver reliable, high quality results to users for years." —— Danny Sullivan + Chris Nelson, 2023/02

Historical analogy: During the 2011 Panda era, Google did not ban "human-written but mass-produced low-quality content"; instead it upgraded its systems to identify quality. Facing the LLM era, Google reiterates the same path —— do not ban AI content, evaluate its quality.

3.2 Boundary: scaled content abuse

"Using automation—including AI—to generate content with the primary purpose of manipulating ranking in search results is a violation of our spam policies."

Judgment is based on intent, not the tool:

✅ AI-generated weather, sports, transcripts → long-term compliant
✅ AI-assisted creative expression → compliant
❌ Mass-produced AI content with the primary goal of "gaming rankings" → violates spam policy

Enforcement systems:SpamBrain.

3.3 Official advice for creators (2023/02 FAQ)

Question	Google's position (condensed from original wording)
Is AI content a violation?	Not a violation —— unless used to manipulate rankings
Why not just ban AI content outright?	Automation has always had legitimate uses in publishing
Will AI content get preferential treatment?	"It's just content." Neither preferential treatment nor suppression
Should I use AI to write?	Using AI to produce useful, original content = okay; using it as a cheap ranking-gaming tool = don't
Do I need to credit an author?	You should credit an author when readers would ask "Who wrote this?"
Do I need to disclose AI involvement?	You should disclose it when readers would ask "How was this created?"
Can I list AI as an author?	"probably not the best way"

3.4 The Who / How / Why three-question framework

Google added a new framework in its Creating helpful, reliable, people-first content help page:

Who Who wrote it
How How it was written (including whether AI tools were used)
Why Why it was written (for people or for search engines)

"Evaluating your content in this way, whether you're using AI-generated content or not, will help you stay on course."

3.5 Compatibility between E-E-A-T and AI

GSC Blog: Evolution of quality policies and E-E-A-T (Webmaster Guidelines → Search Essentials):

Dimension	Meaning	Requirements in the AI era
Experience	First-hand experience	AI cannot replace personal experience → must be augmented by real human experience
Expertise	Expertise	AI-assisted synthesis + human expert review
Authoritativeness	Authoritativeness	Real authors + domain reputation
Trustworthiness	Trustworthiness	Sources, disclosure, traceability

4. The llms.txt proposal (protocol camp)

newtype miscellaneous practice: llms.txt protocol + CrewAI Obsidian project + n8n MCP workflow:

4.1 Current status: still a community proposal

It is not a formal standard at present
Proposed by developers / companies; it can only become a standard after adoption by all parties
There are currently two format proposals that are incompatible with each other(you cannot place both in the root directory at the same time)

4.2 Proposal A: Access Control Model (robots.txt-like)

Goal: Control whether site content is used for model training.

Directive	Purpose
`user-agent`	Specify which models / crawlers the rules apply to
`allow` / `disallow`	Allow / disallow access to specific paths
`license`	Link to license agreement (CC)
`crawl-delay`	Minimum interval (seconds)
`sitemap`	Sitemap URL

4.3 Proposal B: Inference Guidance Model (Markdown format)

Goal: Solve the problem of limited LLM context windows. Use a concise Markdown file to help LLMs quickly understand the core content and structure of a website, so they behave more like an "expert" when answering user questions.

Core motivations:

Improve LLMs' efficiency in understanding and using websites during inference
Overcome difficulties in parsing complex HTML (navigation / ads / JS)
Provide AI-friendly entry points for scenarios such as software documentation, API references, and business introductions

File structure:

Required: H1 title + Blockquote summary
Optional: H2-separated sections / file lists / Optional sections (can be skipped in short contexts)

4.4 Tool ecosystem

llms_txt2ctx —— convert llms.txt into context consumable by LLMs
Some mainstream documentation sites (FastAPI / Anthropic Docs / Hugging Face) have begun testing it

4.5 Karpathy's argument: Rewrite software infrastructure for Agents

Karpathy · Software in the era of AI(Software 3.0) §7:

"robots.txt → llms.txt; HTML → Markdown / API; search → Agent invocation"

Karpathy stated clearly in his YC AI Startup School talk (2025-06):The Agentic Decade requires rewriting the entire set of software infrastructure for Agents, rather than making Agents struggle to parse HTML designed for humans.

5. newtype: The "publishing layer" that proactively bypasses search

Full evolution of the newtype knowledge system and Life OS § three-layer architecture:

5.1 Third layer = publishing layer (wiki.newtype.pro)

┌─────────────────────────────────────┐
│  1. 知识库层(Knowledge Layer)       │
│     - 外部资料 / 个人资料             │
├─────────────────────────────────────┤
│  2. 技能层(Skill Layer)            │
│     - Skills / Sub-agents / OS       │
├─────────────────────────────────────┤
│  3. 发布层(Publish Layer)           │ ← wiki.newtype.pro
│     - LLM 友好语料                    │
│     - Skill 直接引用                  │
│     - 与流量解耦                      │
└─────────────────────────────────────┘

5.2 Design philosophy

Do not do SEO for search engines, build "readable corpus" for LLM Agents
Strict Markdown / H1 / clean links / no navigation pollution
Content is directly cited by one's own Skills (closed-loop)
Also allows other Agents to crawl it (open language model commons)

5.3 Relationship with Google SEO

Dimension	Traditional SEO	newtype publishing layer
Target audience	Google users	LLM Agents + users
Optimization direction	Keywords / links / E-E-A-T	Markdown structure / topic completeness
Traffic source	Google Search	Skill citations / direct links / X shares
Update frequency	Continuous	Continuous
Core KPI	Rankings / clicks	Number of times cited by Skills / conversions

Key insight:Rather than betting on Google AI Overviews handing out sugar, build yourself into a "corpus source cited by LLMs".

6. Position on AI Overviews / Succeeding in AI Search

GSC Blog 2021–2026: The Helpful Content / Page Experience / AI Overviews era + 2025/05 Succeeding in AI Search + Five major Google Search AI algorithm milestones (RankBrain/BERT/MUM/HCU/AIO) §6:

6.1 How AI Overviews works

Google AI Overviews (formerly SGE) = custom Gemini model(2024-05-14 Liz Reid)
Pages that are cited will receive a link citation
It is not zero-click —— Google emphasizes there will still be clicks
Optimization advice: Make your page "the one that gets cited"

6.2 Succeeding in AI Search(2025/05)

Five official Google recommendations:

Authentic, original content —— Do not do spun content
Strong page experience —— Fast loading / mobile-friendly
Meaningful structure —— Clear H1/H2/lists
Multi-modal content —— Images / videos / tables
Trust signals —— author / publisher / citations

6b. The 5 Major Milestones of Google's Algorithms (2015–2024) — The Required Constitution for SEO

See The 5 Major AI Algorithm Milestones of Google Search (RankBrain/BERT/MUM/HCU/AIO) for a complete compilation of official original texts:

Milestone	Year	Practical SEO Implications
RankBrain	2015	First deep learning ranking / words → concepts → Do not stuff keywords
Neural Matching	2018	Fuzzy matching between queries and documents → Write natural-language paragraphs; exact-match terms are not required
BERT	2019	Transformer understands word-to-word relationships within sentences → Small words such as "for/to/no" also matter; write naturally
MUM	2021	1000x BERT / 75 languages / multimodal → High-quality non-English content feeds back into English results; add images and videos
HCU	2022-08	Site-wide"Writing for people vs. writing for search engines" → Cleaning up low-quality content on the site > adding new good pages
AI Overviews	2024-05	Custom Gemini / generative answers → Compete to be "the one that gets cited"

Key Insight(Pandu Nayak official overview, 2022-02):

"Google's Search infrastructure runs hundreds of algorithms and machine learning models that work in combination rather than isolation."

SEO Implication:Any advice to "specifically optimize for algorithm X" is wrong(e.g., "optimize titles for BERT" / "optimize for MUM"). What you optimize is content quality itself.

7. How to Do It — Role-Based SEO × LLM Pathways

Who you are	Primary method	Starting point
Content sites / media	Google how-not-who + E-E-A-T + Succeeding in AI Search	Real human authors + first-hand Experience + Markdown structure
Documentation sites / APIs	llms.txt Inference Guidance Model	Write a /llms.txt + H1 + blockquote summary
AI training data control	llms.txt Access Control Model	disallow + license CC-BY-NC
Creators / knowledge bases	newtype publishing layer pattern	Self-owned wiki + LLM-friendly Markdown + no dependence on Google
Worried about AI content being demoted	Google "It's just content" + disclosure	Treat AI as a collaborator + self-check with the Who/How/Why three questions
Want to bet on AI Overviews	The 5 points of Succeeding in AI Search	original + structure + multi-modal + trust signals

Universal disciplines:

AI content itself is not a violation(explicitly stated by Google)
Intent matters more than the tool(scaled content abuse is judged by primary purpose)
Disclose AI involvement(Who/How/Why)
Do not list AI as the author(Google FAQ)
llms.txt is not mandatory(but there is no harm in writing it, and the file cost is extremely low)
True long-termism: self-owned LLM-friendly corpus layer + do not bet on a single channel

8. Citation List

GSC Blog: The Evolution of AI Search and Google's Stance on AI Content (2015–2026) — 18 Google articles on AI stance / how-not-who / scaled content abuse / Who-How-Why
GSC Blog: The Evolution of Quality Policies and E-E-A-T (Webmaster Guidelines → Search Essentials) — complete explanation of E-E-A-T
GSC Blog 2021–2026: The Helpful Content / Page Experience / AI Overviews Era —— Helpful Content Update / AI Overviews
newtype Miscellaneous Practice: llms.txt Protocol + CrewAI Obsidian Project + n8n MCP Workflow — two format proposals for llms.txt + tool ecosystem
Karpathy · Software in the era of AI(Software 3.0) — Karpathy YC 2025 talk / Agentic Decade / robots.txt → llms.txt
Full Evolution of the newtype Knowledge System and Life OS — three-layer architecture / wiki.newtype.pro publishing layer