Close reading notes for 10 cutting-edge AI papers from 2024–2026. This batch differs from the 2023 AIGC industry reports (see Wang Kai Collection / 2023 AIGC Industry Report Archive) — it is more recent, deeper in perspective, and more impactful, representing, as AGI enters its next phase,the research and manifestos that truly shape ways of thinking.
1. Aschenbrenner, Situational Awareness (June 2024, 165 pages) — The most influential AGI timeline manifesto
Core thesis
AGI by 2027 is strikingly plausible(AGI by 2027 is highly plausible).
Author Leopold Aschenbrenner, former OpenAI researcher, published it in June 2024; it has since become synonymous with "situational awareness" in San Francisco AI circles.
5 major chapters + key judgments
| Chapter | Core claim | Data anchors |
|---|---|---|
| I. From GPT-4 to AGI: Counting the OOMs | GPT-2 → GPT-4 = preschool child → smart high school student over 4 years. compute 0.5 OOM/year + algorithmic efficiency 0.5 OOM/year + unhobbling (chatbot → agent) = another equivalent leap by 2027 | 4 years ≈ 4–5 OOMs; 2024 → 2027 brings another 4–5 OOMs |
| II. From AGI to Superintelligence: the Intelligence Explosion | AI does not stop at human level. Hundreds of millions of AGIs automate AI research, compressing 10 years of algorithmic progress into 1 year (5+ OOMs in 1 year).Human-level → superhuman-level will happen within 1 year | Specific mechanism of "fast takeoff" |
| IIIa. Racing to the Trillion-Dollar Cluster | $10B → $100B → $1T compute clusters. U.S. electricity output to increase by dozens of percentage points by the end of this decade.Reindustrialization: trillions of dollars for GPUs + datacenters + power | The concept of the "trillion-dollar cluster" originates here |
| IIIb. Lock Down the Labs | Safety at U.S. AI labs is a joke (an afterthought).They are giving key AGI secrets away for free to the CCP. State-actor-level threat, and we are not prepared | He was fired after leaving OpenAI because of this chapter |
| IIIc. Superalignment | Controlling AI far smarter than us is an unsolved technical problem.It is solvable, but easy to derail during an intelligence explosion. Failure could be catastrophic | Continues the perspective of Ilya/Jan Leike's "superalignment team" |
| IIId. The Free World Must Prevail | Superintelligence = decisive economic and military advantage.China is still in the game. Existential stakes for the free world | Introduces the geopolitical dimension |
| IV. The Project | Once the AGI race heats up, U.S. national security agencies will intervene.There will be some kind of government AGI program in 2027/2028. "in a SCIF, the endgame will be on" | Prediction of a turning point from private companies → national projects |
Wang Kai's applications
This document had an enormous influence on Wang Kai,and is repeatedly cited in his posts as the source for "viewing the present from an AGI perspective." For example:
- In Wang Kai's Decision-Making System for the AI Era he proposes "viewing the present from an AGI perspective" — a way of thinking directly derived from this paper
- In Wang Kai's Observations on AI Industry Judgments and Big Tech Competitive Landscape the "battle of the three Scaling Laws" corresponds to the paper's "counting the OOMs"
- In Wang Kai's RSI AI Investment Strategy and the "Good Asset, Good Price, Good Volatility" Framework "Interrogating Claude's value in the AI era" — uses Aschenbrenner's timeline as a stress test
Key sentences (original text)
"By 2025/26, these machines will outpace college graduates. By the end of the decade, they will be smarter than you or I; we will have superintelligence, in the true sense of the word."
"Hundreds of millions of AGIs could automate AI research, compressing a decade of algorithmic progress (5+ OOMs) into 1 year."
"The most extraordinary techno-capital acceleration has been set in motion."
Objections (perspectives outside Wang Kai)
- "AGI by 2027" has been criticized by Yann LeCun / Gary Marcus as a linear extrapolation fallacy — the OOMs scaling assumption may hit walls in data / algorithms / physics
- But using it as an "upper-bound scenario" for investment / career decisions is a reasonable stress test(Wang Kai's approach)
2. OpenAI, Why Language Models Hallucinate (September 2025) — The training origins of hallucinations
Core findings
Hallucinations are not a mysterious phenomenon; they are a statistical inevitability.
LLMs are like students encountering hard problems on an exam — when uncertain, they guess, producing outputs that "sound plausible but are wrong." This behavior persists in SOTA systems (OpenAI 2025a) and erodes trust.
Two-stage analysis
| Stage | Error source | Key insight |
|---|---|---|
| Pretraining | Even if training data is perfectly correct, statistical objectives lead to errors | Reduce the generation problem to an Is-It-Valid (IIV) binary classification problem. Generating a valid output ≥ answering "Is this valid?". Therefore LLMs are at least as hard as binary classification,and generating hallucinations is an inevitable product of binary classification errors |
| Post-training | Evaluation methods reward "guessing" rather than "admitting uncertainty" | LLMs are optimized to be good test-takers. Benchmarks like MMLU give 0 points for "I don't know" and full points for a correct guess — in expected value, guessing is always ≥ "I don't know" |
Empirical cases
Ask the LLM: "Adam Tauman Kalai's birthday?"
- Three different wrong dates across three answers (03-07 / 15-06 / 01-01)
- Even when prompted "admit it if you don't know"
Ask the LLM: "How many Ds in DEEPSEEK?"
- DeepSeek-V3 answered "2" or "3" across 10 independent trials (the correct answer is 1)
- Claude 3.7 Sonnet was similar, answering as high as "6" and "7"
Solution
Socio-technical mitigation: instead of introducing more "hallucination evaluations,"change the scoring rules of existing mainstream benchmarks:
- "I don't know" should score higher than a wrong answer
- to give models an incentive to say "I don't know"
Application to value investing
Wang Kai uses this paper as:
A cautionary example for "using AI as a ruler to quantify fuzzy risks" — AI also hallucinates,what you should do is not 100% trust AI's numbers, but use AI's "uncertainty signals"
See details in Wang Kai's RSI Investment Case Collection + AI Value Investing Philosophy in the section "AI is an amplifier of personality."
3. OpenAI, How People Use ChatGPT (September 2025, OpenAI × Duke × Harvard) — First-principles user research data
Data scale
- Time window: May 2024 ~ June 2025
- Sample: random messages from three consumer plans — Free / Plus / Pro
- Classified via a privacy-preserving automated pipeline
- Harvard IRB approved (IRB25-0983)
- July 2025 data points:700M users / 18 billion messages per week / ~10% of global adults
5 major insights
- Early user imbalance → gender gap rapidly narrowing: early male dominance, now approaching parity
- Higher growth rates in low-income countries: global inclusion basically achieved
- Non-work messages growing faster: 53% → 70%+. Chatting / searching / writing daily tasks dominate
- Work use concentrated among educated, high-paid professional roles: knowledge workers use it more intensively
- 3 major conversation themes account for 80%: "Practical Guidance" + "Seeking Information" + "Writing"
- Writing dominates work tasks — highlighting the essential difference between chatbots and search engines (generating digital outputs vs retrieving links)
- Programming accounts for a small share — contrary to developer community intuition
- Self-expression accounts for a small share — contrary to the "AI boyfriend / girlfriend" narrative
Economic conclusions
ChatGPT creates economic value through "decision support" — especially in knowledge-intensive work.
It is not "replacing you," it is "helping you think."
Application to Wang Kai's product judgments
- The bet on "AI replacing search engines" is supported by this data — Writing accounts for the majority of work tasks, which search cannot do
- The "AI stationery / AI Notion-like" track (a main direction in Wang Kai's idea library) is again backed by data
- See details in Wang Kai's Library of 1000+ AI Product Ideas and Business Model Analyses in the "AI 1000+ ideas" list
4. Hosseini & Lichtinger, Generative AI as Seniority-Biased Technological Change (August 2025, Harvard) — Junior positions are being cut
Data scale
- LinkedIn resumes + hiring data (provided by Revelio Labs)
- 2015–2025 time window
- 285,000 U.S. firms / 62 million independent workers / 150 million employment events / 245 million job postings
- "AI integrator role" job posting text identification → flag firm-level adopters
- 10,599 firms (3.7%) identified as GenAI adopters
4 major findings
- During 2015–2022 junior and senior positions grew in sync → starting 2023 Q1 junior hiring at adopting firms plunged 7.7% (after 6 quarters)
- Mechanism: not layoffs, but stopping hiring(slow hiring, not increased separations)
- Industry: retail + wholesale hit hardest(adopting firms' quarterly junior hiring -40% vs non-adopters)
- U-shaped heterogeneity:
- Top Tier 1 schools → small impact
- Tier 2–3 mid-tier schools → largest decline(broad upper-middle hit)
- Tier 4–5 lower-tier schools → small or insignificant impact (AI cannot yet do execution roles)
Triple-difference robustness
- DiD: adopting vs non-adopting × junior vs senior (could be contaminated by firm-level shocks)
- Triple-difference: + firm-by-time fixed effects → difference between junior and senior at the same firm at the same time
- Results consistent → not macro shocks or firm-specific shocks
Implications for individual choices
Logic Wang Kai does not state explicitly but aligns with:
- Junior positions cut → the "entry-level" path no longer works
- Senior positions still growing → people already in the workforce become more valuable
- Entrepreneurship + self-employment is the most direct path to avoiding the junior trap in the AI era — Wang Kai's Entrepreneurship Methodology and Business Judgment Framework in the AI Era The logical foundation for rejecting the lingering internet-era fallacy that "funding = success"
V. Reddit r/ChangeMyView "Can AI Change Your View?" — A Large-Scale Persuasion Experiment
Experimental Design
- Platform: r/ChangeMyView (~4 million users, top 1% subreddit)
- Users post to discuss their own controversial views and seek arguments that could change their minds
- LLMs register accounts, write persuasive arguments, and submit comments
- Users award a "delta" (▲) symbol to indicate "my view was changed"
- Three treatment groups:
- Ordinary LLM
- personalized LLM (customized using users' public profiles)
- human control group
Key Findings
- LLMs match or even exceed humans in persuasiveness(under certain conditions)
- Personalization significantly improves persuasiveness(LLMs adjust arguments based on users' public history)
- "Laboratory settings" in prior research underestimated in-the-wild persuasiveness
Risk Dimensions
- Malicious actors can exploit GenAI to produce large-scale, highly refined deceptive content
- Manipulating public opinion / shaping narratives to serve specific agendas
- AI persuasiveness vs. the information ecosystem of democratic societies core conflict
How Wang Kai Applies It
- Wang Kai's Overseas Promotion SOP and Multi-Account Matrix Field Practice Overseas Promotion SOP — When Wang Kai builds GenAI content matrices,he is leveraging exactly this capability
- Wang Kai's Decision-Making System in the AI Era 5 decisions that can / cannot be delegated to AI — do not grant the "being persuaded" authority (know that AI can persuade you)
VI. OpenAI "A Practical Guide to Building Agents" (2025) — Agent Engineering Methodology
Core Definition
Agent = a system that can independently complete tasks on behalf of users(systems that independently accomplish tasks on your behalf).
What is not an agent: simple chatbots / single-turn LLMs / sentiment classifiers (using only an LLM without controlling workflow execution).
When to Build an Agent (Decision Tree)
| Signals | Examples |
|---|---|
| Complex decisions(nuanced judgment, exceptions, context sensitivity) | Customer service refund approval |
| Hard-to-maintain rules(extremely numerous, messy rules that are difficult to update) | Vendor security review |
| Heavy reliance on unstructured data(natural language, document extraction, conversations) | Insurance claims processing |
If the above 3 criteria are not met → a deterministic solution is sufficient.
3 Core Components
- Model:LLM provides reasoning and decision-making
- Tools:External functions / APIs enable the Agent to take actions
- Instructions:Clear guidelines and guardrails
Guardrails — the key focus of the chapter
Guardrails that production agents must have:
- Input guardrails:Prompt injection detection, jailbreak interception, personal information desensitization
- Output guardrails:Brand alignment, PII leak prevention, unsafe content filtering
- Behavior guardrails:Tool call whitelists, HUMAN-IN-THE-LOOP trigger conditions
- Resource guardrails:Max iterations / max cost / timeout
- Resource guardrails:When guardrails conflict with user goals, guardrails take priority
Wang Kai's Counterparts
See Wang Kai's Agent Engineering Hands-On Practice:
- "Agent Autonomous Decision-Making Experiments" correspond to the 4 major decision patterns in the Practical Guide
- "Multi-browser, multi-account" corresponds to the "tool ecosystem"
- "Agent grading (benchmarked against autonomous driving levels)" is Wang Kai's original addition
VII. Google "Prompt Engineering" by Lee Boonstra (2024-09)
Content Coverage
Not a prompt primer, but a full-stack handbook for production-grade prompt engineering.
Output Configuration:
- Output length control (max tokens)
- Semantics of the three knobs: Sampling controls / Temperature / Top-K / Top-P
Prompting Techniques:
- Zero-shot / One-shot / Few-shot
- System / Contextual / Role prompting Three-layer separation of responsibilities
- Step-back prompting (abstract first, then concrete)
- Chain of Thought (CoT)
- Self-consistency (sample multiple times and take the majority)
- Tree of Thoughts (ToT)
- ReAct (Reason & Act) — foundational primitives for Agents
- Automatic Prompt Engineering (APE)
Code prompting:
- Four independent prompt patterns: Code generation / Explanation / Translation / Debugging+Review
Best Practices:
- Provide examples
- Design with simplicity (contrary to the large Notion template trend)
- Be specific about output(JSON schema)
- Use instructions over constraints (affirmative phrasing > negative phrasing)
- Control max token length
- Use variables in prompts (templating)
- Experiment with input formats (compare Markdown / JSON / XML / YAML for the same question)
- Mix up classes in few-shot (to prevent majority class bias)
- Adapt to model updates (retest after every model update)
- Experiment with output formats
Compared with newtype Prompt template library (4 production-grade prompts + 1 CLAUDE.md sample) Comparison
- newtype leans toward the "long prompt as documentation" philosophy (Super Analyst Skill 7-Stage)
- Boonstra leans toward the "short prompt as design" philosophy
- The two complement each other:
- Reusable SOPs for skill bundles → Boonstra framework
- One-off creative queries → newtype long prompts
VIII. Weijin Research "Looking at DAO 2025" — Chinese AI Annual Outlook
2024 Review (7 Major Judgments)
| Judgment | Key Events |
|---|---|
| Computing power "breaks through walls" | NVDA GPU annual updates / Google TPU v6 / AWS Trainium2 / OpenAI self-developed chips / Huawei 910C / ByteDance 5nm / TSMC building fabs in the US (but packaging returns to Taiwan) |
| Medical agents | Google AI doctor outperforms primary care physicians in diagnosis / Apple Watch apnea detection / Harvard cancer diagnosis model / Tsinghua Agent Hospital with 21 departments and 42 AI doctors / China's first inclusion of AI-assisted diagnosis in medical insurance |
| Autonomous driving large models | Tesla 70,000 H100s / FSD v13 / Cybercab entering production in 2026 / China's BYD + Huawei self-developed chips / WeRide + Pony.ai going public |
| Mass-produced humanoid robots | Multiple small-batch orders / Optimus sharing Tesla parts / Agibot thousand-unit mass production / Unitree G1 at 99,000 yuan |
| Spatial computing moves from metaverse to "new reality" | Vision Pro 420,000 units for the year / Quest 3 price cuts / Google, OpenAI, and World Labs all building world models / Meta Orion glasses roadmap |
| "Universal" gene editing | Xenotransplantation (pig heart / pig kidney / living pig liver) / OpenCRISPR-1 / CHOOSER large model identification / Genome Evo large model |
| US-China venture capital diverges again | 30 companies account for 75% of US VC / China valuation adjustment mechanism repurchase turmoil with exit pressure on 14,000 companies / Middle East sovereign funds + National Integrated Circuit Industry Investment Fund Phase III at 344 billion yuan |
10 AI Outlooks for 2025 (Excerpts)
- Space economy catching up to "cosmic velocity"
- Mass-producing "five-star driver" autonomous driving
- Ultimate agents — humanoid robots
- Global innovative drug secondary manufacturers — biopharmaceuticals
- Preparing for next-generation batteries — solid-state batteries
- Second front of chips — inference chips
- Diffusion of innovation — Global South
Linked with newtype AI Business, Industry, and Investment Observations Association
- newtype leans toward "business wars / valuation data / three-layer framework"
- Looking at DAO leans toward "hardware / policy / China-US landscape"
- Put together, the two = a complete map of the Chinese AI industry for 2024–2025
IX. Tsinghua Metaverse Culture Lab "DeepSeek From Beginner to Master"
Core Distinction
Reasoning models vs. general-purpose models(After the emergence of Chain-of-Thought, large models diverged into two categories):
| Dimension | Reasoning large models (o1 / R1) | General-purpose large models (GPT-4o / Claude / DeepSeek-V3) |
|---|---|---|
| Advantage domains | Mathematical deduction / logical analysis / code generation / complex problem decomposition | Text generation / creative writing / multi-turn dialogue / open-ended Q&A |
| Disadvantage domains | Divergent tasks (poetry) | Strict logical chain tasks (mathematical proofs) |
| Strength assessment | Not comprehensively stronger; only significantly outperform general models in training-target domains | More flexible in general scenarios; specialized tasks rely on prompt compensation |
Fast thinking vs. slow thinking
| Dimension | Fast reaction (ChatGPT 4o) | Chain-of-thought reasoning (o1) |
|---|---|---|
| Response speed | Fast | Slow |
| Compute cost | Low | High |
| Computation principle | Probabilistic prediction | Step-by-step Chain-of-Thought reasoning |
| Decision-making capability | Preset algorithmic rules | Autonomous analysis + real-time decision-making |
| Creativity | Pattern recognition / optimization | Genuine innovation |
| Problem-solving ability | Structured, well-defined problems | Multi-dimensional unstructured problems |
| Ethics | Almost none | Autonomy + control discussion |
Differences in prompting strategies
Reasoning models → no need for CoT prompts (built-in); providing goals and constraints is sufficient General-purpose models → require explicit CoT guidance
How Wang Kai Applies It
Wang Kai's AI Coding Hands-On Practice: From Cursor to Claude Code Terminal Workflow states that "AI Coding is not writing code, but a foundational collaboration skill" — use reasoning models for decision-making, and general models for expression.
X. AGI Introduction (file_027, presentation slides)
Outline
- Past: historical development patterns + AGI definition / inflection points / development trajectory
- Present: state of large models / essence / problems and risks + capital + industry + practice
- Future: directions
- Thinking & learning: reflection + how to follow developments
(Pure slide framework with no substantive content, retained for reference)
Internal Thread Connecting the 10 Documents
Aschenbrenner → "AGI 多快来"
↓
How People Use ChatGPT → "已经怎么用了"
↓
Why LLMs Hallucinate → "用的时候要警惕什么"
↓
Seniority-Biased Change → "对就业格局的实证"
↓
Practical Guide to Building Agents → "如何构建下一代 Agent"
↓
Reddit r/ChangeMyView → "AI 说服力 — 用力的边界"
↓
Lee Boonstra Prompt Engineering → "战术层 — 现在能怎么写好"
↓
看 DAO 2025 + DeepSeek 入门 → "中文视角 — 国内格局与具体工具"
↓
AGI 介绍 → "通识框架"
The 10 documents = Wang Kai's first-hand AI information intake stack for 2024–2026,more complete than reading any single one alone.
See also
- Wang Kai Collection / 2023 AIGC Industry Report Archive — 2023 era archive (read comparatively for temporal depth)
- Wang Kai's Decision-Making System in the AI Era — Wang Kai's personal decision-making methodology / standing from an AGI perspective
- Wang Kai RSI Investment Case Collection + AI Value Investing Philosophy — investment practice + IFS psychological work
- AI Agent Technology Architecture Evolution 2025 (In-Depth Research Report) — newtype perspective on Agent architecture
- Karpathy · Software in the era of AI(Software 3.0) — Karpathy Software 3.0 (a manifesto on par with Situational Awareness)