KNOWLEDGE NOTESKnowledge Notes
Explore curated knowledge, course notes and methods, with their existing content and sources preserved.
This index lists complete English editions. Additional Chinese notes and bilingual updates remain available in the Chinese index.
350knowledge notesA research report on the evolution of AI Agent technology architecture compiled by newtype in January 2026. It is based on papers published from October 2024 to January 2026, official blogs from Anthropic/OpenAI/Google/Microsoft, open-source projects, and real-world deployment cases. It covers 6 major evolution trends / 4 multi-Agent orchestration patterns / Google's 8 foundational patterns / memory management (MemoryOS 3-layer architecture + vector database selection) / tool calling + MCP standardization / reasoning frameworks (ReAct → ToT → GoT → WoT evolution) / Large Reasoning Models (LRM) / 7-layer production architecture / comparison of mainstream frameworks (LangGraph/CrewAI/MAF/AutoGen/LlamaIndex/ADK) / real deployment cases from Morgan Stanley + Klarna + BBVA + Rakuten / 2026 trend forecasts / compliance and governance / production checklist. It serves as a "research report-level" supplement to [[newtype-agent-architecture]] (Huang Yihe's post-level judgments).
3 close readings of Agent research / talks collected by newtype Planet. 05 CAMEL-AI Wendong Fan "Finding the Scaling Laws of Agents" (2025 talk: # Parameters → # Agents analogy / CAMEL + OWL + CRAB + OASIS frameworks / Agentic Data Generation feeding back into training) / 07 "More Agents Is All You Need" (Tencent ICML 2024: brute-force scaling of Agent count via sampling-and-voting, Llama2-13B + 15 agents matches Llama2-70B, 3 dimensions determine gain magnitude) / 08 "ChatQA" (NVIDIA 2024: two-stage instruction tuning, 70B scores 54.14 vs GPT-4's 53.90 across 10 conversational QA datasets, fine-tuned retriever replaces query rewriting to save API costs, "unanswerable" sample training eliminates hallucinations)
Cross-author evals topic summary. Anthropic's 4 key lessons (6 percentage point gap from infrastructure noise / eval awareness where models detect testing / take-home test lifespan / pass^k math) + academic school (full set of SWE-Bench / Terminal-Bench / τ2-Bench / BrowseComp / 𝜏2 / OSWorld / CORE-Bench) + practical perspectives from Wang Kai / Lao Huang / Cal Rueb. Framework selection (Harbor / Braintrust / LangSmith / Langfuse / Arize) + 14-item PM self-check checklist. This wiki is the "horizontal cross-author expansion" of [[anthropic-engineering-demystifying-evals]] (Phase A anchor / Anthropic single-source deep dive)
Comprehensive guide to LLM distributed training. llm-interview-distributed (9 PDFs / distributed overview + pipeline parallelism + DataParallel/DDP + torch.multiprocessing + AMP mixed precision + DeepSpeed + accelerate + ZeRO) + vLLM / inference optimization stack. Covers 4 major parallelism strategies (DP/MP/PP/TP), 3D Parallelism, three stages of ZeRO, and the mainstream Megatron + DeepSpeed stack
Synthesis of LLM fine-tuning and alignment. llm-interview-finetune (19 PDFs / fine-tuning basics + PEFT + LoRA family + QLoRA + continued pretraining + RLHF + PPO + DPO + VRAM optimization + inference + Prompting + SFT data) / newtype fine-tuning practice + Wang Kai's LoRA application notes. Covers full-parameter vs PEFT, LoRA + QLoRA, RLHF vs DPO, SFT data construction, and VRAM optimization strategies
Comprehensive overview of LLM core fundamentals. Covers llm-interview-basics (three Decoder architectures: prefix/causal/encoder-decoder + Layer Norm LN/RMS/Deep + activation functions GeLU/SwiGLU + Attention upgrades MQA/GQA/Flash + Transformer operations + loss/similarity functions + repetition problem + 8 major mitigation strategies)/ Karpathy nanochat (reproducing LM in 243 lines of code)/ newtype Karpathy (Software 2.0 neural network weights). Covers three Transformer paradigms, mainstream LLM configurations (LLaMA/GPT/GLM), Attention evolution, and training experience
A synthesis of the MCP protocol ecosystem from 4 perspectives. newtype Lao Huang (evolution prediction from DOS→Win95→Internet explosion + Lao Huang's own Obsidian/Flux/Prompt House MCPs + commonly used MCP list + 7 major configuration pitfalls) / Wang Kai (FMP financial MCP + Chrome MCP modification + Feishu CLI / self-built MCP to solve data accuracy) / Lao D (domestic rap.dev + Xiaohongshu MCP server + Playwright) / AWS industry (Chang Gaowei's ANP protocol comparison with MCP). Covers the MCP vs GUI Agent debate, Smithery's failure, the default Sequential Thinking + Tavily stack, and the closed-loop MCP+n8n workflow.
Huang Yihe 2025-03 → 2026-05 Full observations on MCP. Basic concepts / MCPs built by Lao Huang himself / List of commonly used MCPs / Configuration troubleshooting / DOS → Win95 evolution prediction / Context engineering vs Prompt engineering / Skill + MCP collaboration / Buffett replication SOP
A fusion of Prompt Engineering from 4 perspectives. newtype Lao Huang (186 posts + 4 production-grade templates + 5-element task decomposition + Thinking-Claude performative thinking + Ultra think three tiers + Karpathy Context Engineering) / Wang Kai (6000+ words on PUA-ing Claude + using AI as a ruler for quantification + risk-scoring prompts + decision interrogation SOP) / Lao D (three-layer prompt architecture: meta/system/user + Markdown imitation writing SOP + 4-step blogger imitation) / Anthropic internal (Cal Rueb CLAUDE.md anti-patterns). Covers the counter-consensus view on "whether Prompt Engineering needs to be learned", Skill-ification as the end state, Prompt scheduling Prompt, and Sequential Thinking scheduling.
Integration of RAG and knowledge base engineering from 5 perspectives. newtype Lao Huang (5 generations of architecture evolution: AnythingLLM → Cursor+Obsidian → Claude Code AI OS → newtype OS → wiki.newtype.pro / 5 key points of RAG optimization / Milvus+MCP database-model separation / end state: own content → Obsidian, others' content → NotebookLM) / LLM Interview Technical School (14 special topics: R/A/G three modules + PDF parsing + table recognition + text chunking + HYDE + hard negative samples + RAG-Fusion + GraphRAG) / Lao D's counterintuitive teaching ("RAG does not make the model stronger; it automatically strengthens the prompt") / Wang Kai (Agentic Search outperforms RAG + self-built data MCP) / Cal Rueb (Anthropic does not use RAG MCP). Covers the three-way choice among RAG vs Long Context vs NotebookLM, the economics of GraphRAG, and the philosophy of knowledge bases.
Integration of Scaling Law + AGI + Software 3.0 from 4 perspectives. Karpathy's June 2025 talk (Software 1.0/2.0/3.0 triad + LLM = OS/Utility/Fab three analogies + LLM psychology + Iron Man Suit 4 principles + Agentic Decade 2025–2035) / Aschenbrenner's 165-page Situational Awareness (AGI by 2027 + OOMs counting + Trillion-dollar cluster + Intelligence Explosion) / Three-school debate (Zhu Xiaohu application layer / Cohere Aidan pragmatic / Kai-Fu Lee ceiling / Jensen Huang synthesis) / Wang Kai "Viewing the present from an AGI perspective" + Embiricos AGI bottleneck = human typing speed. Covers AGI timeline disagreements, Software 3.0 adoption forecasts, Vibe Coding natural language programming
