Unified index for 5 mainstream textbooks. Each book is worth independent intensive reading; this wiki provides reading guides + core takeaways to help quickly decide which to read first.
Depth Status Description:
- 📗 Minimalist Introduction — Fully read cover to cover(177 pages), detailed content
- 📒 Large-Scale LLM — Core chapters completed(key sections out of 297 pages)
- 📕 Black Book / Fine-Tuning with PyTorch — TOC + preface level reading(to be read in depth when users need it, with a review to be added)
- 📓 Understand AIGC in One Book — PDF encoding error, waiting for OCR
1. 📗 Minimalist Introduction to Large Model Application Development (O'Reilly, 2024) — 177 pages
Authors: Olivier Caelen + Marie-Alice Blete (O'Reilly, 2024, translated into Chinese). Positioning:The most practical SDK hands-on guide. Written after ChatGPT reached one million users within five days of launch.
5-Chapter Structure + Core Key Points
Ch1 First Look at GPT-4 and ChatGPT
- LLM basic overview: Role of Transformer in LLMs + Tokenization + Prediction
- GPT history: GPT-1 → GPT-2 → GPT-3 → InstructGPT (RLHF) → GPT-3.5 / Codex / ChatGPT → GPT-4
- 7 use cases: Be My Eyes (assistance for the blind) / Morgan Stanley / Khan Academy (Khanmigo) / Duolingo / Yabble / Waymark / Inworld AI
- AI hallucination warning: GPT calculates
3695×123548confidently gives wrong answers → cannot be used for medical applications where truth is critical - Two optimization methods: plugins + fine-tuning
Ch2 Deep Dive into API
- OpenAI Playground hands-on
- OpenAI Python library (Hello World)
- ChatCompletion endpoint input and output format
- Completion endpoint (legacy API)
- Pricing + token limits + security and privacy
- Other APIs: Embeddings / Content Moderation / Whisper / DALL-E
Ch3 Application Development
- API key management + data security
- Software architecture + LLM vulnerabilities (Prompt Injection cannot be completely avoided)
- 4 example projects:
- Project 1: Press release generator
- Project 2: YouTube video summary (including Map-Reduce to handle long text)
- Project 3:The Legend of Zelda Expert(Redis + vector retrieval + embeddings = self-built RAG)
- Project 4: Voice control (Whisper transcription → GPT processing)
Ch4 Advanced Techniques
- Prompt engineering:Effective prompt design / Chain-of-Thought step-by-step reasoning / few-shot learning / improving prompt effectiveness
- Fine-tuning: Getting started / OpenAI API fine-tuning / fine-tuning costs
- Generating and fine-tuning synthetic data for email marketing campaigns(complete case study)
Ch5 LangChain + Plugins
- LangChain framework:Dynamic prompts / agents + tools / memory / embeddings
- GPT-4 plugins: Overview + API + plugin manifest + OpenAPI specification
3 Most Worthwhile Takeaways from Minimalist Introduction
- Zelda Expert project principle(Ch3 Project 3) — is LLM Interview / Complete Special Topic on Retrieval-Augmented Generation (RAG)concise practical edition of
- AI hallucination example —
3695×123548miscalculation example — is the introductory version of Wang Kai Collection / Intensive Reading of 2024-2026 Cutting-Edge AI Papers and Reports OpenAI Hallucination paper - Email marketing synthetic data fine-tuning case(Ch4) — the minimal complete demo of the Self-Instruct approach
2. 📒 Large-Scale Language Models: From Theory to Practice (2023-09) — 297 pages
Authors: Zhang Qi / Gui Tao / Zheng Rui / Huang Xuanjing (Fudan University school). Positioning:An academic-style Chinese LLM survey textbook. Written in 8 months, published in 2023-09.
Core Stance
"The capabilities demonstrated by ChatGPT in open-domain question answering, various generative tasks, and dialogue understanding far exceed most people's imagination." "Since OpenAI has not disclosed the detailed implementation details of ChatGPT, we need to systematically reconstruct this field starting from first principles."
Timeline Core: 3 Stages of LLM Development
| Stage | Period | Representative Events |
|---|---|---|
| Foundation Model Stage | 2018-2021 | BERT / GPT / Baidu ERNIE / Huawei Pangu-α / PaLM |
| Capability Exploration Stage | 2019-2022 | Instruction fine-tuning / prompt learning / MaaS concept |
| Breakthrough Development Stage | Starting from ChatGPT in 2022-11 | One model completes multiple tasks, outperforming specialized models |
Disciplinary Integration
"Research on large language models integrates multiple disciplinary fields including natural language processing, machine learning, distributed computing, and parallel computing."
Differences from Other Books
- Compared to *Understand AIGC in One Book*100% more academic
- Compared to *Minimalist Introduction*More theory-focused, less hands-on practice
- Compared to *Black Book*Original Chinese work, more suitable for Chinese readers
Target Audience
- AI algorithm engineers / graduate student beginners
- People who want to fully understand the ChatGPT training pipeline
- Chinese writers (convenient for reusing sentence patterns)
3. 📕 Large Model Black Book: Based on GPT-3/4 Transformer NLP (Tsinghua University Press) — 379 pages
Author: Denis Rothman (France, Packt Publishing, 2022 English edition). Positioning:Authoritative monograph on Transformer architecture. Translated by Ye Weimin.
Core Coverage
Complete path of "building, training, fine-tuning" Transformers:
- Hands-on practice with Hugging Face library
- Python implementation
- Detailed explanation of model architecture (Encoder / Decoder / Attention)
- Comparison of BERT / GPT / T5 series
- Fine-tuning + deployment
Suitable For
- People who want to understand Transformer implementation from the bottom up
- Preparing for NLP algorithm engineer interviews
- Needs for paper reading and reproduction
Differences from Other Books
- Compared to *Minimalist Introduction*5 times deeper(code implementation details)
- Compared to *Large-Scale LLM*More focused on the Transformer architecture itself
- 2022 English original edition,some content has been updated by 2024-2026(new advances such as Flash Attention / GQA / Long Context are not covered)
Chapters Pending Deep Intensive Reading
- Original implementation of Encoder-Decoder architecture
- Hands-on fine-tuning with Hugging Face
- Multilingual + multimodal Transformers
4. 📔 Large Model Development and Fine-Tuning from Scratch (Based on PyTorch+ChatGLM) — 376 pages
Positioning:Complete development process textbook for PyTorch + ChatGLM. The hands-on book closest to "domestic model fine-tuning".
Core Coverage (pending deep intensive reading, inferred from title)
- PyTorch basics (Tensor / autograd / optimizers)
- Transformer implementation with PyTorch
- ChatGLM model analysis
- Fine-tuning (LoRA / QLoRA / Continue Pretrain)
- Deployment (single machine / multi-machine)
- Application development
Suitable For
- Domestic algorithm engineers working on large model SFT / RLHF
- Users of ChatGLM series (GLM-130B / ChatGLM3 / ChatGLM4)
- People who want to build large models under domestic compliance frameworks
Differences from Other Books
- Domestic model oriented(ChatGLM rather than OpenAI)
- Code practice oriented(376 pages, 2x deeper than *Minimalist Introduction*)
- Does not cover more advanced RLHF / 3D Parallelism
Pending Deep Intensive Reading
This wiki only provides TOC-level reading guide. Full chapter hands-on practice will be revisited for deep intensive reading of this PDF when users actually fine-tune ChatGLM.
5. 📓 Understand AIGC in One Book (2023) — 256 pages
Author: Anonymous (published 2023, general science popularization orientation). Positioning:The most general AIGC popular science book.
Status
PDF text extraction severely garbled(scanned book or font anomaly). Expected content includes:
- AIGC general knowledge (GAN / Diffusion / Transformer)
- ChatGPT hands-on
- AI painting tools (Midjourney / DALL-E / Stable Diffusion)
- Commercial application cases
Processing Recommendation
- Marked as
partial_scan_pdf_corrupted - waiting for OCR or re-purchase of a legitimate copy before re-ingesting
- Skipping does not affect the knowledge system(same topics are already covered in *Minimalist Introduction* + *Large-Scale LLM* + Wangkai's AIGC 2023 archive)
Reading Selection Recommendations for 5 Books
Priority 1 (Must-read)
📗 Minimalist Introduction to Large Model Application Development — 2024 edition is the most hands-on, 5 chapters 177 pages, finish in 1 week. Already summarized in detail on this wiki.
Priority 2 (Recommended)
📒 Large-Scale LLM — Authoritative Chinese textbook from Fudan academic school, suitable for algorithm engineers. 📕 Large Model Black Book — Authoritative on Transformer architecture, suitable for in-depth interview preparation / paper reading.
Priority 3 (On-demand)
📔 PyTorch+ChatGLM from Scratch — read only when doing ChatGLM fine-tuning. 📓 Understand AIGC in One Book — skip (content already covered by other channels).
Relationship with Existing Wiki
| Book | Replacement / Supplement |
|---|---|
| 📗 Minimalist Introduction | Supplements LLM Interview / Basic Architecture and Components GPT API hands-on details |
| 📒 Large-Scale LLM | Supplements Foundation Models for NLP (Springer Textbook Intensive Reading) Chinese perspective |
| 📕 Black Book | Supplements LLM Interview / Basic Architecture and Components Transformer depth |
| 📔 PyTorch+ChatGLM | 📔 PyTorch+ChatGLM from Scratch Supplements LLM Interview / Complete Special Topic on Fine-Tuning and Reinforcement Learning domestic model practice |
| 📓 AIGC 2023 | Replaces part of Wang Kai Collection / 2023 AIGC Industry Report Archive content |
See also
- LLM Interview / Basic Architecture and Components / LLM Interview / Complete Special Topic on Retrieval-Augmented Generation (RAG) / LLM Interview / Complete Special Topic on Fine-Tuning and Reinforcement Learning / LLM Interview / 8 Illustrated Special Topics on Distributed Training — 4 LLM interview special topics
- AI Product Manager Interview and Transition / Compilation of 12 Materials — AI PM interview
- Foundation Models for NLP (Springer Textbook Intensive Reading) — Springer Foundation Models for NLP 436-page textbook
- Wang Kai Collection / 2023 AIGC Industry Report Archive — Wang Kai 2023 AIGC Industry Report Archive
- Wang Kai Collection / Intensive Reading of 2024–2026 Cutting-Edge AI Papers and Reports — 2024–2026 Cutting-Edge AI Papers
- dontbesilent AI Self-Media Course / 13 Lessons of Systematic Methodology — The "Model Selection" segment in Old D's self-media course (more aligned with SDK hands-on practice)
- newtype · Prompt / Context Engineering in Practice — Old Huang's Prompt Engineering Philosophy from newtype
Subsequent ingest recommendations
- 📕 Black Book: Intensive deep reading for users preparing to read Transformer papers / for algorithm interviews
- 📔 PyTorch+ChatGLM: Intensive reading when users actually fine-tune ChatGLM
- 📓 Understand AIGC in One Book: Ingest after OCR processing (the current PDF has garbled encoding and is unreadable)
- 📒 Large-Scale LLMs: Can be split into 3 sub-wikis: "Basic Theory + Training Practice + Deployment Application" (when demand arises)