Foundation Models for NLP (In-Depth Reading of a Springer Textbook)
Complete in-depth reading of the 2023 open-access Springer textbook by Gerhard Paaß and Sven Giesselbach. 436 pages / arXiv:2302.08575. 8 chapters covering: Ch1 Classical NLP fundamentals (token / embedding / RNN / CNN) / Ch2 Three major PLM architectures (BERT autoencoder + GPT autoregressive + Transformer seq2seq) + training/regularization/uncertainty/interpretability / Ch3 Six major directions for improving PLMs (pretraining task variants + long-range sparse attention + multilinguality + knowledge injection + model scale + fine-tuning / few-shot / CoT) / Ch4 PLM knowledge evaluation (GLUE / SuperGLUE / BIG-bench + Probing) / Ch5 Information extraction (classification / NER / relation extraction) / Ch6 Text generation (retrieval / QA / translation / summarization / stories / code / dialogue) / Ch7 Multimodality (speech / image ViT / video / control Decision Transformer + GATO / DNA / Protein) / Ch8 Summary and outlook (new paradigm of Foundation Models / economic applications / 5 categories of harms / Grounding / Fast & Slow Thinking / Planning). Serves as the "academic foundation" supplement to [[newtype-agent-architecture-research-2025]] and [[newtype-agent-research-papers]]