AI Engineering ／ Large language model learning
LLM Interview / Distributed Training: 8 Illustrated Articles
Complete illustrated 8-PDF set for LLM large model distributed training interviews. Covers distributed training for large models (overview) + Pipeline Parallelism + nn.DataParallel + nn.parallel.DistributedDataParallel + torch.multiprocessing + AMP mixed precision training + DeepSpeed + accelerate + ZeRO learning. Systematically covers the three mainstream distributed training stacks: PyTorch, DeepSpeed, and Megatron