1 Introduction
2 Deep Learning
2.1 Linear Classification
2.2 Deep Learning
2.3 Discriminative and Generative Modeling
2.4 The Variational Autoencoder (VAE) framework
2.5 The Generative Adversarial Network (GAN) Framework
2.6 Autoregressive and Language Models
2.6.1 Autoregressive Modeling
2.6.2 Autoregressive Generation
2.6.3 Tokenizer
2.6.4 Introducing Attention
2.6.5 Attention Mechanism
2.6.6 Multi-Head Attention
2.6.7 Self-Attention
2.6.8 Masked Self-Attention
2.6.9 Transformer Architecture
2.6.10 KV cache
2.6.11 Knowledge Distillation
2.6.12 MoE and Conditional Computation
2.6.13 Reinforcement Learning Through Human Feedback (RLTHF)
2.6.14 Efficient Finetuning: LoRA & QLoRA
2.6.15 In-Context Learning
2.6.16 Large Reasoning Models versus Vanilla LLMs
2.6.17 Retrieval Augmented Generation (RAG)
2.6.18 Function Calling and MCP
2.6.19 Multi-Agent Systems
2.6.20 Language Models for Code
3 Tensors
3.1 Mapping High-Dimensional Arrays to Memory
3.2 Tensor, TensorImpl, and StorageImpl
3.3 Torch Dispatching and Operators
3.4 Torch Gen
4 Automatic Differentiation and Computation Graph
4.1 Building Computation Graphs
4.2 Deriving the Computation Graph by Hand
4.3 Executing The Computation Graph
5 Torch Layers
5.1 Linear
5.2 Sigmoid
5.3 Dropout
5.4 Batch Normalization (BN)
5.5 Convolutional Neural Networks (CNNs)
5.6 MSE Loss
5.7 Embedding
6 Optimization
6.1 Stochastic Gradient Descent
6.2 Optimizer Step
7 Adversarial Attacks
8 Inference Engines
9 Graphic Processing Unit (GPU)
9.0.1 GEMM CUDA Implementation
A Monte Carlo
B Information Theory
C Intrusive Pointers
D Integer Arithmetic
E Floating Point Arithmetic