Understanding Deep Learning Systems With PyTorch (Draft)

Mouad Kondah

September 2026

Contents
1 Introduction
2 Deep Learning
2.1 Linear Classification
2.2 Deep Learning
2.3 Discriminative and Generative Modeling
2.4 The Variational Autoencoder (VAE) framework
2.5 The Generative Adversarial Network (GAN) Framework
2.6 Autoregressive and Language Models
3 Tensors
3.1 Mapping High-Dimensional Arrays to Memory
3.2 Tensor, TensorImpl, and StorageImpl
3.3 Torch Dispatching and Operators
3.4 Torch Gen
4 Automatic Differentiation and Computation Graph
4.1 Building Computation Graphs
4.2 Deriving the Computation Graph by Hand
4.3 Executing The Computation Graph
5 Torch Layers
5.1 Linear
5.2 Sigmoid
5.3 Dropout
5.4 Batch Normalization (BN)
5.5 Convolutional Neural Networks (CNNs)
5.6 MSE Loss
5.7 Embedding
6 Optimization
6.1 Stochastic Gradient Descent
6.2 Optimizer Step
7 Adversarial Attacks
8 Inference Engines
9 Graphic Processing Unit (GPU)
A Monte Carlo
B Information Theory
C Intrusive Pointers
D Integer Arithmetic
E Floating Point Arithmetic
References