PyTorch is a general-purpose deep learning framework used to train deep learning models, including LLMs. Because it was designed for flexibility (e.g we already covered with dynamic computation graphs), large-scale inference has different requirements, such as serving production traffic with high throughput, low latency, and efficient memory management. This is why specialized inference engines exist. In this section, we will examine different serving engines, including llama.cpp, TensorRT-LLM and vLLM, and develop a deep understanding of how they optimize LLM inference.