Large Language Models (LLMs) often feel like magic black boxes. We use them via APIs, but do we truly understand what happens inside? In this hands-on tutorial, we will demystify modern LLMs by building a functional clone of Google’s Gemma model from scratch using PyTorch.

Moving beyond basic transformer theory, we will implement the specific architectural components that power today’s state-of-the-art models. You will write the code for Rotary Positional Embeddings (RoPE), RMSNorm, Grouped-Query Attention (GQA), and SwiGLU activation functions.

By the end of this session, you will have a working model architecture that can load official pre-trained weights and generate text. This tutorial is designed for Python developers and data scientists who want to bridge the gap between high-level APIs and the low-level mechanics of deep learning. No magic, just code.