Umar Jamil

Wav2Lip (generate talking avatar videos) - Paper reading and explanation

Variational Autoencoder - Model, ELBO, loss function and maths explained easily!

Attention is all you need (Transformer) - Model explanation (including math), Inference and Training

Distributed Training with PyTorch: complete tutorial with cloud infrastructure and code

Reinforcement Learning from Human Feedback explained with math derivations and the PyTorch code.

LLaMA explained: KV-Cache, Rotary Positional Embedding, RMS Norm, Grouped Query Attention, SwiGLU

Retrieval Augmented Generation (RAG) Explained: Embedding, Sentence BERT, Vector Database (HNSW)

Coding Stable Diffusion from scratch in PyTorch

Coding a Multimodal (Vision) Language Model from scratch in PyTorch with full explanation

LoRA: Low-Rank Adaptation of Large Language Models - Explained visually + PyTorch code from scratch

ML Interpretability: feature visualization, adversarial example, interp. for language models

Mistral / Mixtral Explained: Sliding Window Attention, Sparse Mixture of Experts, Rolling Buffer

Segment Anything - Model explanation with code

Coding a Transformer from scratch on PyTorch, with full explanation, training and inference.

CLIP - Paper explanation (training and inference)

Coding LLaMA 2 from scratch in PyTorch - KV Cache, Grouped Query Attention, Rotary PE, RMSNorm

BERT explained: Training, Inference, BERT vs GPT/LLamA, Fine tuning, [CLS] token

How diffusion models work - explanation and code!

Direct Preference Optimization (DPO) explained: Bradley-Terry model, log probabilities, math

Quantization explained with PyTorch - Post-Training Quantization, Quantization-Aware Training

Mamba and S4 Explained: Architecture, Parallel Scan, Kernel Fusion, Recurrent, Convolution, Math

Wav2Lip (generate talking avatar videos) - Paper reading and explanation

LongNet: Scaling Transformers to 1,000,000,000 tokens: Python Code + Explanation

Kolmogorov-Arnold Networks: MLP vs KAN, Math, B-Splines, Universal Approximation Theorem