Compressing Large Language Models (LLMs) | w/ Python Code
🗞️ Get exclusive access to AI resources and project ideas: https://the-data-entrepreneurs.kit.co...
🧑🎓 Learn AI in 6 weeks by building it: https://maven.com/shaw-talebi/ai-buil...
--
Here, I discuss 3 ways to do model compression on LLMs (i.e. Quantization, Pruning, and Knowledge Distillation/Model Distillation) with example Python code.
Resources:
📰 Blog: https://medium.com/towards-data-scien...
🎥 Training the Teacher Model: • Fine-Tuning BERT for Text Classificat...
💻 GitHub Repo: https://github.com/ShawhinT/YouTube-B...
👩🏫 Teacher Model: https://huggingface.co/shawhin/bert-p...
🧑🎓 Student Model: https://huggingface.co/shawhin/bert-p...
👾 4-bit Student Model: https://huggingface.co/shawhin/bert-p...
💿 Dataset: https://huggingface.co/datasets/shawh...
References:
[1] https://arxiv.org/abs/2001.08361
[2] https://arxiv.org/abs/1710.09282
[3] https://machinelearning.apple.com/res...
[4] https://arxiv.org/abs/1710.09282
[5] https://arxiv.org/abs/2308.07633
[6] https://arxiv.org/abs/2402.17764
[7] https://arxiv.org/abs/1710.01878
[8] https://arxiv.org/abs/1503.02531
[9] https://crfm.stanford.edu/2023/03/13/...
[10] https://arxiv.org/abs/2305.14314
[11] https://www.researchgate.net/publicat...
--
Homepage: https://www.shawhintalebi.com
Intro - 0:00
"Bigger is Better" - 0:40
The Problem - 1:35
Model Compression - 2:14
1) Quantization - 3:11
2) Pruning - 5:44
3) Knowledge Distillation - 8:04
Example: Compressing a model with KD + Quantization - 11:10
Смотрите видео Compressing Large Language Models (LLMs) | w/ Python Code онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь Shaw Talebi 31 Август 2024, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 7,490 раз и оно понравилось 320 людям.