Developing an LLM: Building, Training, Finetuning
REFERENCES:
1. Build an LLM from Scratch book: https://mng.bz/M96o
2. Build an LLM from Scratch repo: https://github.com/rasbt/LLMs-from-sc...
3. Slides: https://sebastianraschka.com/pdf/slid...
4. LitGPT: https://github.com/Lightning-AI/litgpt
5. TinyLlama pretraining: https://lightning.ai/lightning-ai/stu...
DESCRIPTION:
This video provides an overview of the three stages of developing an LLM: Building, Training, and Finetuning. The focus is on explaining how LLMs work by describing how each step works.
---
To support this channel, please consider purchasing a copy of my books: https://sebastianraschka.com/books/
---
/ rasbt
/ sebastianraschka
https://magazine.sebastianraschka.com
---
OUTLINE:
00:00 – Using LLMs
02:50 – The stages of developing an LLM
05:26 – The dataset
10:15 – Generating multi-word outputs
12:30 – Tokenization
15:35 – Pretraining datasets
21:53 – LLM architecture
27:20 – Pretraining
35:21 – Classification finetuning
39:48 – Instruction finetuning
43:06 – Preference finetuning
46:04 – Evaluating LLMs
53:59 – Pretraining & finetuning rules of thumb
Watch video Developing an LLM: Building, Training, Finetuning online without registration, duration hours minute second in high quality. This video was added by user Sebastian Raschka 06 June 2024, don't forget to share it with your friends and acquaintances, it has been viewed on our site 59,752 once and liked it 1.9 thousand people.