Instruct-Tune Llama to Create ChatGPT Like Chatbots | Custom Dataset, Huggingface, SFT
We're going instrcut-tune or SFT TinyLlama so that it'll follow our instructions and respond accordingly. It's the first step to create ChatGPT-like applications or chatbots. GitHub below ↓
I'm going to use TinyLlama, but you can also use Llama 2. You'll probably need to follow a similar procedure with any other LLMs.
Want to support the channel? Hit that like button and subscribe!
GitHub Link of the Code
https://github.com/uygarkurt/SFT-Tiny...
Notebook
https://github.com/uygarkurt/SFT-Tiny...
TinyLlama Repository
https://github.com/jzhang38/TinyLlama
Llama 2 Paper
https://arxiv.org/abs/2307.09288
What should I implement next? Let me know in the comments!
00:00 Introduction
01:57 Imports and Hyperparameter Definitions
12:13 Tokenizer Definition
13:38 Load Dataset From Huggingface
14:45 Llama Prompt Formats / Data Formats Explained
20:20 Alpaca Data Overview
23:12 Create and Load Custom Dataset
25:50 Dataset Preparation
27:44 BitsAndBytes Quantization
28:44 Load TinyLlama Model
29:15 LoRA Definition
30:06 Training Arguments
32:51 Start Llama Training
33:29 Prompting Llama / Getting Results
Buy me a coffee! ☕️
https://ko-fi.com/uygarkurt
Watch video Instruct-Tune Llama to Create ChatGPT Like Chatbots | Custom Dataset, Huggingface, SFT online without registration, duration hours minute second in high quality. This video was added by user Uygar Kurt 14 February 2024, don't forget to share it with your friends and acquaintances, it has been viewed on our site 1,268 once and liked it 50 people.