LESS VRAM, 8K+ Tokens & HUGE SPEED INCRASE | ExLlama for Oobabooga
Oobabooga WebUI had a HUGE update adding ExLlama and ExLlama_HF model loaders that use LESS VRAM and have HUGE speed increases, and even 8K tokens to play around with compared to the previous limit of 2K! This is insanely powerful and will be a huge timesaver for creators, and may even help users with less powerful graphics cards use LLMs!
OpenAI Tokenizer: https://platform.openai.com/tokenizer
Timestamps:
0:00 - What's new (It's CRAZY!)
0:44 - Open Oobabooga install directory
1:02 - Update Oobabooga WebUI
1:18 - VRAM usage & speed before update (4.3 tokens/s)
1:56 - Fix missing option or update errors
2:33 - Choosing new ExLlama model locader
2:52 - Downloading new model types (8k models)
4:25 - New VRAM & Speed (20 tokens/s! INSANE!)
5:25 - Raise token limit from 2,000 to 8,000+!
7:17 - How many tokens is your text?
7:50 - How long is 8k tokens?
8:45 - EVEN LESS VRAM with ExLlama_HF
#Oobabooga #AI #LLM
-----------------------------
💸 Found this useful? Help me make more! Support me by becoming a member: / @troublechute
-----------------------------
💸 Support me on Patreon: / troublechute
💸 Direct donations via Ko-Fi: https://ko-fi.com/TCNOco
💬 Discuss the video & Suggest (Discord): https://s.tcno.co/Discord
👉 Game guides & Simple tips: / troublechutebasics
🌐 Website: https://tcno.co
📧 Need voiceovers done? Business query? Contact my business email: TroubleChute (at) tcno.co
-----------------------------
🎨 My Themes & Windows Skins: https://hub.tcno.co/faq/my-windows/
👨💻 Software I use: https://hub.tcno.co/faq/my-software/
➡️ My Setup: https://hub.tcno.co/faq/my-hardware/
🖥️ My Current Hardware (Links here are affiliate links. If you click one, I'll receive a small commission at no extra cost to you):
Intel i9-13900k - https://amzn.to/42xQuI1
GIGABYTE Z790 AORUS Master - https://amzn.to/3nHuBHx
G.Skill RipJaws 2x(2x32G) [128GB] - https://amzn.to/42cilxN
Corsair H150i 360mm AIO - https://amzn.to/42cznvP
MSI 3080Ti Gaming X Trio - https://amzn.to/3pdnLdb
Corsair 1000W RM1000i - https://amzn.to/42gOTGY
Corsair MP600 PRO XT 2TB - https://amzn.to/3NSvwzx
🎙️ My Current Mic/Recording Gear:
Shure SM7B - https://amzn.to/3nDGYo1
Audient iD14 - https://amzn.to/3pgf2XK
dbx 286s - https://amzn.to/3VNaq7O
Triton Audio FetHead - https://amzn.to/3pdjIgZ
Everything in this video is my personal opinion and experience and should not be considered professional advice. Always do your own research and ensure what you're doing is safe.
Watch video LESS VRAM, 8K+ Tokens & HUGE SPEED INCRASE | ExLlama for Oobabooga online without registration, duration hours minute second in high quality. This video was added by user TroubleChute 28 June 2023, don't forget to share it with your friends and acquaintances, it has been viewed on our site 28,174 once and liked it 522 people.