Llama 3.1 405B & 70B vs MacBook Pro. Apple Silicon is overpowered! Bonus: Apple's OpenELM
The largest model, Llama 3.1 405B, has arrived! Remember the $5k MacBook Pro? We’re about to push it to its limits and see if it can handle the heat from the newly released Llama models. Can Apple’s Silicon take on these AI behemoths? Plus, we’ll give you a sneak peek at Apple’s OpenELM. Buckle up for a fun and fascinating tech showdown!
Don’t forget to like, subscribe, and hit the bell icon for more awesome content!
Hardware Specs: 16‑inch MacBook Pro Apple M3 Max chip with 128GB unified memory
Benchmark: Generation speed for Meta-Llama-3.1-70B-Instruct-4bit is around 8.333-9.49 tokens per second on MLX.
Instructions:
pip install mlx-lm
mlx_lm.server --model mlx-community/Meta-Llama-3.1-70B-Instruct-4bit
The Open Source UI used in the video: https://github.com/EmbeddedLLM/JamAIBase
#llama #llama3 #AppleSilicon #MacBookPro #OpenELM #AI #llm
Chapters
0:00 - Intro
0:26 - What is mlx-lm?
01:55 - Get Llama 3.1 Model !!!
04:53 - Let's Test Llama 3.1
05:55 - Llama 3.1 vs OpenELM
11:55 - Llama 3.1 8B vs Llama 3.1 70B
17:30 - Llama 3.1 405B on Macbook?
21:28 - Ending
Watch video Llama 3.1 405B & 70B vs MacBook Pro. Apple Silicon is overpowered! Bonus: Apple's OpenELM online without registration, duration hours minute second in high quality. This video was added by user Techie Tales 28 July 2024, don't forget to share it with your friends and acquaintances, it has been viewed on our site 27,195 once and liked it 501 people.