Llama 3.1 405B & 70B vs MacBook Pro. Apple Silicon is overpowered! Bonus: Apple's OpenELM
The largest model, Llama 3.1 405B, has arrived! Remember the $5k MacBook Pro? We’re about to push it to its limits and see if it can handle the heat from the newly released Llama models. Can Apple’s Silicon take on these AI behemoths? Plus, we’ll give you a sneak peek at Apple’s OpenELM. Buckle up for a fun and fascinating tech showdown!
Don’t forget to like, subscribe, and hit the bell icon for more awesome content!
Hardware Specs: 16‑inch MacBook Pro Apple M3 Max chip with 128GB unified memory
Benchmark: Generation speed for Meta-Llama-3.1-70B-Instruct-4bit is around 8.333-9.49 tokens per second on MLX.
Instructions:
pip install mlx-lm
mlx_lm.server --model mlx-community/Meta-Llama-3.1-70B-Instruct-4bit
The Open Source UI used in the video: https://github.com/EmbeddedLLM/JamAIBase
#llama #llama3 #AppleSilicon #MacBookPro #OpenELM #AI #llm
Chapters
0:00 - Intro
0:26 - What is mlx-lm?
01:55 - Get Llama 3.1 Model !!!
04:53 - Let's Test Llama 3.1
05:55 - Llama 3.1 vs OpenELM
11:55 - Llama 3.1 8B vs Llama 3.1 70B
17:30 - Llama 3.1 405B on Macbook?
21:28 - Ending
Смотрите видео Llama 3.1 405B & 70B vs MacBook Pro. Apple Silicon is overpowered! Bonus: Apple's OpenELM онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь Techie Tales 28 Июль 2024, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 27,195 раз и оно понравилось 501 людям.