How did the Attention Mechanism start an AI frenzy? | LM3

Опубликовано: 15 Апрель 2024
на канале: vcubingx

19,485

668

The attention mechanism is well known for its use in Transformers. But where does it come from? It's origins lie in fixing a strange problems of RNNs.
Support me on Patreon!   / vcubingx
Language Modeling Playlist:    • Language Modeling

3blue1brown series on Transformers:    • Transformers (how LLMs work) explaine...

The source code for the animations can be found here:
https://github.com/vivek3141/dl-visua...

These animation in this video was made using 3blue1brown's library, manim:
https://github.com/3b1b/manim

Sources (includes the entire series): https://docs.google.com/document/d/1e...
Chapters
0:00 Introduction
0:22 Machine Translation
2:01 Attention Mechanism
8:04 Outro

Music (In Order):
Helynt - Route 10
Helynt - Bo-Omb Battlefield
Helynt - Underwater
Philanthrope, mommy - embrace https://chll.to/7e941f72
Helynt - Twinleaf Town

Follow me!
Website: https://vcubingx.com
Twitter:   / vcubingx
Github: https://github.com/vivek3141
Instagram:   / vcubingx
Patreon:   / vcubingx

Смотрите видео How did the Attention Mechanism start an AI frenzy? | LM3 онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь vcubingx 15 Апрель 2024, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 19,485 раз и оно понравилось 668 людям.

1,373