How did the Attention Mechanism start an AI frenzy? | LM3
The attention mechanism is well known for its use in Transformers. But where does it come from? It's origins lie in fixing a strange problems of RNNs.
Support me on Patreon! / vcubingx
Language Modeling Playlist: • Language Modeling
3blue1brown series on Transformers: • Transformers (how LLMs work) explaine...
The source code for the animations can be found here:
https://github.com/vivek3141/dl-visua...
These animation in this video was made using 3blue1brown's library, manim:
https://github.com/3b1b/manim
Sources (includes the entire series): https://docs.google.com/document/d/1e...
Chapters
0:00 Introduction
0:22 Machine Translation
2:01 Attention Mechanism
8:04 Outro
Music (In Order):
Helynt - Route 10
Helynt - Bo-Omb Battlefield
Helynt - Underwater
Philanthrope, mommy - embrace https://chll.to/7e941f72
Helynt - Twinleaf Town
Follow me!
Website: https://vcubingx.com
Twitter: / vcubingx
Github: https://github.com/vivek3141
Instagram: / vcubingx
Patreon: / vcubingx
Watch video How did the Attention Mechanism start an AI frenzy? | LM3 online without registration, duration hours minute second in high quality. This video was added by user vcubingx 15 April 2024, don't forget to share it with your friends and acquaintances, it has been viewed on our site 19,485 once and liked it 668 people.