Stanford CS25: V2 I Introduction to Transformers w/ Andrej Karpathy
January 10, 2023
Introduction to Transformers
Andrej Karpathy: https://karpathy.ai/
Since their introduction in 2017, transformers have revolutionized Natural Language Processing (NLP). Now, transformers are finding applications all over Deep Learning, be it computer vision (CV), reinforcement learning (RL), Generative Adversarial Networks (GANs), Speech or even Biology. Among other things, transformers have enabled the creation of powerful language models like GPT-3 and were instrumental in DeepMind's recent AlphaFold2, that tackles protein folding.
In this speaker series, we examine the details of how transformers work, and dive deep into the different kinds of transformers and how they're applied in different fields. We do this by inviting people at the forefront of transformers research across different domains for guest lectures.
More about the course can be found here: https://web.stanford.edu/class/cs25/
View the entire CS25 Transformers United playlist: • Stanford CS25 - Transformers United
0:00 Introduction
0:47 Introducing the Course
3:19 Basics of Transformers
3:35 The Attention Timeline
5:01 Prehistoric Era
6:10 Where we were in 2021
7:30 The Future
10:15 Transformers - Andrej Karpathy
10:39 Historical context
1:00:30 Thank you - Go forth and transform
Watch video Stanford CS25: V2 I Introduction to Transformers w/ Andrej Karpathy online without registration, duration hours minute second in high quality. This video was added by user Stanford Online 19 May 2023, don't forget to share it with your friends and acquaintances, it has been viewed on our site 774,386 once and liked it 14 thousand people.