The Narrated Transformer Language Model
AI/ML has been witnessing a rapid acceleration in model improvement in the last few years. The majority of the state-of-the-art models in the field are based on the Transformer architecture. Examples include models like BERT (which when applied to Google Search, resulted in what Google calls "one of the biggest leaps forward in the history of Search") and OpenAI's GPT2 and GPT3 (which are able to generate coherent text and essays).
This video by the author of the popular "Illustrated Transformer" guide will introduce the Transformer architecture and its various applications. This is a visual presentation accessible to people with various levels of ML experience.
Intro (0:00)
The Architecture of the Transformer (4:18)
Model Training (7:11)
Transformer LM Component 1: FFNN (10:01)
Transformer LM Component 2: Self-Attention(12:27)
Tokenization: Words to Token Ids (14:59)
Embedding: Breathe meaning into tokens (19:42)
Projecting the Output: Turning Computation into Language (24:11)
Final Note: Visualizing Probabilities (25:51)
The Illustrated Transformer:
https://jalammar.github.io/illustrate...
Simple transformer language model notebook:
https://github.com/jalammar/jalammar....
Philosophers On GPT-3 (updated with replies by GPT-3):
https://dailynous.com/2020/07/30/phil...
-----
Twitter: / jayalammar
Blog: https://jalammar.github.io/
Mailing List: https://jayalammar.substack.com/
More videos by Jay:
Jay's Visual Intro to AI
• Jay's Visual Intro to AI
How GPT-3 Works - Easily Explained with Animations
• How GPT3 Works - Easily Explained wit...
Смотрите видео The Narrated Transformer Language Model онлайн без регистрации, длительностью часов минут секунд в хорошем качестве. Это видео добавил пользователь Jay Alammar 26 Октябрь 2020, не забудьте поделиться им ссылкой с друзьями и знакомыми, на нашем сайте его посмотрели 327,576 раз и оно понравилось 5.6 тысяч людям.