Monte Carlo And Off-Policy Methods | Reinforcement Learning Part 3
The machine learning consultancy: https://truetheta.io
Join my email list to get educational and useful articles (and nothing else!): https://mailchi.mp/truetheta/true-the...
Want to work together? See here: https://truetheta.io/about/#want-to-w...
Part three of a six part series on Reinforcement Learning. It covers the Monte Carlo approach a Markov Decision Process with mere samples. At the end, we touch on off-policy methods, which enable RL when the data was generate with a different agent.
SOCIAL MEDIA
LinkedIn : / dj-rich-90b91753
Twitter : / duanejrich
Github: https://github.com/Duane321
Enjoy learning this way? Want me to make more videos? Consider supporting me on Patreon: / mutualinformation
SOURCES
[1] R. Sutton and A. Barto. Reinforcement learning: An Introduction (2nd Ed). MIT Press, 2018.
[2] H. Hasselt, et al. RL Lecture Series, Deepmind and UCL, 2021, • DeepMind x UCL | Deep Learning Lectur...
SOURCE NOTES
The video covers topics from chapters 5 and 7 from [1]. The whole series teaches from [1]. [2] has been a useful secondary resource.
TIMESTAMP
0:00 What We'll Learn
0:33 Review of Previous Topics
2:50 Monte Carlo Methods
3:35 Model-Free vs Model-Based Methods
4:59 Monte Carlo Evaluation
9:30 MC Evaluation Example
11:48 MC Control
13:01 The Exploration-Exploitation Trade-Off
15:01 The Rules of Blackjack and its MDP
16:55 Constant-alpha MC Applied to Blackjack
21:55 Off-Policy Methods
24:32 Off-Policy Blackjack
26:43 Watch the next video!
NOTES
Link to Constant-alpha MC applied to Blackjack: https://github.com/Duane321/mutual_in...
The Off-Policy method you see at 25:00 is different from the rule you'll see in the textbook at eq 7.9 (which will be MC if n goes to inf). That's because they are showing re-weighted IS and I'm showing plain ( high variance) IS.
Watch video Monte Carlo And Off-Policy Methods | Reinforcement Learning Part 3 online without registration, duration hours minute second in high quality. This video was added by user Mutual Information 26 October 2022, don't forget to share it with your friends and acquaintances, it has been viewed on our site 61,821 once and liked it 1.4 thousand people.