AdamW Optimizer Explained | L2 Regularization vs Weight Decay
In this video I cover the AdamW optimizer in comparison with the classical Adam. Also, I underline the differences between L2 Regularization and Weight Decay which stays at the core of understanding what new things AdamW brings to the table.
Resources
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
An overview of gradient descent optimization algorithms: https://ruder.io/optimizing-gradient-...
AdamW original paper: https://arxiv.org/abs/1711.05101
Adam original paper: https://arxiv.org/abs/1412.6980
Related Videos
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
AMSGrad Explained: • AMSGrad - Why Adam FAILS to Converge
Why neural networks can learn any function: • Why Neural Networks Can Learn Any Fun...
Why weight regularization reduces overfitting: • Why Weight Regularization Reduces Ove...
Contents
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
00:00 - Intro
00:29 - L2 Regularization vs Weight Decay
00:48 - SGD L2 vs Weight Decay
01:40 - SGD with momentum L2 vs Weight Decay
02:14 - Adam vs AdamW
03:13 - Outro
Follow Me
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
🐦 Twitter: @datamlistic / datamlistic
📸 Instagram: @datamlistic / datamlistic
📱 TikTok: @datamlistic / datamlistic
Channel Support
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
The best way to support the channel is to share the content. ;)
If you'd like to also support the channel financially, donating the price of a coffee is always warmly welcomed! (completely optional and voluntary)
► Patreon: / datamlistic
► Bitcoin (BTC): 3C6Pkzyb5CjAUYrJxmpCaaNPVRgRVxxyTq
► Ethereum (ETH): 0x9Ac4eB94386C3e02b96599C05B7a8C71773c9281
► Cardano (ADA): addr1v95rfxlslfzkvd8sr3exkh7st4qmgj4ywf5zcaxgqgdyunsj5juw5
► Tether (USDT): 0xeC261d9b2EE4B6997a6a424067af165BAA4afE1a
#adamw #adam #l2regularization #weightdecay
Watch video AdamW Optimizer Explained | L2 Regularization vs Weight Decay online without registration, duration hours minute second in high quality. This video was added by user DataMListic 02 January 2023, don't forget to share it with your friends and acquaintances, it has been viewed on our site 9,029 once and liked it 165 people.