The Exponential Family (Part 2)
The machine learning consultancy: https://truetheta.io
Join my email list to get educational and useful articles (and nothing else!): https://mailchi.mp/truetheta/true-the...
Want to work together? See here: https://truetheta.io/about/#want-to-w...
This is part 2 on the Exponential Family where I cover its useful and remarkable properties. This helps explain why distributions within the Family are so frequently utilized and how it could be more generically exploited for sophisticated applications.
SOCIAL MEDIA
LinkedIn : / dj-rich-90b91753
Twitter : / duanejrich
Enjoy learning this way? Want me to make more videos? Consider supporting me on Patreon: / mutualinformation
SOURCES
Chapter 9 of [2] is where I first learned of the Exponential Family. It covers its definition/properties and shows why it's so well adopted for statistics/machine learning. If you're looking to supplement this video with more detail, this is the place to start.
[1] is where I learned how to precisely interpret the components of the Exponential Family and how that maps onto the special cases.
[4] was my primary source for understanding conjugacy of the exponential family. It's where I discovered the specific setting to the exponential family to yield conjugate pairs.
[3] provides an in depth view of the Exponential Family and it's usefulness for statistical modeling. It resolves a lot of ambiguity by discussing the sometimes fuzzy relationship between our language and the notation's precise meaning. It's also where I learned why the mean-parameterization is really what you want to deal with while modeling.
[5] showed me how the Exponential Family is used in more sophisticated applications (specifically, for general graphical models). Also, it's where I discovered some of the more technical/theoretical details of the Exponential Family (e.g. there is a 1-to-1 mapping between the mean and canonical parameters if and only if the Exponential Family choices are minimal).
---------------------------
[1] M. I. Jordan, Exponential Family: Basics, University of California, Berkeley, https://people.eecs.berkeley.edu/~jor...
[2] K. P. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012
[3] C. J. Geyer, "Stat 8054 Lecture Notes: Exponential Families", University of Minnesota Twin Cities, 2020, https://www.stat.umn.edu/geyer/8054/n...
[4] D. M. Blei, "The Exponential Family", Columbia University, 2016, http://www.cs.columbia.edu/~blei/fogm...
[5] M. J. Wainwright, M. I. Jordan, Graphical Models, Exponential Families, and Variational Inference, Foundation and Trends in Machine Learning, 2008
EXTRA NOTES
In the video, I say "*The* Exponential Family" quite a bit, but Geyer thinks that isn't correct. He says (from [3]) : "Many people also use an older terminology that says a statistical model is in the exponential family, where we say a statistical model is an exponential family. Thus the older terminology says the exponential family is the collection of all of what the newer terminology calls exponential families. The older terminology names a useless mathematical object, a heterogeneous collection of statistical models not used in any application. The newer terminology names an important property of statistical models."
Timestamps
0:00 Intro
0:30 Review of the Exponential Family Definition
1:54 Mean and Covariance
5:34 Maximum Likehood Estimation
8:39 Difficulties from Wild Choices
10:41 Conjugacy
16:50 Outro
Watch video The Exponential Family (Part 2) online without registration, duration hours minute second in high quality. This video was added by user Mutual Information 16 September 2021, don't forget to share it with your friends and acquaintances, it has been viewed on our site 8,323 once and liked it 408 people.