After the explosive growth of open source machine learning and deep learning frameworks, the field is more accessible than ever. Thanks to this, it went from a tool for researchers to a widely adopted and used method, fueling the insane growth of technology we experience now. Understanding how the algorithms really work can give you a huge advantage in designing, developing and debugging machine learning systems. Due to its mathematical nature, this task can seem daunting for many. However, this does not have to be the way.
From a high level, there are four pillars of mathematics in machine learning.
- Linear algebra
- Probability theory
- Multivariate calculus
- Optimization theory
It takes time to build a solid foundation of these and understand the inner workings of the state of the art machine learning algorithms such as convolutional networks, generative adversarial networks, and many others. This won’t be an afternoon project, but given that you consistently dedicate time for this, you can go pretty far in a short amount of time. There are some great resources to guide you along the way. In this post, I have selected the five which were most helpful for me.
Linear Algebra Done Right by Sheldon Axler
Linear algebra is a beautiful but tough subject for beginners if it is taught the “classical” way, which is determinants and matrices first, vector spaces later. However, when it is done the other way around, it is surprisingly intuitive and clear. This book presents linear algebra in a very friendly and insightful way. I wish I had learned it from this book, instead of the old way.
You can find the author’s page about the book here.
Probability: For the Enthusiastic Beginner by David Morin
Most machine learning books don’t introduce probability theory properly and they use confusing notation, often mixing up density functions and discrete distributions. This can be very difficult to get through without a solid background in probability.
This book will provide you with just that: a detailed, mathematically correct yet user friendly introduction to the subject. This is suitable for learners without any previous exposure on probability.
If you want to learn what probability really is, I wrote a an introduction to probability from a more abstract perspective.
The mathematical foundations of probability
A measure-theoretic introduction
towardsdatascience.com
Multivariate Calculus by Denis Auroux (from MIT OpenCourseWare)
I have cheated a little bit here, since this is not a book but an actual university course on multivariate calculus at MIT, recorded and made available for the public. Out of all the resources I know, this is by far the best introduction to the subject. It doesn’t hurt to have a background in univariate calculus, but the lectures can be followed without it as well.
You can find the full course here.
One thing this course doesn’t cover well is the gradient descent algorithm, which is fundamental for neural networks. If you would like to learn more about this, I wrote an introductory post on the subject, which explains gradient descent from scratch.
The mathematics of optimization for deep learning
A brief guide about how to minimize a function with millions of variables
towardsdatascience.com
Grokking Deep Learning by Andrew Trask…
Pinterest faces EU privacy complaint over tracking ads
When it comes to privacy nightmares, Pinterest is unlikely to be the first social app that…
Schrems II Judgment Day
The Court of Justice of the European Union (“CJEU“) issued its judgment today in …
