RL. part 0.Absolute minimum amount of math, necessary for studying Reinforcement Learning.

     Math is an absolute must have for anyone trying to learn Reinforcement Learning techniques. Writing any king of RL program requires precise understanding of the algorithms and underlying math. It will make your life easier, otherwise things will not work and the agent will not learn the way you expect it to and there will a lot of hair pulling and head clearing walks just to realize that you should have read that paper more closely and tried to see why authors decided to put so many equation in.

Uhhh, math again…  In a lot of careers, especially in technical sector; it is assumed to have life long learning habit. Also it is important to be proactive and know where the industry is heading and what skills are needed in the future to stay relevant in a job market. The feeling of comfortable employment is an evil thing.

I still remember while working on my Bachelor degree the moment I walked out after a final on Discrete mathematics and realized there will be no more of this “nonsense”(I blame a professor, of course). The rest of the program was practical, and writing code was my Jam.  How wrong and naive that assumption was. Few years later I had to prove every commonly known algorithm for Analysis of Algorithm class in grad school using mostly dreadful discrete math. 
   

    So I’ve compiled a list of videos containing professionally explained essential math terms and concepts. I personally find it much easier to watch a video than to read an article or a wiki page. It might take you a considerable amount of time to go through each supplied clip, but it will pay off in a long run. 

                                   Probability and statistics

    First things first, we need to understand what Random Variables and Expected Values are. I personally couldn’t find a better explanation than this video. Try binge watching the whole playlist the way you did with the Game of Thrones. I dare you. 

    It is also helpful to understand bizarre Bayes’ Theorem and Chain Rule.

Next on the list is a Standard Deviation. Once again beautifully explained by the Khan Academy.

Importance Sampling concludes our list on Probability and Statistics.  It might be too much to take in without knowing how to apply it while working on RL problem. But it is a part of the learning process. It will get clearer (I hope) as we get to actual algorithms.

                                        Multivariate Calculus

Don’t you love multivar calc? There is only one term from everyone’s favorite subject that is important to understand – The Gradient. 

    If you’ve gone through all of the videos and find all presented material easy to understand, you are golden and ready to move forward to actual RL subject. Otherwise you might want to learn some more before you move on. Anyways. I know this post looks like I’ve put random youtube videos together, in fact it took considerable amount of time to find out what kind of math prerequisites are absolutely essential to start diving in RL field. If you have anything to add to the list, please post it in comments and I’ll update the post.    

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.