Reinforcement Learning (RL) is an exciting new field of machine learning, in which bots learn by playing games. The biggest breakthrough was made by Deepmind, who programmed an AI to play Breakout.
This is a sample Bot that just presses random buttons to hit the ball:
It is different from traditional machine learning (supervised or unsupervised) in that there are no training samples with expected outputs.
In RL, the bots are thrown into a computer game (and gaming is a field they are most extensively tested in), and then trained to learn by observing their actions and rewards.
This is the most radical thing about RL: We, as programmers, only tell the algorithm what we expect, like finish the game with at least 100 score. The algorithm then randomly tries different combinations, learning as it goes, until it reaches our desired goal (which can take hours or days).
Teaching the computer how to play games
Training a modern game, even something 2D like Pacman or Breakout can take days, even weeks, on high powered GPU machines. To make our task easy, we will learn by toy games.
Some will be text based, others will be simple GUI based.
Games like cartpole, where you have to balance a cartpole on a wire:
Another will be Mountain car, where you have to drive a car up a steel hill:
Stuff we will go over
- A simple text game to learn the concepts of RL. This will be a made up escape the room type game
- Q Learning is a popular RL algorithm. We will see how to implement it using our toy games
- Deep Learning with Neural Networks
- An introduction to OpenAI.
Deep Learning with Neural Networks
Q Learning is good, but the problem with it is that it stores everything in an array. Which is fine for simple games with limited states, but as soon as you start going into complex stuff, you can have millions of states, which means you will need a lot of memory.
To fix this problem (and others), Neural Networks were brought in. They can allow the AI to handle more complex states, but introduce a different level of complexity. We will look at these.
Introduction to OpenAI
Once you understand the basics of RL, you can try it on more advanced games. I'll introduce you to OpenAI.
OpenAI is an new frame work that allows you to train your algorithms on games like Pacman:
And even flash games (using a VNC server and a local browser):
You need to know Python, of course, but you also must have some basic knowledge of Neural Networks and Keras. If you don't, my previous course Build Your Neural Network from Scratch can help, and is included in the rewards.
Bonus 1: A Virtual Machine to run the code
Most of this code (especially OpenAI stuff) only works on Linux, and even there, it's a pain to install. I'll provide you with a virtual machine with all the tools installed, so you can just start coding (and playing!)
Bonus 2: Intro to VizDoom
VizDoom is another library that allows you to program bots to play Doom, and this works on Windows too (and Mac, Im told).
I'll give you a quick introduction to VizDoom, so you can understand it's basics.
Risks and challenges
I have written 3 books on Kickstarter, so I am confident I can finish this too.Learn about accountability on Kickstarter
- (30 days)