Share this project

Done

Share this project

Done
$2,268
pledged of $2,573 goal
backers

All or nothing. This project will only be funded if it reaches its goal by .

$2,268
pledged of $2,573 goal
backers

All or nothing. This project will only be funded if it reaches its goal by .

About this project

Reinforcement Learning (RL) is an exciting new field of machine learning, in which bots learn by playing games. The biggest breakthrough was made by Deepmind, who programmed an AI to play Breakout.

This is a sample Bot that just presses random buttons to hit the ball:

It is different from traditional machine learning (supervised or unsupervised) in that there are no training samples with expected outputs.

In RL, the bots are thrown into a computer game (and gaming is a field they are most extensively tested in), and then trained to learn by observing their actions and rewards

This is the most radical thing about RL: We, as programmers, only tell the algorithm what we expect, like finish the game with at least 100 score. The algorithm then randomly tries different combinations, learning as it goes, until it reaches our desired goal (which can take hours or days).

Teaching the computer how to play games

Training a modern game, even something 2D like Pacman or Breakout can take days, even weeks, on high powered GPU machines. To make our task easy, we will learn by toy games.

Some will be text based, others will be simple GUI based.

Games like cartpole, where you have to balance a cartpole on a wire:

Another will be Mountain car, where you have to drive a car up a steel hill:

Stuff we will go over

  • A simple text game to learn the concepts of RL. This will be a made up escape the room type game
  • Q Learning is a popular RL algorithm. We will see how to implement it using our toy games
  • Deep Learning with Neural Networks  
  • An introduction to OpenAI.

Deep Learning with Neural Networks

Q Learning is good, but the problem with it is that it stores everything in an array. Which is fine for simple games with limited states, but as soon as you start going into complex stuff, you can have millions of states, which means you will need a lot of memory.

To fix this problem (and others), Neural Networks were brought in.  They can allow the AI to handle more complex states, but introduce a different level of complexity. We will look at these.

Introduction to OpenAI

Once you understand the basics of RL, you can try it on more advanced games. I'll introduce you to OpenAI.

OpenAI is an new frame work that allows you to train your algorithms on games like Pacman:

And even flash games (using a VNC server and a local browser):

 

Pre-Requisites

You need to know Python, of course, but you also must have some basic knowledge of Neural Networks and Keras. If you don't, my previous course Build Your Neural Network from Scratch can help, and is included in the rewards.

Bonus 1: A Virtual Machine to run the code

Most of this code (especially OpenAI stuff) only works on Linux, and even there, it's a pain to install. I'll provide you with a virtual machine with all the tools installed, so you can just start coding (and playing!)

Bonus 2: Intro to VizDoom

VizDoom is another library that allows you to program bots to play Doom, and this works on Windows too (and Mac, Im told).

I'll give you a quick introduction to VizDoom, so you can understand it's basics.

 

Risks and challenges

I have written 3 books on Kickstarter, so I am confident I can finish this too.

Learn about accountability on Kickstarter

Questions about this project? Check out the FAQ

Support this project

  1. Make a pledge without a reward

    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  2. Select this reward

    Pledge £15 or more About $19

    Get the eBook (Early Bird 1)

    Get the eBook. Also get a Virtual Machine to runn all your code

    Less
    Estimated delivery
    Limited (1 left of 40) 39 backers
    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  3. Select this reward

    Pledge £25 or more About $31

    Get the eBook

    Get the eBook. Also get a Virtual Machine to runn all your code

    Less
    Estimated delivery
    6 backers
    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  4. Select this reward

    Pledge £50 or more About $63

    Get the Neural Network book

    In addition to above, get my previous book on Neural Networks (a pre-requisite to this course)

    Less
    Estimated delivery
    3 backers
    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  5. All gone!
  6. Reward no longer available

    Pledge £20 or more About $25

    Get the eBook (Early Bird 2)

    Get the eBook. Also get a Virtual Machine to runn all your code

    Less
    Estimated delivery
    Reward no longer available 12 backers
    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  7. Reward no longer available

    Pledge £35 or more About $44

    Get the Neural Network book(Early Bird)

    In addition to above, get my previous book on Neural Networks (a pre-requisite to this course)

    Less
    Estimated delivery
    Reward no longer available 18 backers
    £
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.