cs294 note1

Posted by ZhY on September 1, 2018

Supervised Learning of Behaviors

Examples for reinforcement learning

These pictures shows that there are a lot of problems could solved by reinforcement learning, such as training a dog, running a robot, or solving inventory management.

Why we use Deep RL

Deep models are what allow reinforcement learning algorithms to solve complex problems end to end.

Reinforcement learning provides us with a conceptual framework for thinking about how to solve the system making problems, and deep learinig provides us with the models that can represent solutions to very complex problems. So deepRl could solve extremly complex decision problems.

Deep RL examples

Robotics example

First of all, the robot receive some of the observations, which can be pixels in an image. It need to be understand what’s in it so it’s going to do state estimation. Then the robot need to detect things so it can predict the action. And it plans what to do next then provides inputs into low-level control commands to get robot perform.

Playing video games

This pipline is similar to the robotics control pipline.


Because every stage of the representation introduces some abstraction it loses some kind of information. So we should make sure that the abstraction is the right one.

When we don’t need RL

e.g. classification, regression

When we need RL

Why now

L.-J.Lin,"Reinforcement learning for robots using neural networks." 1993

Some other forms of supervision

Inverse RL

Behavior goes in and the robot analyze this behavior to figure out what is the goal of that behavior

Prediction is a central component of what brain actually do

Evidence for deep learning

the features deep learning extracted is similar to the statistics features.

Evidence for reinforcement learning

What can we do now