Machine Learning: What is Reinforcement Learning?

In this article we will go through a basic introduction to a sub field of Machine Learning know as reinforcement learning.

Reinforcement learning is a branch of Artificial Intelligence in which the model learns by continuous interactions with its environments. Let's say I want to get a cookie from a jar that's on a tall shelf. There isn't one right way to get the cookies. Maybe I will find a ladder or build a complicated system of pulleys. These could all be brilliant or terrible ideas but if something works I get the sweet taste of victory and I learned that doing the same thing could get me another cookie in the future. We learn lots of things by trial and error and this kind of learning is called reinforcement learning. As a basic general principle reinforcements are provided to the model based on the impact of a particular action in relation to the target. If the action made by the model is takes it closer to the set target, a positive reinforcement is provided, and a negative reinforcement otherwise .Reinforcement learning is a type of machine learning where an agent learns to make decisions by interacting with an environment to achieve a specific goal. It learns through trial and error by receiving feedback(reinforcements) in the form of rewards(positive reinforcement) or penalties(negative reinforcement) based on its actions. The key to reinforce learning is just trial and error over and over again. For humans a reward might be a cookie or the joy of winning a board game. But for an AI system a reward is just a small positive signal that basically tells it, good job and do that again. The fundamental components of reinforcement learning are:

1. Agent: This is our model, the learner or decision-maker that interacts with the environment in order to learn from it and takes actions based on a policy to maximize cumulative rewards.

2. Environment: The external system with which the agent interacts. It responds to the actions of the agent and provides feedback in the form of rewards or penalties.

3. Actions: This represents the choices available to the agent at each step of interaction with the environment.

4. Rewards: Numeric signals provided by the environment to indicate the desirability of the agent's actions. Positive rewards encourage the agent to repeat similar actions, while negative rewards or penalties discourage undesired behaviour.

We don't pause to think after every action the agent ends up interacting with its environment for a while whether that's a game board, a virtual maze or real life kitchen and the agent takes many actions until it gets a reward which we give out. When it wins a game or gets that cookie jar from that really tall shelf then every time the agent wins or succeeds at his task we look back on the actions it took and figure out what game states were helpful and which ones weren't. During this reflection we're assigning value to those different game states and deciding on a policy for which actions work best. We need values and policies to get anything done in reinforcement learning.

Basic Steps in Reinforcement Learning

An agent makes predictions or performs actions like moving a tiny bit forward or picking the next best move in a game and it performs actions based on its current inputs which we call the states. In supervised learning, after each action we would have a training label that tells our AI whether it did the right thing or not. We can't do that with reinforcement learning because we don't know what the right thing actually is until it's completely done with the task. This difference actually highlights one of the hardest parts of reinforcement learning called credit assignment. It's hard to know which actions helped us get the reward and should get the credit and which action slowed down our AI. The reinforcement learning process involves:

1. Exploration and Exploitation: The agent explores the environment initially to understand the consequences of its actions and gradually exploits this knowledge to maximize rewards.

2. Learning from Feedback: Through continuous interaction, the agent learns to associate actions with outcomes by adjusting its policy to optimize long-term rewards.

3. Temporal Credit Assignment: The agent attributes credit (or blame) to its actions concerning the obtained rewards, taking into account delayed consequences.

Reinforcement Learning Industry uses

Google’S Deep Mind got some pretty impressive results when they used reinforcement learning to teach virtual AI systems to walk, jump and even duck under obstacles. It looks kind of silly but works pretty well. Reinforcement learning finds applications in various domains:

• Game Playing: Teaching agents to play games like chess, Go, or video games.

• Robotics: Training robots to perform tasks by interacting with the physical world.

• Autonomous Vehicles: Enabling vehicles to learn driving strategies in simulated environments.

• Recommendation Systems: Optimizing recommendations based on user interactions.

Popular reinforcement learning algorithms include Q-Learning, Deep Q Networks (DQN), Policy Gradient methods, and Actor-Critic methods. These algorithms, coupled with the exploration-exploitation trade-off, enable agents to learn and make decisions in complex and dynamic environments.

List of Reinforcement Learning Algorithms

In Machine Learning Reinforcement Learning (RL)  falls somewhere between supervised and unsupervised. It is not supervised learning since it doesn't absolutely rely on a set of labeled training data. At the same time it can not classified as unsupervised learning since we're looking for our reinforcement learning agent in order to maximise a reward. For the agent to attain its main goal, it must determine the correct set of actions to be taken in different scenarios. Following are the list of various reinforcement learning techniques:

1. Markov decision process (MDP)

2. Bellman equation

3. Dynamic programming

4. Value iteration

5. Policy iteration

6. Q-learning.