What Reinforcement Learning Is and Why It Matters in 2026

Reinforcement learning (RL) is the branch of machine learning concerned with learning to act. Instead of learning from a fixed dataset of labelled examples, an RL agent learns by interacting with an environment, taking actions, observing the consequences, and gradually improving its behaviour to maximise a numerical reward. It is the framework behind game-playing systems that beat world champions, robots that learn to walk, recommendation systems that optimise long-term engagement, and — most importantly for this course — the alignment of the large language models you use every day.

Educational note: This course is for learning. Any preference data, human feedback, or training corpus you use in practice must respect data-protection law (GDPR in the EU), dataset licences, and copyright. Alignment work carries real ethical weight — you are shaping how a system behaves toward people — so we return to legal and safety obligations throughout the course.

Supervised, unsupervised, and reinforcement learning

It helps to place RL against the other two paradigms. In supervised learning you have inputs paired with correct outputs, and the model learns to reproduce the mapping. In unsupervised learning you have inputs with no labels, and the model finds structure. Reinforcement learning is different in three fundamental ways:

1. There is no supervisor, only a reward signal. Nobody tells the agent the correct action. It only receives a scalar reward that says how good the outcome was, and it must figure out which actions led to that reward.
2. Feedback can be delayed. An action taken now may only pay off many steps later. Winning a chess game depends on moves made long before the final position. This is the credit assignment problem.
3. Data is not independent and identically distributed. The agent's own choices determine what it sees next. Explore poorly and you never even observe the states where the good rewards live.

These properties make RL powerful and also notoriously difficult. Much of this course is about the algorithms and tricks that make it work anyway.

The agent-environment loop

At the heart of RL is a simple loop. At each time step t the agent observes a state s_t, chooses an action a_t, and the environment responds with a reward r_{t+1} and a new state s_{t+1}. The agent's goal is to choose actions that maximise the cumulative reward over time, not just the immediate one.

# The canonical reinforcement learning loop, expressed against the
# Gymnasium API (the maintained successor to OpenAI Gym).
import gymnasium as gym

env = gym.make("CartPole-v1")
observation, info = env.reset(seed=42)

total_reward = 0.0
for _ in range(1000):
    action = env.action_space.sample()          # a random policy, for now
    observation, reward, terminated, truncated, info = env.step(action)
    total_reward += reward
    if terminated or truncated:
        observation, info = env.reset()
        print("Episode finished, resetting")

env.close()
print("Total reward collected:", total_reward)

This snippet already contains the entire vocabulary of RL: an environment, observations (states), an action space, a reward, and the notions of terminated (the task ended, e.g. the pole fell) and truncated (a time limit was hit). Everything else in this course is about replacing env.action_space.sample() — a random policy — with something that learns.

Reward is the only objective

A defining principle of RL is the reward hypothesis: any goal can be framed as the maximisation of expected cumulative reward. This is deceptively powerful. You do not program how to solve the task; you specify what success looks like via the reward, and learning discovers the how. But it is also the source of the field's deepest hazard. If your reward does not capture exactly what you want, the agent will happily maximise the reward you wrote while doing something you never intended — a failure mode called reward hacking that becomes central when we align language models later in the course.

Why RL matters right now

For a while, RL was seen as a niche of games and robotics. That changed dramatically. The modern conversational assistants — the systems behind ChatGPT, Claude Opus 4.8, Claude Sonnet 5, GPT-5.5 and Gemini 3.1 Pro — are not merely trained to predict the next token. After that pretraining, they are aligned using human (and increasingly AI) feedback, and the mathematical engine of that alignment is reinforcement learning. Reinforcement Learning from Human Feedback (RLHF), and its newer relatives such as Direct Preference Optimization, are the reason these models are helpful, follow instructions, and refuse harmful requests instead of simply continuing text. Understanding RL is now a prerequisite for understanding how state-of-the-art AI is actually built.

What this course covers

We proceed in two arcs. The first builds classical and deep RL from the ground up: Markov decision processes, value functions and the Bellman equations, exploration, Q-learning and Deep Q-Networks, policy gradients, actor-critic methods, and PPO — the workhorse algorithm. The second arc applies this machinery to language models: reward modelling from human preferences, PPO-based RLHF as used to train assistants, and the simpler modern alternatives (DPO, GRPO, RLAIF, Constitutional AI), followed by reward hacking, alignment and safety, and hands-on pipelines with Gymnasium, Stable-Baselines3 and Hugging Face TRL.

By the end you will not only be able to train agents and align models — you will understand why each method works, which is what lets you debug the inevitable failures and adapt as the field keeps moving. Let us begin with the formal object that underlies everything: the Markov decision process.