Skip to content
anythingai.space
AI Foundationsintermediate16 min · 11 steps

Reinforcement Learning: Q-Learning to PPO

How an agent learns from reward alone — by estimating action values (Q-learning), by pushing its policy uphill (policy gradients), or both at once (actor-critic).

action aₜstate sₜ₊₁, reward rₜONE EPISODE — THE REWARD ONLY ARRIVES AT THE ENDs0a0r=0s1a1r=0s2a2r=0…r=+1AgentEnvironmentgame · robot · market
requestresponsedatasignal
1 / 11
ProblemStep 1 of 11

Learning from reward, not labels

In reinforcement learning nobody tells the agent the right action. It observes a state, takes an action, and the environment answers with a new state and a reward — often zero for many steps, then a win or a loss.

The goal is to maximize the return: the sum of (discounted) future rewards. The hard part is credit assignment — which of the many earlier actions earned that final reward?

What changed
  • +Agent
  • +Environment
  • +Action → state, reward loop

Lesson 23 of 34 in AI Foundations