Learning · 7 min read
Q-Learning
Learns which moves pay off from each cell. Learns action values per cell across episodes, then walks epsilon-greedy among safe neighbours.
Why this category
Learning. Update heuristics or action values across attempts from what failed before.
How it explores
Exploring: 1 of 22 states checked
Why this category
Q-Learning is Learning: a table Q[cell][direction] persists across calls and updates from rewards. Behaviour improves from what paid off or failed, not from a fixed distance heuristic alone.
When knownUnsafe is empty the table resets. That keeps a fresh level from inheriting another maze's action values.
How it picks the next move
Each attempt runs several episodes. From the current cell, list safe neighbours at the next tick. With probability epsilon pick one at random; otherwise pick the action with the highest Q. Step, then update Q with a step penalty, a heavy dead-end penalty, or a large exit reward, bootstrapping from the next cell's best action.
The returned path is the best successful episode (final episode is greedy). Illegal "x,y,phase" cells never enter the action list.
What it stores after a fail
Shared death calendar plus the Q-table. A catch grows "x,y,phase" bans for everyone; Q-Learning also keeps whichever action values it already updated during prior episodes and attempts.
Unlike Learning Real-Time A*, the memory is over actions, not a single heuristic number per cell — so the same square can learn "go north" while "go east" stays cold.
More learning
Try it live
Run this algorithm on a built-in level, or paint your own grid and guards.