Q vs V in Reinforcement Learning, the Easy Way

Update: The best way of learning and practicing Reinforcement Learning is by going to http://rl-lab.com

In the Math Behind Reinforcement Learning, the Easy Way we explained the formula

In this article we will see what is Q and how does it relate to V. For this we have to make some medieval scenario.

Medieval Battle

Suppose a medieval commander is about to attack a fortress. To breach its walls he has 3 options (or actions) he can order his troops to use the rope, the ladder or the siege tower.
Each option comes with its own risk and reward.

Before delving more into details, let’s assume that the battle environment has four states:

• Attack: this is the initial state, when the commander has completed his preparations and ready to attack.
• Breach: this state is when the fortress walls have been breached and the troops are fighting inside the fortress.
• Victory: this is when the commander’s army takes control of the fortress.
• Failure: this is the state when the commander fails to breach the walls of the fortress or fails to take control of the fortress after breaching its walls.