Markov Decision Process (MDP)

Youtube search... ...Google search

• Markov Model (Chain, Discrete Time, Continuous Time, Hidden)

• Reinforcement Learning (RL)
  • Monte Carlo (MC) Method - Model Free Reinforcement Learning
  • Markov Decision Process (MDP)
  • State-Action-Reward-State-Action (SARSA)
  • Q Learning
    • Deep Q Network (DQN)
  • Deep Reinforcement Learning (DRL) DeepRL
  • Distributed Deep Reinforcement Learning (DDRL)
  • Symbiotic Intelligence ... Bio-inspired Computing ... Neuroscience ... Connecting Brains ... Nanobots ... Molecular ... Neuromorphic ... Evolutionary/Genetic
  • Actor Critic
    • Asynchronous Advantage Actor Critic (A3C)
    • Advanced Actor Critic (A2C)
    • Lifelong Latent Actor-Critic (LILAC)
  • Hierarchical Reinforcement Learning (HRL)

Solutions:

• Dynamic Programming
• Monte Carlo
• Difference Learning

Used where outcomes are partly random and partly under the control of a decision maker. MDP is a discrete time stochastic control process. At each time step, the process is in some state s, and the decision maker may choose any action a that is available in state s. The process responds at the next time step by randomly moving into a new state s', and giving the decision maker a corresponding reward R_{a}(s,s')} R_a(s,s'). The probability that the process moves into its new state s' is influenced by the chosen action. Helping the convergence of certain algorithms a discount rate (factor) makes an infinite sum finite.

(Richard) Bellman Equation

• Reinforcement Learning : Markov-Decision Process (Part 1) | Ayush Singh - Towards Data Science
• Reinforcement Learning: Bellman Equation and Optimality (Part 2) | Ayush Singh - Towards Data Science