Lectures

Several key dates: Everyone has two weeks for each homework. HW1 will be posted after Lecture 6 is done. HW2 will be posted after Lecture 12 is done. HW3 will be posted after Lecture 20 is done. Project proposal is due on the date when HW3 is posted. The final project is due on December 20th.

• Lecture 1: Overview

Lecture Note 1

• Lecture 2: Unifying the Analysis in Control and Optimization via Semidefinite Programs

Lecture Note 2, Some Useful References

• Lecture 3a & 3b: Uncertainty Modeling and Robustness Analysis

Lecture Note 3

• Lecture 4a & 4b: Connection between Stochastic Optimization Methods and Feedback Systems

Lecture Note 4

• Lecture 5a & 5b: Lure-Postnikov Lyapunov Functions

Lecture Note 5

• Lecture 6: Markov Chains and Markov Decision Processes

Lecture Note 6, Some Useful References

• Lecture 7: Introduction to Reinforcement Learning

Lecture Note 7

• Lecture 8: Model-based Policy Evaluation

Lecture Note 8

• Lecture 9: Optimal Bellman Equation and Value Iteration

Lecture Note 9, Some Useful References

• Lecture 10: Q-factors and Policy Iteration

Lecture Note 10

• Lecture 11a & 11b & 11c: Temporal Difference Learning

Lecture Note 11

• Lecture 12: Value-Based RL

Lecture Note 12

• Lecture 13a & 13b & 13c: Policy-Based RL: Policy Gradient, Actor-Critic, Natural Policy Gradient, and TRPO

Lecture Note 13, Some Useful References

• Lecture 14: Stabilization

Lecture Note 14

• Lecture 15: Imitation Learning

Lecture Note 15, Some Useful References

• Lecture 16: Model Predictive Control and Model-Based RL

Some Useful References