Dimitrios' note on basics of Markov chains
http://katselis.web.engr.illinois.edu/ECE586/Lecture1.pdf

http://katselis.web.engr.illinois.edu/ECE586/Lecture2.pdf