This is now published. It is due 2018-09-14 11:59 PM for the Saturday class and 2018-09-16 11:59 PM for the MW class. 

Yesterday I told the Saturday class that this would be another easy week, with the more demanding material starting the following week. As it turned out, I was not able to find a way to break up the material to do that. So 4 MDPs - The Value Function is fairly demanding. 

Sure. What's most important is that you understand the diagram in Example 3.3 and how the table and the diagram relate to each other. In particular, what do the open circles and large black dots mean? How do they relate to the actions and to the effects caused by the environment?