(This is about part A of this week's homework.)

It was necessarily to modify the code for value iteration to accommodate both the BookGrid and Q-Value iteration. Here is the updated version.  Here are iteration steps on both grids using both techniques.

To run one or the other use the following at the bottom of the file:

ValueIterationAgent(BookGrid(), valueType='value') 
ValueIterationAgent(BookGrid(), valueType='qvalue')
ValueIterationAgent(Sutton_Ex3_5(), valueType='value')
ValueIterationAgent(Sutton_Ex3_5(), valueType='qvalue')

The BookGrid class is sketched out for you. Your job is to fill in the missing parts.