View Forum Topic

Forums
CS461
Problems with Gradient Checking?

Author	Message
msargent Posts: 519	Posted 15:10 Mar 15, 2015 \| Look for hard-coded values, as your code should work for neural nets of any size. Also, if you used vectorization, try a non-vectorized version for your gradient calculation. Also, Python is not pass-by-value, but pass-by-object-reference (see http://robertheaton.com/2014/02/09/pythons-pass-by-object-reference-as-explained-by-philip-k-dick/); if you pass a list or an array into a function, it will not copy it to use it locally in the function. If you want to do that to, say, an numpy array, you have to use numpy.copy(). Last edited by msargent at 15:12 Mar 15, 2015.
wsakura Posts: 64	Posted 21:45 Mar 15, 2015 \| I was wondering, for little delta of layer 2 in back propogation, am I supposed to remove the bias unit? When I do, grad_check returns a value of 31 and checkGradient won't work.

Author

Message

msargent

Posts: 519

Posted 15:10 Mar 15, 2015 |

Look for hard-coded values, as your code should work for neural nets of any size. Also, if you used vectorization, try a non-vectorized version for your gradient calculation.

Also, Python is not pass-by-value, but pass-by-object-reference (see http://robertheaton.com/2014/02/09/pythons-pass-by-object-reference-as-explained-by-philip-k-dick/); if you pass a list or an array into a function, it will not copy it to use it locally in the function. If you want to do that to, say, an numpy array, you have to use numpy.copy().

Last edited by msargent at 15:12 Mar 15, 2015.

wsakura

Posts: 64

Posted 21:45 Mar 15, 2015 |

I was wondering, for little delta of layer 2 in back propogation, am I supposed to remove the bias unit? When I do, grad_check returns a value of 31 and checkGradient won't work.