Gradient check only works on the first 3 columns of the y-vector data; digits 1-3. That's okay, as we aren't trying to make a prediction or anything like that: we are just plugging in the same data into two functions to see if the same results come out. All we are doing is checking if the algorithms are implemented correctly. 

The pdf from Coursera recommends using a smaller neural net for gradient checking, which is what the code does. 

Here's a printout of what it should look like: you actual numbers might be different, but the columns should be as close as these are. 

You are getting that because the Y values haven't been converted to vectors of 1s and 0s yet, that is supposed to happen in the function.

Edit: You could replace the Y[:10, :3] with Y[:10, :] and get the same result, as the Y matrix going in only has one column. That is, it's the 3 argument in "gradApprox(params_check[:35], 4, 4, 3, X[:10, :4], Y[:10, :], lambd)" that specifies that the output of the net only considers the first three digits. The rest are just ignored.