Don't grind it off, or use an impact. I've done about 450 pullies and never ground one off. You have to be doing something wrong or the puller has to have something wrong with it. I suggest getting our Craven puller and using it that should work fine.
http://www.waymotorworks.com/pulley-puller.html

 

Had a puller like that Craven on the one we did. It did not work and we did want to take the chance of damaging the shaft. Of the few I have done, only one has been really tough.

 

I just changed my pulley this past weekend. I used the Alta split collar tool with a crows foot type puller attached. The only problem I had was to find clearance for the puller. I had to buy a smaller puller to fit between the frame rail and the pulley.

My only regret is that I didn't change the motor mount at the same time.