Don't worry, it is resetable. I put a single point from 2nd job into Magic Armor (a 1st job skill) and I took it out later with a 2nd job SP reset so I could max slow with 3rd job points. Don't worry about it, the game appears to be able to tell the difference between a 2nd job SP and 1st job SP. I'm sure it'll work with 4th job SP... good luck 
Don't let the posts before scare you. I CONFIRMED it is possible.

Don't let the posts before scare you. I CONFIRMED it is possible.

