Should I ditch my mage?

You have to decide what's more valuable to you. In my opinion, bossing is not as important as training. That is, you get splits and into loot order for stuff like Bliz 30, but ultimately you spend most of your time grinding. In 4th job any mage can train very fast, faster than anyone else. Also, you're not as heavily dependent on gear as other characters, and you can train effectively without "godly" stuffs.

Also I might add that at like 17x, you'll get into boss parties anyway. Getting to 17x as an i/l will be easier than with a bowman or thief.