Phantom - Stolen Skills and Auto Buff Gem

While this isnt entirely related, Has anyone noticed that if you use a buff, then switch to another skill for that same slot (IE, Having HS and then switching to say, Triple Throw), You get a message that says HS is no longer in use and will be removed from your buff? And then the buff disappears. Is that normal?

Yes, that's how it works.

And if you switch a gem buff, the buff disappears from the gem, and you have to re apply.