How does the scroll system work?

Really, unless Nexon itself releases all of their RNG systems on how their percentages are generated, we will never know for certain. From a semi legal/ethical standpoint, the percentages should be a fix percent they work at, regardless of any outside variables: item being scrolled, type of scroll being used, location of character when scrolling, etc.

With that being said however, it is totally possible that the servers could house within their item databases a "rarity" rank or whatever you would care to call it that impacts the probability that a scroll will fail/pass. To take things further you could even say that those probabilities are more than likely directly linked to drop chance, seeing as how most people tend to claim the more rare the item, the less likely the scroll seems to work. This would best chart out as an inverse variation, as the item rarity goes up, the actual scroll chance to work goes down. This chart, of course, would only apply to scrolls under 100%, and to not make things too unfair or noticeable would be a fairly small curve to it. Adding in event and quest item's "rarity" as seen fit by Nexon.

But, that is total speculation and will never be verified without the raw equations from their own servers.