Herb/Mineral Bag Limit

Is there any way to open a new bag if you drop a previous herb/mineral bag? I was always under the impression that you could have 4 herb/mineral bags open in your inventory at any time, but after dropping Saffron's 4-slot herb bag and trying to open a 12-slot herb bag, I get a message saying "You cannot open more than 4 bag-type items."

So basically if you open a smaller slotted bag, you're stuck with it forever?