When you spent Gems without meaning to.
If you tap on things that are not ready to collect yet, you see the Gem cost to speed it up. One more tab and you pay that cost. (there is no confirmation, no 'are you sure you want to spent 50 Gems speeding up this building?')
It feels truly awful to see your Gems be taken away for something you didn't want/ need. (and also in my opinion, the biggest flaw of this game)