Honestly, this is just ridiculous.
I get that the game is free, and you're trying to get people to spend money on it, but not having some sort of confirmation when you spend gems is wrong. After saving up for over a week to get 50 gems and then double tapping on a lemon tree and spending 40 really really rubs me the wrong way.
Completely frustrated and at this point not sure I want to continue to play if this design remedied.