You won't get 1 a day, more likely 1 every 2 weeks. Until the drop rate was increased recently, I've had 2 in 2 years. It may take a month or so to get 5 pies. Keep collecting from Ivy's Hut and there is also a chance to get a pie from the 2 Belladonna trades in the Royal Exchange as well.
I think we need to think of this as a semi long term goal.
No gems necessary to finish this goal, just patience!