I turned off all my walkers. Someone earlier posted they noticed a lag trying to load neighbors with a lot of walkers. Do you have Location Services enabled (except when using app), background apps allowed to auto-refresh, Mail or other apps that Push set to auto push instead of you collecting them? What about your Notifications? Things other than the game running in the background use processor power, in addition to Game Settings like Music, Sounds, Notifications, and Facebook.
I have an iPad Air 2, since I shut off a ton of stuff I have less problems.