yes. They are optional. I was fortunate to have monsters spawn when I needed them and to have some trees ready, so thus I did them.
In my opinion, for the late starters , if you want to do the side goals, you better just take a step back and just concentrate on collecting grapes.
I'm letting the later sidegoals expire.