The moon dust nightmare doesn't last long. Part 5 involves collecting moon beams from ponds. About 10, if I remember correctly. Then in Part 6, we collect moon stones from animals (and raise 10 pigs). Upgrading continues to require star dust from trees but the mushrooms, ponds, and animals are only used for a short while. That's why the gem items only drop star dust.