Ive only recently started doing this trade. But only because my gold nuggets were maxed out. I didn't want to see a black flag on gold nuggets. The other requirements are pretty attainable in my kingdom, but I go about getting it done as a side quest so its pretty slow going since I want to maintain a good stockpile of nuggets.
I can understand why some folks wouldn't bother with it. I didn't for a long time.
I guess I should add that I don't do the gold trade. If I want gems, I'll buy them and use them immediately.