You'll only get red fish when you start the Adamant goal chain. (And once you finish the step with the red fish, they stop showing up.)