Long-standing theory predicts that the intensity of consumer–prey interactions declines with increasing latitude, yet for plant–herbivore interactions, latitudinal changes in herbivory rates and plant palatability have received variable support. The topic is of growing interest given that lower-latitude species are moving poleward at an accelerating rate due to climate change, and predicting local interactions will depend partly on whether latitudinal gradients occur in these critical biotic interactions. Here, we assayed the palatability of 50 seaweeds collected from polar (Antarctica), temperate (northeastern Pacific; California), and tropical (central Pacific; Fiji) locations to two herbivores native to the tropical and subtropical Atlantic, the generalist crab Mithraculus sculptus and sea urchin Echinometra lucunter. Red seaweeds (Rhodophyta) of polar and temperate origin were more readily consumed by urchins than were tropical reds. The decline in palatability with decreasing latitude is explained by shifts in tissue organic content along with the quantity and quality of secondary metabolites, degree of calcification or both. We detected no latitudinal shift in palatability of red seaweeds to crabs, nor any latitudinal shifts in palatability of brown seaweeds (Phaeophyta) to either crabs or urchins. Our results suggest that evolutionary pressure from tropical herbivores favored red seaweeds with lower palatability, either through the production of greater levels of chemical defenses, calcification, or both. Moreover, our results tentatively suggest that the “tropicalization” of temperate habitats is facilitated by the migration of tropical herbivores into temperate areas dominated by weakly defended and more nutritious foods, and that the removal of these competing seaweeds may facilitate the invasion of better-defended tropical seaweeds.