We examined contrast sensitivity and suprathreshold apparent contrast with natural images. The spatial-frequency components within single octaves of the images were removed (notch filtered), their phases were randomized, or the polarity of the images was inverted. Of Michelson contrast, root-mean-square (RMS) contrast, and band-limited contrast, RMS contrast was the best index of detectability. Negative images had lower apparent contrast than their positives. Contrast detection thresholds showed spatial-frequency-dependent elevation following both notch filtering and phase randomization. The peak of the spatial-frequency tuning function was approximately 0.5-2 cycles per degree (c/deg). Suprathreshold contrast matching functions also showed spatial-frequency-dependent contrast loss for both notch-filtered and phase-randomized images. The peak of the spatial-frequency tuning function was approximately 1-3 c/deg. There was no detectable difference between the effects of phase randomization and notch filtering on contrast sensitivity. We argue that these observations are consistent with changes in the activity within spatial-frequency channels caused by the higher-order phase structure of natural images that is responsible for the presence of edges and specularities. © 2002 Optical Society of America.