Natural selection is the result of organisms’ interactions with their environment, but environments vary in space and time, sometimes in extreme ways. Such variation is generally thought to play an important role in evolution by natural selection, maintaining genetic variation within and between populations, increasing the chance of speciation, selecting for plasticity of responses to the environment, and selecting for behaviors such as habitat selection and niche construction. Are there different roles that environmental variation plays in natural selection? When biologists make choices about how to divide up an environment for the sake of modeling or empirical research, are there any constraints on these choices? Since diverse evolutionary models relativize fitnesses to component environments within a larger environment, it would be useful to understand when such practices capture real aspects of evolutionary processes, and when they count as mere modeling conveniences. In this paper, I try to provide a general framework for thinking about how fitness and natural selection depend on environmental variation. I’ll give an account of how the roles of environmental conditions in natural selection differ depending the probability of being experienced repeatedly by organisms, and how environmental conditions combine probabilistically to help determine fitness. My view has implications for what fitness is, and suggests that some authors have misconceived its nature.