Consumers' guides that profile the quality of care of individual health care providers may be influential in shaping health care markets. We propose four simple questions that can be used to evaluate such guides: (a) Does the guide measure distinct and important domains of health care quality? (b) Are the individual measures of quality described simply and precisely? (c) Do the measures take into account relevant differences between patients? (d) Are the ratings of quality presented fairly? Using these four questions, we examine the validity of one prominent guide that annually identifies America's best hospitals and present a set of recommendations for the design of future guides. Although the evaluation of health care quality is undoubtedly complex, the four questions that we pose provide a basis for developing a more rational approach to informing the public about health care quality.