Background. Cancer treatment -related side effects may have a negative impact on quality of life among cancer survivors and may limit participation in physical activity (PA). Hypothesis. Cancer-specific concerns will be reduced throughout a 10-month diet and exercise intervention among recently diagnosed cancer survivors. Additionally, participants reporting greater levels of PA will also report fewer cancer-specific concerns. Study design. This study is an exploratory analysis of 452 recently diagnosed, early-stage breast and prostate cancer survivors who participated in the FRESH START diet and exercise trial. Data were collected at baseline and 1-year follow-up. Results. At baseline, chief concerns among prostate cancer survivors included ability to have an erection (mean score [standard deviation] = 1.0 [1.3]) and urinary frequency (2.5 [1.4]), whereas among breast cancer survivors, eminent concerns were not feeling sexually attractive (2.0 [1.3]) and worry about cancer in other members of their family (2.1 [1.3]). At 1 year, there was a significant improvement in cancer-specific concerns on breast cancer-specific concerns (P <.01) but not on prostate cancer-specific concerns. At baseline, women who were self-conscious about their dress had higher levels of PA, whereas men reporting issues with incontinence reported lesser increases in PA in response to the intervention. Conclusion. Cancer-specific concerns diminish over time, especially among breast cancer survivors. Among prostate cancer survivors, incontinence is a significant barrier that hinders benefit from PA interventions. Thus, there is a need either for medical interventions to ameliorate incontinence or for behavioral interventions to address this issue among survivors. © The Author(s) 2012.