Background: Physical inactivity is prevalent among African-American women in rural Alabama. The purpose of this study was to explore personal, social, and physical environmental factors associated with activity to help plan interventions. Methods: As part of the Women's Cardiovascular Health Network Project, telephone surveys were collected from African-American women residing in three rural counties. The women reported the number of minutes engaged in moderate or vigorous activities and were categorized into the following groups: (1) inactive (no moderate or vigorous activities), (2) insufficient (not meeting recommendations), and (3) meets recommendations (engaged in moderate activity five times per week for at least 30 minutes or vigorous activity three times per week for at least 20 minutes). Logistic regression modeling was used to identify personal, social, and physical environmental factors associated with the more active groups. Results: Among the 567 women who were classified in physical activity groups, 221 (39%) met the recommendations, 260 (46%) were insufficiently active, and 86 (15%) were inactive. In the adjusted model, the social environmental factors associated with women meeting the recommendations (versus inactive) were attending religious services and seeing people exercise in the neighborhood. Attending religious services, knowing people who exercise, and a higher social issue score were associated with women who reported any activity (versus inactive). No physical environmental factors were associated with the more active groups. Conclusions: Social environmental factors were associated with higher levels of activity and need to be considered when planning interventions. More research is needed to identify associations between specific aspects of the social environment and physical activity behaviors.