Background: Moderate-to-vigorous intensity physical activity (MVPA) is associated with favorable self-rated mental and physical health. Conversely, poor self-rated health in these domains could precede unfavorable shifts in activity. We evaluated bidirectional associations of accelerometer-estimated time spent in stationary behavior (SB), light intensity physical activity (LPA), and MVPA with self-rated health over 10 years in in the CARDIA longitudinal cohort study. Methods: Participants (n = 894, age: 45.1 ± 3.5; 63% female; 38% black) with valid accelerometry wear and self-rated health at baseline (2005–6) and 10-year follow-up (2015–6) were included. Accelerometry data were harmonized between exams and measured mean total activity and duration (min/day) in SB, LPA, and MVPA; duration (min/day) in long-bout and short-bout SB (≥30 min vs. < 30 min) and MVPA (≥10 min vs. < 10 min) were also quantified. The Short-Form 12 Questionnaire measured both a mental component score (MCS) and physical component score (PCS) of self-rated health (points). Multivariable linear regression associated baseline accelerometry variables with 10-year changes in MCS and PCS. Similar models associated baseline MCS and PCS with 10-year changes in accelerometry measures. Results: Over 10-years, average (SD) MCS increased 1.05 (9.07) points, PCS decreased by 1.54 (7.30) points, and activity shifted toward greater SB and less mean total activity, LPA, and MVPA (all p < 0.001). Only baseline short-bout MVPA was associated with greater 10-year increases in MCS (+ 0.92 points, p = 0.021), while baseline mean total activity, MVPA, and long-bout MVPA were associated with greater 10-year changes in PCS (+ 0.53 to + 1.47 points, all p < 0.005). In the reverse direction, higher baseline MCS and PCS were associated with favorable 10-year changes in mean total activity (+ 9.75 cpm, p = 0.040, and + 15.66 cpm, p < 0.001, respectively) and other accelerometry measures; for example, higher baseline MCS was associated with − 13.57 min/day of long-bout SB (p < 0.001) and higher baseline PCS was associated with + 2.83 min/day of MVPA (p < 0.001) in fully adjusted models. Conclusions: The presence of bidirectional associations between SB and activity with self-rated health suggests that individuals with low overall activity levels and poor self-rated health are at high risk for further declines and supports intervention programming that aims to dually increase activity levels and improve self-rated health.