Introduction Higher sedentary time (ST) and lower moderate- to vigorous-intensity physical activity (MVPA) have each been associated with greater adiposity, but most studies are cross-sectional and measure ST and MVPA by self-report. This study evaluated associations between objective ST and MVPA with current and 5-year changes in BMI and waist circumference. Methods The Coronary Artery and Risk Development in Young Adults longitudinal cohort study recruited black or white young adults from four U.S. cities. This analysis (conducted in 2016) used data from 2005 to 2006 as baseline and 2010 to 2011 as 5-year follow-up. Accelerometers measured baseline ST (total and prolonged in bouts of ≥10 minutes) and MVPA (bouts of ≥10 minutes). BMI and waist circumference were assessed at baseline and repeated 5 years later. Regression models included sedentary time and MVPA simultaneously with adjustment for demographics and lifestyle factors. Results Participants (n=1,826) were 57% female; 40% black; aged 38–50 years; and had a BMI of 28.7 (SD=6.3). At baseline, total and prolonged ST were directly associated with BMI and waist circumference, whereas MVPA was inversely related (all p<0.05). Longitudinally, only prolonged ST (per hour/day) was associated with greater increases in BMI (0.077, p=0.033) and waist circumference (0.198 cm, p=0.028). Associations between ST and adiposity were more apparent in less active participants. Risk of ≥5% increase in BMI across assessments increased by 8%–10% (p<0.05) per hour/day of ST. Conclusions Time spent sedentary was associated with increases in adiposity over time. Reducing sedentary time may be a novel strategy for weight control.