Patients with pulmonary arterial hypertension (PAH) and chronic thromboembolic pulmonary hypertension (CTEPH) have low levels of physical activity (PA). Increased PA has health benefits including improved quality of life. This study aimed to identify patient-perceived barriers to PA that correlate with objectively measured PA in this population. We performed a cross-sectional survey of 40 patients with PAH and CTEPH. Participants rated how often 15 barriers interfere with being physically active on a 5-point Likert Scale. The primary outcome measure was PA quantified using the Fitbit Zip activity tracker for two weeks. The primary independent variables were the 15 barriers and a summary score (total average barriers). Separate multivariable linear regressions were performed to assess the association between the 15 barriers and the summary score and PA adjusting for age, sex, and PAH etiology. Of the participants, 85% (34/40) had valid step counts and were included. Of these 34, 85% (n = 29) were female and 91% (n = 31) had PAH. The median (interquartile range [IQR]) number of daily steps was 3913 (2309–6313). The barriers endorsed most strongly were lack of self-discipline, lack of energy, and lack of interest. In the multivariable analysis, a 1-unit increase in perceived lack of interest, lack of enjoyment, and lack of skills was associated with a significant decrease in step counts of -1414 steps (95% confidence interval [CI] = (−2580 – −248), −1458 steps (−2404 – −511), and −1533 steps (−2910 – −156), respectively. Counseling and interventions aimed at increasing PA in patients with PAH should address interest, enjoyment, and skill development.