Dozens of published papers cite factors related to leisure-time physical activity (LTPA) participation among people with physical disabilities. Unfortunately, there has been little effort to synthesise this literature in a manner that is accessible and useful to the sectors (e.g., health care, recreation) responsible for LTPA promotion in disability populations. In this systematic review, over 200 factors were extracted from 22 review articles addressing barriers and facilitators to LTPA in children and adults with physical disabilities. Factors were grouped according to common themes, classified into five levels of a social ecological model, and coded according to whether they could be affected by the health-care and/or recreation sectors. Findings are discussed with regard to key factors to target in LTPA-enhancing interventions, relevant theories and models in which to frame interventions, the levels at which the interventions can be implemented, and intervention priorities. The synthesis provides a blueprint and a catalyst for researchers and practitioners to shift focus from conducting studies that merely describe LTPA barriers and facilitators, to developing and delivering strategies to increase LTPA among persons with physical disabilities.