BACKGROUND AND PURPOSE: There is debate regarding the approach for analysis of modified Rankin scale scores, the most common functional outcome scale used in acute stroke trials. METHODS: We propose to use tests to assess treatment differences addressing the metric, "if a patient is chosen at random from each treatment group and if they have different outcomes, what is the chance the patient who received the investigational treatment will have a better outcome than will the patient receiving the standard treatment?" This approach has an associated statement of treatment efficacy easily understood by patients and clinicians, and leads to statistical testing of treatment differences by tests closely related to the Mann-Whitney U test (Wilcoxon Rank-Sum test), which can be tested precisely by permutation tests (randomization tests). RESULTS: We show that a permutation test is as powerful as are other approaches assessing ordinal outcomes of the modified Rankin scores, and we provide data from several examples contrasting alternative approaches. DISCUSSION: Whereas many approaches to analysis of modified Rankin scores outcomes have generally similar statistical performance, this proposed approach: captures information from the ordinal scale, provides a powerful clinical interpretation understood by both patients and clinicians, has power at least equivalent to other ordinal approaches, avoids assumptions in the parameterization, and provides an interpretable parameter based on the same foundation as the calculation of the probability value.