Modified Rankin Scale (MRS)

Overview

We conducted a literature search to identify all relevant publications on the psychometric properties of the MRS.

Floor/Ceiling Effects

Dromerick, Edwards, and Diringer (2003) administered the MRS to 95 stroke rehabilitation inpatients and reported that the MRS displayed an adequate floor effect (18%) at admission to rehabilitation.

Reliability

Test-retest:
Wolfe, Taub, Woodrow, and Burney (1991) examined the test-retest reliability of the MRS in 50 patients with stroke of varying severity. Two out of three research nurses interviewed patients on two occasions that were 2-3 weeks apart. The test-retest reliability using the weighted kappa statistic was excellent (kappa w = .95).

Wilson et al. (2005) examined the test-retest reliability of the MRS in patients at least 6 months post-stroke, using two raters who performed repeat assessments with a mean test-retest interval of 7 days. Agreement was measured using the kappa statistic. Comparison of Rankin grades showed that there was excellent agreement between the first and second assessments. Agreement between the first and second assessments was found in 85% of cases for rater 1 (kappa = 0.81; kappa w = 0.94), and in 96% for rater 2 (kappa = 0.95; kappa w = 0.99).

Intra-rate:
Wolfe et al. (1991) examined the intra-rater reliability of the MRS in a sample of 14 patients who were assessed twice by the same observer within a 2-week period at least 3 months post-stroke. Exact agreement was reported in 86% of observations (kappa w = 0.95). The intra-rater reliability of the MRS as reported in this study is considered to be excellent.

Inter-rater:
van Swieten et al. (1988) examined the inter-rater reliablity of the MRS in 100 patients who were interviewed by two physicians using kappa statistics. Physician agreement on the degree of handicap of the patients occurred for 65% of the patients. The physicians differed by one Rankin grade in 32% of the patients and by two grades in 3% of the patients. The kappa for all pairwise observations was adequate (kappa = 0.56; kappa w = 0.91). For the outpatient group, the kappa was excellent (kappa = 0.82). For the inpatient group, the kappa was adequate (kappa = 0.51).

Wolfe et al (1991) examined the inter-rater reliability of the MRS in 50 patients with stroke of varying severity. Two out of three research nurses interviewed patients. The kappa coefficients were excellent and ranged from kappa = 0.75 to kappa = 0.96. However, analysis of variance revealed that there was evidence of a systematic difference between the raters (F 2,48 = 6.02, p = 0.005), with raters 1 and 3 estimating the grade 0.42 and 0.33 points higher than rater 2.

Wilson et al. (2002) examined the inter-rater reliability of the MRS in 63 patients with stroke. The MRS was administered by two raters. Inter-rater reliability was measured with the kappa statistic and was found to be excellent (kappa w = 0.78). However, overall agreement between the 2 raters was only 57%, and one rater assigned significantly lower grades than the other (p = 0.048).

Wilson et al. (2005) examined the inter-rater reliability of the MRS in patients at least 6 months post-stroke. Fifteen raters were recruited for the study and pairs of raters assessed a total of 113 patients on the MRS. Agreement between raters was observed in only 43% of cases (kappa = 0.25, kappa w = 0.71).

Shinohara, Minematsu, Amano, and Ohashi (2006) examined the inter-rater reliability of the MRS when an expanded guidance scheme (a guided interview format) and corresponding questionnaire was used. Twenty raters (neurologists and nurses) watched videotapes of 30 patients interviewed and scored each patient. Inter-rater reliability was calculated using the intraclass correlation coefficient (ICC). In this study, inter-rater reliability was excellent (ICC = 0.95 for neurologists and ICC = 0.96 for nurses).

Quinn, Dawson, Walters, and Lees (2008) assessed the inter-rater reliability of the MRS among 2942 evaluators from 30 different countries. The evaluators rated 5 non-scripted videotaped interviews. Inter-rater reliability was calculated using Kappa statistics. The overall inter-rater reliability of the MRS was adequate (kappa = 0.67). The agreement level at each grade of the MRS was poor for a score of 0 (kappa = 0.19), adequate for a score of 2 (kappa = 0.48) and 3 (kappa = 0.74), and excellent for a score of 4 (kappa = 0.95). The agreement level for scores of 0 and 5 were not reported since the videotaped interviews did not include clients with a full range of disabilities. The inter-rater reliability by country was poor for Italy (kappa = 0.34), adequate for Belgium (kappa = 0.73), Czech Republic (kappa = 0.68), France (kappa = 0.64), Hungary (kappa = 0.70), Netherlands (kappa = 0.50), South Korea (kappa = 0.67), Sweden (kappa = 0.65), Unites States (kappa = 0.73) and the United Kingdom (kappa = 0.69) and excellent for Australia (kappa = 0.77), Germany (kappa = 0.78), Portugal (kappa = 0.80), Slovakia (kappa = 0.75) and Spain (kappa = 0.84). The agreement level was excellent for both native and non-native English speakers (kappa = 0.77; kappa = 0.76). Among assessors from the United Kingdom the inter-rater reliability was adequate for all professional backgrounds: general medicine (kappa = 0.66), geriatrics (kappa = 0.54), neurology (kappa = 0.56), and research assistants (kappa = 0.65).
Note: The inter-reliability by country was calculated only for countries with more than 50 certified evaluators.

Validity

Criterion:
Concurrent:
Cup, Scholte op Reimer, Thijssen, and van Kuyk-Minis (2003) examined the concurrent validity of the MRS with the Canadian Occupational Performance Measure (COPM), the Barthel Index (BI), the Frenchay Activities Index (FAI), the Stroke-Adapted Sickness Impact Profile-30 (SA-SIP30), and the Euroqol 5D (EQ-5D) in 26 patients post-stroke at their place of residence. The MRS had a statistically significant correlation with the BI, FAI, SA-SIP30 and the EQ-5D. Spearman’s rho correlation coefficients were excellent for the BI, FAI and EQ-56 (r = -0.81, -0.80, and 0.68, respectively). An adequate correlation was found between the MRS and the SA-SIP30 (r = 0.47).
Note: Some correlations are negative because a high score on the MRS indicates increased impairment whereas a low score on other measures indicates increased impairment.

Kwon, Harzema, Duncan, and Min-Lai (2004) examined the concurrent validity of the Barthel Index (BI), the motor component of the Functional Independence Measure (M-FIM), and the MRS using Spearman correlation coefficients. Excellent correlations were observed between the MRS and the BI (r = -0.89) and between the M-FIM and the MRS (r = -0.89).

Weimar et al. (2002) examined the concurrent validity of the MRS from a sample of 4,264 patients with acute ischemic stroke from 30 hospitals in Germany during a 1-year period. The patients were administered the Barthel Index (BI), the MRS, the Short Form-36 Physical Functioning subscale (SF-36 PF), and the Center for Epidemiologic Studies-Depression short form (CES-D). The MRS had an excellent correlation with the SF-36 PF (r = 0.84) and with the BI (r = 0.82).

Schaefer, Huisman, Sorensen, Gonzalez, and Schwamm (2004) examined whether diffusion-weighted Magnetic Resonance Imaging (MRI) findings (thought to demonstrate lesions that are not visualized with conventional MRI sequences) and conventional MRI findings correlate with discharge MRS and Glasgow Coma Scale scores in 26 patients with diffuse axonal injury. Using Spearman rank correlation coefficients, the results of this study showed that the strongest correlation was between signal-intensity abnormality volume on diffusion-weighted images and MRS score, which was excellent (r = 0.77). For lesion number, the strongest correlation was between lesion number on images acquired and all sequences and MRS score, which was also excellent (r = 0.66). For lesion location, the strongest correlation was between lesion location in the corpus callosum and MRS score, which was adequate (r = 0.51). There was an adequate correlation between the MRS and the Glasgow Coma Scale.

Predictive:
Weimar et al. (2002) identified the most important predictors of adverse outcomes on the Barthel Index (BI) and MRS following stroke. The most relevant predictors were MRS scores before the stroke event, the presence of diabetes, and severity of left arm weakness.
Note: Although MRS scores > 3 was an inclusion criterion in this study, it did not specify how the MRS scores were obtained before the stroke event.

Construct:
Convergent/Discriminant :
Tilley et al. (1996) found that the MRS was closely related to the Glasgow Outcome Scale (94% agreement; Φ = 0.88) and with impairment measured by the NIH Stroke Scale (86% agreement; phi coefficient = 0.67) and the Barthel Index (87% agreement; Φ = 0.76). These results raise concern about the construct validity of the MRS. The results of this study lends support to the assertion that the MRS is closer to a disability scale than a handicap scale.

de Haan, Horn, Limburg, van Der Meulen, and Bossuyt (1993) evaluated 87 patients who had a stroke 6 months prior to evalutation. Impairments were scored on five stroke scales: the Orgogozo Scale, the National Institutes of Health Stroke Scale, the Canadian Neurological Scale, the Mathew scale, and the Scandinavian Stroke Scale. Disability was assessed with the Barthel Index, handicap with the MRS, and quality of life with the Sickness Impact Profile. The correlations between MRS and the 5 impairment scales using Pearson’s coefficients ranged from adequate to excellent (ranging from r = -0.56 to r = -0.71).
Note: Some correlations are negative because a high score on the MRS indicates increased impairment whereas a low score on other measures indicates increased impairment.

de Haan, Limburg, Bossuyt, van der Meulen, and Aaronson (1995) reported a strong relationship (using Somers’ D) between activities of daily living as measured by the Barthel Index (0.73) and the subscales of the Sickness Impact Profile including Instrumental activities of daily living (0.65), Mobility (0.60) and Living arrangements (0.74) The weakest associations reported were between the MRS and the Sickness Impact Profile subscales of Cognitive Alertness (0.34) and Social Interaction (0.37).

Wolfe et al. (1991) administered the MRS and the Barthel Index (which assesses disability) to 50 patients post-stroke. The correlation between the MRS and the Barthel Index was measured using kappa statistics. There was an excellent correlation (kappa = 0.72; weighted kappa = 0.91) between the two scales, which lends support to the assertion that the MRS is closer to a disability scale than a handicap scale.

Responsiveness

Dromerick et al. (2003) examined the responsiveness of the MRS in comparison to 3 other disability scales (the International Stroke Trial Measure; the Barthel Index (BI); the Functional Independence Measure (FIM). The MRS was administered to 95 stroke rehabilitation inpatients at admission and at discharge. The MRS was poor at detecting change. When compared to the FIM, the receiver operating characteristics analysis showed that the MRS (C-statistic C = 0.59) was much less sensitivity to change compared with the BI (C-statistic C = 0.82), indicating a corresponding lower specificity for the MRS. The MRS detected change in 55 subjects, including all who changed on the International Stroke Trial Measure. The BI detected change in 71 patients and the FIM detected change in 91 patients. The results of this study suggest that the global scales (MRS and the International Stroke Trial Measure) are much less sensitive to changes in disability than the activities of daily living scales (the BI and the FIM).

References
  • Bamford, J. M., Sandercock, A. G., Warlow, C. P., Slattery, J. (1989). Interobserver agreement for the assessment of handicap in stroke patients (letter). Stroke, 20, 828.
  • Bamford, J. M., Vessey, M., Fowler, G., Molyneux, A., Hughes, T., Burn, J., et al. (1988). A prospective study of acute cerebrovascular disease in the community: The Oxfordshire Community Stroke Project 1981-1986. 1. Methodology, demography and incident cases of first-ever stroke. J Neurol Neurosurg Psychiatry; 51, 1373-1380.
  • Berger, K., Weltermann, B., Kolominsky-Rabas, P., Meves, S., Heuschmann, P., Bohner, J., Neundorfer, B., Hense, H. W., Buttner, T. (1999). The reliability of stroke scales. The german version of the NIHSS, ESS and Rankin scales. Fortschr Neurol Psychiatr, 67(2), 81-93.
  • Cup, E. H. C., Scholte op Reimer, W. J. M., Thijssen, M. C. E., van Kuyk-Minis, M. A. H. (2003). Reliability and validity of the Canadian Occupational Performance Measure in stroke patients. Clin Rehabil, 17, 402-409.
  • de Haan, R., Horn, J., Limburg, M., van Der Meulen, J., Bossuyt, P. (1993). A comparison of five stroke scales with measurement of disability, handicap and quality of life. Stroke, 24, 1178-1181.
  • de Haan, R., Limburg, M., Bossuyt, P., van der Meulen, J., Aaronson, N. (1995). The clinical meaning of Ranking ‘handicap’ grades after stroke. Stroke, 26, 2027-2030.
  • Dromerick, A. W., Edwards, D. F., Diringer, M. N. (2003). Sensitivity to changes in disability after stroke: A comparison of four scales useful in clinical trials. Journal of Rehabilitation Research and Development, 40(1), 1-8.
  • Hop, J., Rinkel, G. J. E., Algra, A., van Gijn, J. (1998). Quality of life in patients and partners after aneurismal subarachnoid hemorrhage. Stroke, 29, 798-804.
  • Kwon, S., Harzema, A. G., Duncan, P. W., Min-Lai, S. (2004). Disability measures in stroke: Rehationship among the Barthel Index, the Functional Independence Measure, and the Modified Rankin Scale. Stroke, 35, 918-923.
  • Lai, S. M., Duncan, P. W. (2001). Stroke recovery profile and the Modified Rankin Assessment. Neuroepidemiology, 20, 26-30.
  • New, P. W., Buchbinder, R. (2006). Critical appraisal and review of the Rankin Scale and its derivatives. Neuroepidemiology, 26, 4-15.
  • Oveisgharan, S., Shirani, S., Ghorbani, A., Soltanzade, A., Baghaei, A., Hosseini, S., Sarrafzadegan, N. (2006). Barthel Index in a middle-east country: Translation, validity and reliability. Cerebrovascular Diseases, 22, 350-354.
  • Quinn, T.J., Dawson, J., Walters, M.R., Lees, K.R. (2008). Variability in Modified Rankin Score across a large cohort of international observers. Stroke, 39, 2975-2979.
  • Rankin, J. (1957). Cerebral vascular accidents in patients over the age of 60. Scott Med J, 2, 200-215.
  • Schaefer, P. W., Huisman, T., Sorensen, G., Gonzalez, G., Schwamm, L. (2004). Diffusion-weighted MR Imaging in closed head injury: High correlation with initial Glasgow Coma Scale score and score on Modified Rankin Scale at discharge. Neuroradiology, 233, 58-66.
  • Shinohara, Y., Minematsu, K., Amano, T., Ohashi, Y. (2006). Modified Rankin Scale with expanded guidance scheme and interview questionnaire: interrater Agreement and Reproducibility of Assessment. Cerebrovasc Dis, 21, 271-278.
  • Sulter, G., Steen, C., De Keyser, J. (1999). Use of the Barthel index and modified Rankin scale in acute stroke trials. Stroke, 30, 1538-1541.
  • Tilley, B. C., Marler, J., Geller, N. L., Lu, M., Legler, J., Brott, T., et al (1996). Use of a global test for multiple outcomes in stroke trails with application to the National Institute of Neurological Disorders and Stroke t-PA stroke trial. Stroke, 27, 2136-2142.
  • UK-TIA Study Group. (1988). The UK-TIA aspirin trial: Interim results. Br Med J, 296, 316-320.
  • van Swieten, J. C., Koudstaal, P. J., Visser, M. C., Schouten, H. J., van Gijn, J. (1988). Interobserver agreement for the assessment of handicap in stroke patients. Stroke, 19, 604-607.
  • Weimar, C., Kurth, T., Kraywinkel, K., Wagner, M., Busse, O., Ludwig, R., Diener, H-C. (2002). Assessment of functioning and disability after ischemic stroke. Stroke, 33, 2053-2059.
  • Weisscher, N., Vermeulen, M., Roos, Y.B., de Haan, R.J. (2008). What should be defined as good outcome in stroke trials; a modified Rankin score of 0-1 or 0-2? J Neurol, 255, 867-874.
  • Wilson, L. J. T., Hareendran, A., Hendry, A., Potter, J., Bone, I., Muir, K. W. (2005). Reliability of the Modified Rankin Scale across multiple raters: Benefits of a structured interview. Stroke, 36, 777-781.
  • Wilson, L. J. T., Harendran, A., Grant, M., Baird, T., Schultz, U. G. R., Muir, K. W., Bone, I. (2002). Improving the assessment of outcomes in stroke: Use of a structured interview to assign grades on the Modified Rankin Scale. Stroke, 33, 2243-2246.
  • Wolfe, C. D., Taub, N. A., Woodrow, E.J., Burney, P. G. (1991). Assessment of scales of disability and handicap for stroke patients. Stroke, 22, 1242-1244.