Content » Vol 48, Issue 10

Short communication

The Scandinavian Stroke Scale is equally as good as The National Institutes of Health Stroke Scale in identifying 3-month outcome

Torunn Askim, PhD1,2, Julie Bernhardt, PhD3,4, Leonid Churilov, PhD3,5 and Bent Indredavik, MD, PhD1,6

From the 1Department of Neuroscience, Faculty of Medicine, NTNU, Norwegian University of Science and Technology, 2Department of Physiotherapy, Faculty of Health and Social Science, NTNU, NorwegianUniversity of Science and Technology, Trondheim, Norway, 3Stroke Division, Florey Institutes of Neuroscience and Mental Health, 4Faculty of Health Sciences, La Trobe University, 5School of Mathematical and Geospatial Sciences, RMIT University, Melbourne, Australia and 6Stroke Unit, Department of Medicine, St Olavs Hospital, Trondheim, Norway

BACKGROUND: The National Institutes of Health Stroke Scale (NIHSS) is the first choice among stroke scales. The Scandinavian Stroke Scale (SSS) is an alternative scale that is easy to apply in the clinic.

Aim: To compare the ability of the SSS with that of the NIHSS in identifying patients who are dead or dependent at 3-month follow-up.

METHODS: A prospective study including patients with acute stroke. NIHSS and SSS measurements were obtained during index hospital stay. The receiver operating characteristics curve was used to determine the optimal dichotomization of the NIHSS and the SSS by using a modified Rankin Scale (mRS) >2 at 3-month follow-up as the criterion standard. Positive and negative predictive values (PPV and NPV) were calculated.

RESULTS: A total of 104 patients (mean age 79 years, 57.7% men) were included. Median (interquartile range (IQR)) NIHSS and SSS score were 6.0 (2.0–11.8) and 43.5 (30.0–51.0), respectively. The areas under the curve were 0.769 and 0.796 for NIHSS and SSS, respectively, χ2 (p = 0.303). The best cut-off point for NIHSS was 6/7 points (PPV = 76.2%, NPV = 69.0%) while for SSS it was 42/43 points (PPV = 71.4%, NPV = 73.2%).

CONCLUSION: The SSS was as good as the NIHSS in identifying patients who had died or were dependent at 3-month follow-up. The measurement properties of the SSS should be investigated further.

Key words: acute stroke; prediction; outcome assessment; ROC curve; neurological examination.

J Rehabil Med 2016; 00: 00–00

Correspondence address: Torunn Askim, Department of Neuroscience, NTNU, Faculty of Medicine, Postbox 8905, NO-7491 Trondheim, Norway. E-mail: torunn.askim@ntnu.no

Accepted Aug 22, 2016; Epub ahead of print Oct 13, 2016

INTRODUCTION

The National Institutes of Health Stroke Scale (NIHSS) is a well-established and extensively used measure in acute stroke treatment, which is recommended for use in clinical trials after stroke (1). The reliability of the scale is clear, but some items have consistently shown moderate to low inter-rater reliability (Kappa score less than 0.75). These items are level of consciousness, gaze, facial palsy, ataxia and dysarthria (2, 3). The ability of the NIHSS to identify 3-month outcome has shown to be superior to other stroke scales, such as the Canadian Neurological Scale and the Middle Cerebral Artery Neurological Score (1). Furthermore, a baseline NIHSS score ≤ 6 is associated with a high probability of good recovery, and the ability to predict 6-month outcome is shown to be equally good at 2 days vs 5 days or 9 days post-stroke (4, 5).

The Scandinavian Stroke Scale (SSS) is an alternative stroke scale, which is frequently used in Scandinavian countries and has recently also been validated in the Portuguese language (6). Inter-rater reliability of the items varies from excellent for conscious level, orientation and gait, (kappa 0.84, 0.86 and 1.0, respectively) to moderate for facial palsy (kappa 0.59) (6). In a multivariate logistic regression model, neurological recovery, as measured by the SSS change score during the first week after onset of stroke, was shown to be an independent predictor of good functional outcome (7, 8).

The advantage of the SSS is its simplicity, which makes it easy to perform repeated measures in the very acute phase after stroke (9). However, the ability of the SSS to identify outcome at 3 months after onset of stroke has not been validated. As the NIHSS is regarded as the gold standard measure, the purpose of the present study was to compare the SSS with the NIHSS to identify patients who are dead or dependent at 3-month follow-up. A secondary aim was to compare their ability to identify outcome in patients from different age groups and with different severity levels.

MATERIAL AND METHODS

Methods

This was a prospective cohort study with an initial assessment within 14 days after onset of stroke and a follow-up assessment conducted in the patient’s home 3 months later.

All patients admitted to the Stroke Unit at Trondheim University Hospital, Norway, with the diagnosis of stroke were eligible for inclusion, except for those with a devastating stroke receiving end-of-life palliative care. Eligible patients were included if they were able and willing to sign informed consent. Patients who were not able to give informed consent were also included if their next of kin gave oral consent to participation. The study was approved by the Regional Committee of Medical and Health Research Ethics and Norwegian Social Science Data Services.

Age, sex, time since stroke, stroke type, Oxfordshire Classification of Stroke, NIHSS, total score ranges from 0 (no symptoms) to 42 and Scandinavian Stroke Scale, total score ranges from 0 to 58 (no symptoms) were assessed at baseline. At 3 months follow-up a home visit was conducted for all surviving patients. Death or dependency 3 months after stroke was determined by modified Rankin Scale (mRS) (11). Two well-trained assessors performed the baseline assessments. The same assessor obtained the NIHSS and the SSS score. A third assessor, who was blinded to the initial assessment, performed the follow-up assessments.

Statistical analysis

Descriptive statistics were used for the baseline characteristics. Primary outcome was comparison of the area under the receiver operating characteristics (ROC) curve (AUC) for both NIHSS and SSS. The χ2 test was used to compare the AUCs for the NIHSS and the SSS. ROC curve analysis was used to determine the optimal dichotomization of the NIHSS and the SSS, respectively, using Youden’s criteria (12). The criterion standard was mRS > 2. Since the SSS ranges from 0 to 58, with 0 as the worst score, and the NIHSS ranges from 0 to 42, with 42 as the worst score, the SSS sum score was transformed to an SSS inverse score according to the following equation; SSS inverse = maximum SSS score minus original SSS score. The SSS inverse score was used in the ROC analysis to make it possible to compare the 2 scales. In all other analyses, the original score of the SSS has been reported. Sensitivity and specificity were reported for the optimal dichotomization, while positive and negative predictive values (PPV and NPV) were standardized to a 50% pre-test chance of responding. For the subgroup analysis, patients were divided into subgroups according to age (dichotomized at 80 vs 81 years) and according to stroke severity (NHISS< 8, NIHSS 8–16, NIHSS> 16). The statistical analysis was conducted using IBM SPSS Statistics version 19 and Stata v13.

RESULTS

Over 18 months, 124 patients were included in the study; 23% of all patients admitted to the stroke unit during this period. Two patients were excluded because inclusion exceeded 14 days post-stroke, one patient was found not to have stroke, and 6 patients were excluded because of incomplete data at baseline. Eleven patients were lost to follow-up, leaving a total of 104 patients in the analysis.

At baseline, the median age was 81 years and 57.7 % were male. Median (IQR) NIHSS and SSS scores were 6.0 (2.0–11.75) and 43.5 (30.0–51.0), respectively (Table I).

Table I. Baseline characteristics (n = 104)

Baseline characteristics

Men, n (%)

60 (57.7)

First ever stroke, n (%)

76 (73.1)

Age, years, mean (SD)

79.1 (9.0)

Days since stroke, median (IQR)

6.0 (4.0–9.0)

SSS score, median (IQR)

43.5 (30.0–51.0)

NIHSS score, median (IQR)

6.0 (2.0–11.75)

Severity groups, n (%)

Mild stroke (NIHSS < 8)

63 (60.6)

Moderate stroke (NIHSS 8–16)

27 (26.0)

Severe stroke (NIHSS > 16)

14 (13.5)

Stroke classification, n (%)

Infarction

81 (77.9)

Haemorrhage

21 (20.2)

Unknown

2 (1.9)

SD: standard deviation; IQR: interquartile range; SSS: Scandinavian Stroke Scale; NIHSS: National Institutes of Health Stroke Scale.

At 3-month follow-up, a total of 59 patients were classified as dependent (mRS > 2) and 45 as independent. Nine patients (15.3%) died in the 3-month follow-up interval. The ROC curve (Fig. 1) revealed an AUC of 0.769 and 0.796 for the NIHSS and the SSS, respectively, p = 0.303. The optimal dichotomization was between 6 and 7 points for the NIHSS (64.4% sensitivity and 80.0% specificity), while for the SSS it was between 42 and 43 points (69.5% sensitivity and 82.2% specificity).

13160.png

Fig. 1. Receiver operating characteristics (ROC) curves for the National Institutes of Stroke Scale (NIHSS) and the Scandinavian Stroke Scale (SSS) with the modified Rankin Scale > 2 as criterion standard.

Ninety-three patients (89.4%) were equally classified as dependent or independent with the SSS and the NIHSS, while 11 (10.6%) patients were incongruent in classification.

Table II shows the predictive values standardized to a 50% pre-test chance of being dependent. The PPVs showed a 76% chance of being dependent at 3-months post-stroke if the NIHSS score was 7 points or more, and a 71% chance of being dependent if the SSS score was 42 points or less. The NPVs were 69% and 73% for NHISS and SSS, respectively.

The results of subgroup analysis are shown in Table III, showing no significant differences between the two scales for any of the subgroups. However, both scales showed better measurement properties for patients of older age (≤ 80 years) and those with moderate strokes (NIHSS 8–16).

Table II. Predictive values for National Institutes of Health Stroke Scale (NIHSS) and Scandinavian Stroke Scale (SSS) with modified Rankin Scale (mRS)> 2 as the criterion standard

True positive
n

False positive
n

True negative
n

False negative
n

Sensitivity
%

Specificity
%

Positive predictive value*
%

Negative predictive value* %

SSS (42/43)

41

8

37

18

69.5

82.2

71.4

73.2

NIHSS (6/7)

38

9

36

21

64.4

80.0

76.2

69.0

*Predictive values were standardized to a 50% pre-test chance of responding.

Table III. Subgroup analysis

Variables

NIHSS

SSS

p-valuea

All patients (n = 104)

AUC

0.769

0.796

0.303

Sensitivity

64.4

69.5

Specificity

80.0

80.2

Optimal cut-off

6/7

42/43

Age >80 years (n = 54)

AUC

0.806

0.827

0.592

Sensitivity

68.4

76.3

Specificity

87.5

81.2

Optimal cut-off

6/7

45/46

Age ≤ 80 years (n = 50)

AUC

0.755

0.778

0.613

Sensitivity

66.7

76.2

Specificity

62.1

75.9

Optimal cut-off

5/6

42/43

NIHSS <8 (n = 63)

AUC

0.608

0.632

0.709

Sensitivity

65.4

61.5

Specificity

54.1

64.9

Optimal cut-off

2/3

48/49

NIHSS 8–16 (n = 27)

AUC

0.809

0.901

0.246

Sensitivity

78.9

84.2

Specificity

75.0

87.5

Optimal cut-off

10/11

33/34

NIHSS >16b (n = 14)

NIHSS: National Institutes of Health Stroke Scale; SSS: Scandinavian Stroke Scale; AUC: area under the curve.

aχ2 test for differences in AUC.

bReceiver operating characteristics (ROC) curves could not be drawn because no patients were classified as independent at 3-month follow-up.

DISCUSSION

This study showed that the SSS was equally as good as the NIHSS in identifying patients who were dead or dependent 3 months after stroke. Subgroup analysis showed that the measurement properties for both scales were better for patients with moderate strokes and for those of older age.

Modified Rankin Scale>2 is the most widely used definition of death and dependency after stroke (13). Using this definition as the criterion standard for the ROC analysis revealed an AUC of 0.769 for NIHSS and 0.796 for SSS, which should be regarded as adequate (14). Even though the AUC for SSS was slightly larger compared with NIHSS, there were no statistically significant differences between the two scales.

From a clinical perspective the PPV and NPV are of even greater interest. However, these two values are known to be prevalence dependent. To make comparison between scales more feasible the predictive values should be standardized to a 50% pre-test chance of responding. A NPV of 73% for the SSS and 69% for the NIHSS means that patients have a 73% chance of truly being independent if the initial SSS score is 43 points or more, or a 69% chance if the NIHSS score is 6 points or less, indicating that the SSS is superior to the NIHSS in ruling out the problem. However, the PPV was 71% for the SSS vs 76% for the NIHSS, indicating that the NIHSS might be better than the SSS in predicting the chance of being dependent at 3 months. Whether one should use the PPV or the NPV is a matter of discussion. From the patients’ perspective it might be of greater value to know the chance of being independent, while for the healthcare system it might be of greater value to know the chance of being in need of future healthcare services.

Although the measurement properties of the NIHSS and the SSS are equal in their ability to identify outcome, the superiority of the SSS lies with its simplicity and ease of use in the clinic. An example is the difference in measuring motor function between the two scales. While in the NIHSS patients are asked to keep their limb against gravity for 10 s, which means that you need a watch, in the SSS patients must keep their limb against manual resistance to get a full score. This advantage of the SSS is of particular importance when repeated measures of selected items are used to detect early neurological deterioration (9).

In the NIHSS muscle power is measured in both the affected and the unaffected limb, while SSS measures only the affected limb. This might indicate that the two scales act differently in patients with first ever and recurrent stroke. Future studies should therefore assess measurement properties in these subgroups of patients.

The present study included a rather unselected stroke sample; however, the prevalence of death (15.5% at 3 months) and dependency appears to be a little lower than in the general stroke population (15). Hence, the next step should be to validate the predictive capacity of the SSS and the NIHSS in a new patient sample.

In conclusion, the SSS and the NIHSS showed equally good measurement properties. The SSS is a simple tool, which is easy to apply in the acute clinical setting, and its measurement properties should be further investigated.

REFERENCES

1. Muir KW, Weir CJ, Murray GD, Povey C, Lees KR. Comparison of neurological scales and scoring systems for acute stroke prognosis. Stroke 1996; 27: 1817–1820.

2. Meyer BC, Lyden PD. The modified National Institutes of Health Stroke Scale: its time has come. Int J Stroke 2009; 4: 267–273.

3. Dewey HM, Donnan GA, Freeman EJ, Sharples CM, Macdonell RA, McNeil JJ, et al. Interrater reliability of the National Institutes of Health Stroke Scale: rating by neurologists and nurses in a community-based stroke incidence study. Cerebrovasc Dis 1999; 9: 323–327.

4. Adams HP, Jr, Davis PH, Leira EC, Chang KC, Bendixen BH, Clarke WR, et al. Baseline NIH Stroke Scale score strongly predicts outcome after stroke: a report of the Trial of Org 10172 in Acute Stroke Treatment (TOAST). Neurology 1999; 53: 126–131.

5. Kwakkel G, Veerbeek JM, van Wegen EE, Nijland R, Harmeling-Van der Wel BC, Dippel DW. Predictive value of the NIHSS for ADL outcome after ischemic hemispheric stroke: does timing of early assessment matter? J Neurol Sci 2010; 294: 57–61.

6. Luvizutto GJ, Monteiro TA, Braga G, Pontes-Neto OM, de Lima Resende LA, Bazan R. Validation of the Scandinavian Stroke Scale in a multicultural population in Brazil. Cerebrovasc Dis Extra 2012; 2: 121–126.

7. Jorgensen HS, Reith J, Nakayama H, Kammersgaard LP, Raaschou HO, Olsen TS. What determines good recovery in patients with the most severe strokes? The Copenhagen Stroke Study. Stroke 1999; 30: 2008–2012.

8. Christensen H, Boysen G, Truelsen T. The Scandinavian stroke scale predicts outcome in patients with mild ischemic stroke. Cerebrovasc Dis 2005; 20: 46–48.

9. Birschel P, Ellul J, Barer D. Progressing stroke: towards an internationally agreed definition. Cerebrovasc Dis 2004; 17: 242–252.

10. Gray LJ, Ali M, Lyden PD, Bath PM. Interconversion of the National Institutes of Health Stroke Scale and Scandinavian Stroke Scale in acute stroke. J Stroke Cerebrovasc Dis 2009; 18: 466–468.

11. Banks JL, Marotta CA. Outcomes validity and reliability of the modified Rankin scale: implications for stroke clinical trials: a literature review and synthesis. Stroke 2007; 38: 1091–1096.

12. Youden WJ. Index for rating diagnostic tests. Cancer 1950; 3: 32–35.

13. Stroke Unit Trialists Collaboration. Organised inpatient (stroke unit) care for stroke. Cochrane database of systematic reviews (online) 2013; 9: CD000197.

14. StrokeEngine. Statistical evaluation criteria for outcome measures. 2015 [accessed Jul 2016]. Available from: http://www.strokengine.ca/statistics/.

15. Appelros P, Jonsson F, Asberg S, Asplund K, Glader EL, Asberg KH, et al. Trends in stroke treatment and outcome between 1995 and 2010: observations from Riks-Stroke, the Swedish stroke register. Cerebrovasc Dis 2014; 37: 22–29.

Comments

Do you want to comment on this paper? The comments will show up here and if appropriate the comments will also separately be forwarded to the authors. You need to login/create an account to comment on articles. Click here to login/create an account.
Advertisement