Content » Vol 93, Issue 4

Investigative Report

Interobserver Variability of Histopathological Prognostic Parameters in Cutaneous Malignant Melanoma: Impact on Patient Management

Hanna Eriksson1,2, Margareta Frohm-Nilsson1 Mari-Anne Hedblad3, Henrik Hellborg4, Lena Kanter-Lewensohn2, Kamilla Krawiec4, Barbro Lundh Rozell5,6, Eva Månsson-Brahme2 and Johan Hansson2

1Department of Medicine, Unit of Dermatology, 2Department of Oncology-Pathology, 3Dermatologic Diagnostic Centre/DDC, Department of Medicine, Unit of Dermatology, Karolinska Institutet, Karolinska University Hospital Solna, Stockholm, 4Regional Cancer Centre, Karolinska University Hospital Solna, Stockholm, 5Department of Pathology and Cytology, Division of Pathology, Karolinska Institutet, Karolinska University Hospital Huddinge, Huddinge, Sweden, and 6Department of Immunology, Genetics and Pathology, Uppsala University, Rudbeck Laboratory, Uppsala, Sweden

Clinical management of primary cutaneous melanomas is based on histopathological staging of the tumour. The aim of this study was to investigate, in a non-selected population in clinical practice, the agreement rate between general pathologists and pathologists experienced in melanoma in terms of the evaluation of histopathological prognostic parameters in cutaneous malignant melanomas, and to what extent the putative variability affected clinical management. A total of 234 cases of invasive cutaneous malignant melanoma were included in the study from the Stockholm–Gotland Healthcare Region in Sweden. Overall interobserver variability between a general pathologist and an expert review was 68.8–84.8%. Approximately 15.5% of melanomas ≤ 1 mm were re-classified either as melanoma in situ or melanomas >1 mm after review. In conclusion, review by a pathologist experienced in melanoma resulted in a change in recommendations about surgical excision margins and/or sentinel node biopsy in subgroups of T1 melanomas. Key words: melanoma; pathology; prognosis; interobserver; variability; treatment.

(Accepted September 25, 2012.)

Acta Derm Venereol 2013; 93: XX–XX.

Hanna Eriksson, Department of Medicine, Unit of Dermatology, Karolinska Institutet, Karolinska University Hospital Solna, SE-176 76 Stockholm, Sweden. E-mail: hanna.eriksson.4@ki.se

In patients with primary localized cutaneous malignant melanoma (CMM), the information collected in the histopathological report on the primary tumour plays a major role in the diagnosis, treatment strategy and prediction of prognosis. Tumour staging may determine the extent of primary surgery and whether sentinel node (SN) biopsy is required. It has been reported recently that histopathology also provides important information regarding the efficacy of adjuvant therapy (1). Moreover, histopathological staging of the primary CMM is of importance when comparing trends in incidence and survival in epidemiological studies on CMM. Several histopathological variables are known to be independent prognostic factors for survival in CMM. Tumour thickness, measured according to Breslow, and tumour ulceration are considered the most powerful prognostic factors for primary localized CMM (2–4). The mitotic rate has recently replaced the level of invasion according to Clark in the American Joint Committee on Cancer (AJCC) 2009 Melanoma Staging and Classification in defining T1 sub-categories compared with the 2001 AJCC (4, 5). The prognosis may also be influenced by a number of other histopathological features, such as tumour regression, lymphocyte infiltration and histological tumour type, but the results of large studies are not consistent (2, 5, 6).

Pathology reports on prognostic characteristics of the tumour are subject to interobserver variability. Several previous studies analyzing interobserver variability indicate that the 2 major prognostic factors in localized CMM, tumour thickness and ulceration, have the highest reproducibility (7–16). The assessment of other prognostic factors, such as level of invasion, mitotic rate and histological type, tend to be less consistent, and the interobserver concordance varies from low to intermediate (9, 12–15, 17–18).

In 1976 the Swedish Melanoma Study Group (SMSG) was established and issued national guidelines for CMM diagnosis, treatment and follow-up (19). The guidelines included recommendations about referral, diagnosis, staging, treatment, registration, and follow-up of all CMM patients in Sweden. Regional melanoma groups were established in each of the 6 Swedish healthcare regions, where regional CMM care programmes were gradually implemented to ensure a uniform standard of care for all patients. In addition, Regional Melanoma Registries were affiliated with the care programmes. Uniquely to the Stockholm–Gotland Region, all histopathological slides primarily analyzed by a general pathologist are routinely reviewed by a pathologist with expertise in CMM.

We report here the results of a population-based cohort including all incident CMM cases prospectively registered during 2006 in the Regional Melanoma Register of the Stockholm–Gotland Health Care Region (population, approximately 2 million).

The aim of this study was to determine, in a daily clinical setting, the agreement rate of the histopathological evaluation of CMM made by general pathologists compared with pathologists with expertise in CMM. A further aim was to determine to what extent interobserver variability influenced patient management. The studied parameters were: tumour thickness according to Breslow, Clark’s level of invasion, histological tumour type, ulceration, as well as to what extent the putative variability affected recommendations on performance of a SN biopsy and total surgical margins.

MATERIALS AND METHODS

Study population

From 1 January through 31 December 2006, a total of 681 cases of CMM were diagnosed and reported to the Swedish Cancer Registry from the Stockholm–Gotland Health Care Region. Of these, 664 cases were registered in the Regional Melanoma Register, and were thus included in the study, which was covering 97.5% of all cases. Clinical and histopathological data were registered according to the care programme. The staging at the time was performed according to the 2001 final version of the AJCC staging system for CMM (20). Concordance and disconcordance between pathologists were registered for each of the histopathological parameters included.

The study was approved by the Regional Ethics Committee at Karolinska Institutet, Stockholm, Sweden.

Swedish Cancer Registry and Regional Melanoma Registries

In Sweden, the reporting of all new cases of cancer is compulsory for clinicians, pathologists and cytologists diagnosing malignant tumours. The estimated overall coverage rate in the Swedish Cancer Registry is approximately 96% depending on cancer site, gender, age and type of institution treating the patient (21). Approximately 99% of all malignant tumours are confirmed by histopathological examination. The Swedish Cancer Registry does not include information on all histopatho­logical characteristics, treatment and follow-up.

In each healthcare region the Regional Melanoma Registries are responsible for the registration and coding of each new CMM case, as well as for monitoring and quality assurance work. Individual information on clinical data, tumour characteristics, surgical treatment and follow-up are collected continuously and prospectively registered. Data are reported annually to the Swedish Melanoma Register.

Management guidelines

The regional CMM care programme was implemented in the Stockholm Gotland area in 1976. According to the guidelines, all histopathological slides primarily analyzed by a general pathologist should routinely be reviewed by a pathologist with special expertise in CMM. If the primary histopathological analyzis was performed by a pathologist with expertise in CMM, review was not performed. CMM pathologists at Karolinska University Hospital were responsible for the review of all slides. Decisions on surgical treatment and a SN biopsy are based on the results of the expert classification of the tumour.

According to national management guidelines, surgical treatment of primary CMMs with a thickness of 1.0 mm or less (T1-CMM) is performed with an excision of skin and subcutaneous tissue down to the underlying muscular fascia with 1 cm free lateral margins. In T1b CMM also a SN biopsy is considered. Thicker CMMs (T2–T4) are excised with a 2 cm margin and usually a sentinel biopsy is performed. In routine practice, and also in this study, margins of excision refer to the surgical margins measured by the operating physician, not to the pathology report. During 2006 the margins were as described above.

Study parameters and histopathological evaluation

For each patient, information on study parameters was obtained from the Regional Melanoma Register.

In the histopathological evaluation the following parameters were recorded: (i) tumour thickness according to Breslow, in mm (categorized as: ≤ 1.0, > 1.0–2.0, > 2.0–4.0, > 4.0 mm, in situ tumour, data not reported (in the primary report) and not classifiable) (22); (ii) level of invasion according to Clark level (I–V, data not reported, not classifiable) (23); (iii) histological type (superficial spreading melanoma (SSM), lentigo malignant melanoma (LMM), nodular melanoma (NM), acral lentiginous melanoma (ALM), data not reported (in the primary report), other) (23, 24); and (iv) ulceration (present, absent, not reported (in the primary report), in situ tumour, not classifiable, not classified (after review) (4). Ulceration status is described in subclassification as a/b (T1a/b CMMs without/with ulceration or Clark IV or V, respectively; T2a/b–T4a/b CMMs without/with ulceration, respectively (20).

Surgical management was reported as recommended surgical margins based on the primary report (1 cm, 2 cm, not specified because of not reported histopathological data) compared with the recommended surgical margins after review (1 cm, 2 cm, not specified because of non-classifiable histopathological data).

SN biopsy was reported as recommended SN biopsy based on the primary report (yes, no/not specified because of not reported histopathological data) compared with recommended SN biopsy after review (yes, no/not specified because of non-classifiable histopathological data).

Agreement was measured, analyzed and reported in percentages of agreement vs. disagreement as well as kappa-values, and shown in more detail in cross-tabulations.

RESULTS

For this study, we excluded all cases primarily analyzed by a pathologist with melanoma expertise (n = 160, 24.1%) and the cases where at the time of the study the slides had not yet been reviewed by a pathologist with melanoma expertise (n = 187, 27.5%). Of a total of 664 pathology reports on CMM, 317 (47.7%) cases remained, which had been both analyzed primarily by a general pathologist and reviewed by a pathologist with melanoma expertise. Eighty-three cases of melanoma in situ were studied separately. Overall, 234 cases (35.2%) of primarily reported invasive melanomas were thus included in the study (Table SI; available from http://www.medicaljournals.se/acta/content/?doi=10.2340/00015555-1517).

Histopathological review

Tables I–III show the results of the histopathological expert review compared with the primary report concerning tumour thickness, ulceration and level of invasion. Cases lacking reported histopathological parameters were excluded from the analyzes. The best agreement was achieved for Breslow thickness, with an overall agreement of 86.5% (к = 0.806) (Table I and SII; available from; http://www.medicaljournals.se/acta/content/?doi=10.2340/00015555-1517). In a subgroup of CMMs classified as ≤ 1.0 mm by the general pathologist, 15.5% (16 of 103 cases) were re-classified either as in situ or CMM > 1 mm. The level of agreement was good with respect to ulceration, with 85.6% (к = 0.690) overall agreement when reported (Table II and SII). However, in 52.1% (122 of 234 cases), ulceration was not reported in the primary histopathological report (Table II and SII). The agree­ment was fair (68.8%, к = 0.561) with respect to level of invasion, with the most pronounced discrepancies within invasion level II and III (Table III and SII). Histological type had an overall good agreement between pathologists of 78.7% (к = 0.664) (Table SII and SIII; available from: http://www.medicaljournals.se/acta/content/?doi=10.2340/00015555-1517). Tables IV and SIV (available from: http://www.medicaljournals.se/acta/content/?doi=10.2340/00015555-1517) show the frequencies of agreement between the primary evaluation and review concerning ulceration and level of invasion within different thickness groups. Disagreement will mainly affect the management of T1 CMMs according to the National Management Guidelines.

We also analyzed the change in primary diagnosis from in situ to a final diagnosis of invasive CMM after the review and we found a change in 17 of 83 cases (20.5%, data not shown). Of these, 16 cases were classified as T1a CMMs and one case was classified as T1b. This is to be compared with a change from invasive to in situ CMM in 7.3% of the cases after the review (Table III).

Table I. Results of the histopathological review of 234a cases of invasive cutaneous malignant melanoma (CMM) compared with the results of the primary histopathological report concerning Breslow thickness

Breslow thickness, primary report (mm)

Tumour thickness, reviewed report (mm)

All

≤ 1.0

> 1.0–2.0

> 2.0–4.0

> 4.0

In situ tumour

Agreement n (%)

≤ 1.0

103

87

8

8

87 (84.5)

> 1.0–2.0

49

44

4

1

44 (89.8)

> 2.0–4.0

41

3

34

4

34 (82.9)

> 4.0

21

1

20

20 (95.2)

185 (86.4)

aNot reported data (n = 20) excluded from the analyzes.

Table II. Results of the histopathological review of 234a cases of invasive cutaneous malignant melanoma compared with the results of the primary histopathological report concerning ulceration

Ulceration, primary report

Ulceration, reviewed report

All

Present

Absent

Not classifiable

In situ tumour

Agreement n (%)

Present

36

27

7

2

27 (75.0)

Absent

72b

4

67

1b

66 (93.0)

Not classifiable

4

3

1

1 (25.0)

94 (85.6)

aNot reported data (n = 122) excluded from the analyzes.

bIn situ CMM reported after review (n = 1) and the corresponding tumour in the primary report (n = 1) were not included in the analyzes since ulceration is not reported in pre-invasive lesions.

Table III. Results of the histopathological review of 234a cases of invasive cutaneous malignant melanoma compared with the results of the primary histopathological report concerning level of invasion according to Clark

Clark level, primary report

Clark level, reviewed report

All

I

II

III

IV

V

Not classifiable

Agreement n (%)

II

59

5

34

18

2

34 (57.6)

III

77

3

4

45

24

1

45 (58.4)

IV

69

1

1

61

6

61 (88.4)

V

10

2

8

8 (80.0)

148 (68.8)

aNot reported data (n = 19) excluded from the analyzes.

Table IV. Results of the histopathological review of ulceration in 108 of 234 cases of invasive cutaneous malignant melanoma by the primary report of tumour thickness according to Breslow in relation to the results of the histopathological primary report of ulceration. The analyzis could only be performed where both ulceration and tumour thickness were reported in the primary histopathological report

Breslow thickness, primary report

Ulceration, primary report

Ulceration, reviewed report

All

Present

Absent

Not classifiable

In situ tumour

Agreement n (%)

T1 melanomas (≤ 1.0 mm)

Present

4

2

2

 

 

2 (50.0)

Absent

33

1

31

1

31 (93.9)

Not classifiable data

3

3

0 (0.0)

T2–T4 melanomas (> 1.0 mm)

Present

30

24

5

1

 

24 (80.0)

Absent

38

3

35

35 (92.1)

Clinical consequences of the interobserver variability

Overall, the recommended surgical margins and performance of SN biopsy were similar whether based on initial histopathological evaluation or expert review (agreement 92.3% and 91.9%, respectively) (Tables V and VI). Thus, the surgical recommendations after review resulted in a change in recommended surgical margins from 1 to 2 cm in only 9 of 114 cases (7.9%). In 7 of 8 cases (87.5%) with no reported tumour thickness in the primary histopathological evaluation, the review resulted in a recommendation for surgical margins of 1 cm in 2 cases and 2 cm in 5 cases (Table V). In 15.7% of the cases (18 of 115 cases) in which SN biopsy was not recommended or not determined according to the primary report, the review resulted in a recommendation to perform SN biopsy (Table VI).

Table V. Recommended surgical margins of 234 cases of invasive cutaneous malignant melanoma according to the primary histo­pathological report vs. the reviewed histopathological report

Recommended surgical margins (cm), primary report

Recommended surgical margins (cm), reviewed report

All

1

2

Not specified because of non-classifiable histopathological data

Agreement n (%)

1

114

104

9

1

104 (91.2)

2

112

0

111

1

111 (99.1)

Not specified because of not reported histopathological data

8

2

5

1

1 (12.5)

216 (92.3)

Table VI. Recommended sentinel node biopsy of 234 cases of invasive cutaneous malignant melanoma according to the primary histopathological report vs. the reviewed histopathological report

Recommended sentinel node biopsy, primary report

Recommended sentinel node biopsy, reviewed report

All

Yes

No/Not specified because of non-classifiable histopathological data

Agreement n (%)

Yes

119

118

1

118 (99.2)

No/ Not specified because of not reported histopathological data

115

18

97

97 (84.3)

215 (91.9)

DISCUSSION

Histopathological evaluation of CMMs is the basis for staging the tumour and provides critical information on therapeutic recommendations. This study corroborates how an expert review of the primary histopathological report may lead to significant changes in tumour classification, resulting in a change in clinical management. Even an apparently small change in the histopathological assessment may result in a change in surgical management, particularly among T1 lesions.

Our study analyzes a population-based cohort of non-selected patients where data on tumour characteristics were collected prospectively and continuously during 2006 as a clinical routine. The concordance in Breslow thickness, the most important histopathological prognostic marker, was generally excellent, but in a subgroup of CMMs with thickness ≤ 1.0 mm we found a variability of 15.5% between general and CMM pathologists. The reviews led to a change in recommended management mainly among patients with T1 CMMs (≤ 1.0 mm), resulting in a recommended widening of surgical margins in 9 cases and altered expected prognosis.

The overall concordance was high (86.4%) when comparing the results for tumour thickness between the pathologists, which accords with previous data (7–16). However, there is still a potential variability of almost 20% between pathologists and this seems to affect T1 CMMs to a greater extent. In addition, a substantial proportion of the in situ CMMs were re-classified as invasive tumours following review (data not shown), which is a more pronounced variability than reported previously (25). Difficulty in determining the true depth is a common problem, particularly in the presence of a co-existing benign melanocytic naevus or a periadnexal extension of the tumour (15).

The cohort included tumours not classified for ulceration (52.1%) in the primary histopathological report. However, compared with previous studies, the observed interobserver reproducibility of ulceration was intermediate to high (84.8%) in our cohort, taking only the classified CMMs in the primary report into account (9, 12–16). Ulceration not being reported may be explained by difficulties in assessing this parameter for pathologists without extensive experience of analyzing CMMs. One difficulty for pathologists is to distinguish between a traumatic and a non-traumatic ulceration due to epidermal invasion and destruction by neoplastic cells in the CMM (26). It may also reflect an inconsistency in reporting all of the pathological information required for making management decisions. Disconcordance in reporting ulceration may affect both the management of T1 CMMs and the prognostic information to the patient.

It is established that the survival rates are decreased for patients with ulcerated primary CMMs compared with patients with non-ulcerated tumours in the same T-category and this also affects the choice of management mainly in T1 CMMs (5). Width of excision margins and SN biopsy in localized primary CMMs are primarily based on tumour thickness. However, in T1b tumours lateral excision margins of 1 cm as well as SN biopsy are recommended according to the national management guidelines. Other markers may increase the importance of ulceration as a prognostic parameter in future. For example, in a previous study by our group BRAF codon 600 mutations were associated with significantly lower survival in patients with ulcerated primary tumours, whereas no effect on survival was described in ulcerated tumours without BRAF mutations (27).

The lower concordance for Clark’s level of invasion seen in our study between general pathologists and pathologists with expertise in CMM is consistent with previous studies showing a low to intermediate agree­ment between pathologists (7–10, 12, 13, 15, 16, 28).

Previous published studies comparing interobserver variability of histopathological factors in CMM have shown variability among pathologists (7–17, 25). The studies vary in methodology and only a few were performed in a similar clinical setting to that of our study. Stockholm–Gotland is the only region in Sweden where it is possible to analyze the effect of a review in clinical practice, as this is not routinely performed in other parts of Sweden. This gives the pathologists reviewing the CMMs more experience of evaluating difficult cases which have not been studied extensively previously. For example, in the observations by Murali et al. (16), matched pairs of pathology records drawn from routine clinical practice at the Sydney Melanoma Unit were reviewed retrospectively, evaluating the concordance between general and CMM specialized pathologists. Kappa-values analyzed in the Australian setting were к = 0.883 for tumour thickness and к = 0.832 reported for ulceration. This corresponds to к = 0.806 and к = 0.690, respectively, in our study. In the Australian study the agreement was considerably higher with respect to ulceration. Differences between the results might be partially explained by experienced pathologists being more attentive when reviewing a case retrospectively. Santillan et al. (25) found alterations in diagnosis, staging and management after retrospectively reviewing T1 and in situ CMMs referred to the multidisciplinary clinic H. Lee Moffitt Cancer Center from 2006 to 2009. Shoo et al. (28) have analyzed the discordance rate of CMM diagnosis that was routinely re-evaluated when referred to the UCSF Pigmented Lesion Clinic from outside pathologists during 2006 and 2007, concluding that disconcordance can be high between pathologists interpreting melanocytic neoplasms.

The limitations of this study include the lack of certification of pathologists specialized in CMMs; thus the definition of a pathologist with special CMM expertise is informal. In our study the presence/absence of at least 1 mitosis per mm2 was not reported in 2006, and data are thus missing. However, the impact of any uncertainty with respect to this parameter is limited to T1 CMMs, further emphasizing our recommendation to perform an expert histopathology review of thin tumours.

In summary, the present study showed that the interobserver concordance between general pathologists and pathologists with expertise in CMM is good overall in clinical practice. Although it is the most reproducible parameter in our study, Breslow thickness was altered by the review in subgroups of T1 tumours and in situ CMMs. This may also concern CMMs with a thickness just above 1 mm. Moreover, T1 tumours were also, to a larger extent, under consideration for a different surgical approach after the review. Even a small change in tumour thickness, as well as discrepant assessment of ulceration, may result in altered management and prognosis. As management protocols are based on histopathological classification of the primary tumour, we therefore suggest that a review by a CMM pathologist is performed routinely in order to increase quality assurance. The review should, at least, be performed for T1 CMMs and CMMs with a thickness just above 1 mm primarily analyzed by a general pathologist. Our results also highlight the importance of using uniform pathology protocols that clearly state the parameters needed for the histopathological evaluation of CMM in order to avoid missing data in the primary report. Considering the fact that the majority of the CMMs diagnosed in the Stockholm–Gotland region during 1990–2008 were T1 tumours, even minor disagreement about the histopathological prognostic parameters may affect many patients (29). Similar findings may also affect the quality of studies based on register data from pathology reports. Further analyzes of the histopathological reports of severe dysplastic naevi and CMMs with thickness just above 1 mm is warranted to clarify possible changes in diagnosis in these subgroups of pigmented lesions.

ACKNOWLEDGEMENTS

The authors would like to thank the members of the Stockholm Melanoma Group for supplying data, and Ms Mari Lindeborg at the Oncologic Centre at Karolinska University Hospital Solna for excellent management of the data. Research funding was received from The Swedish Cancer Society, The Radiumhemmet Research Funds and The Genomel Network of Excellence (European Union, FP6).

The authors declare no conflicts of interest.

REFERENCES

1. Eggermont AM, Suciu S, Testori A, Kruit WH, Marsden J, Punt CJ, et al. Ulceration and stage are predictive of interferon efficacy in melanoma: results of the phase III adjuvant trials EORTC 18952 and EORTC 18991. Eur J Cancer 2011; 48: 218–225.

2.
Masback A, Olsson H, Westerdahl J, Ingvar C, Jonsson N. Prognostic factors in invasive cutaneous malignant melanoma: a population-based study and review. Melanoma Res 2001; 11: 435–445.

3.
Lindholm C, Andersson R, Dufmats M, Hansson J, Ingvar C, Moller T, et al. Invasive cutaneous malignant melanoma in Sweden, 1990–1999. A prospective, population-based study of survival and prognostic factors. Cancer 2004; 101: 2067–2078.

4.
Balch CM, Soong SJ, Gershenwald JE, Thompson JF, Reintgen DS, Cascinelli N, et al. Prognostic factors analysis of 17,600 melanoma patients: validation of the American Joint Committee on Cancer melanoma staging system. J Clin Oncol 2001; 19: 3622–3634.

5.
Balch CM, Gershenwald JE, Soong SJ, Thompson JF, Atkins MB, Byrd DR, et al. Final version of 2009 AJCC melanoma staging and classification. J Clin Oncol 2009; 27: 6199–6206.

6.
Thorn M, Ponten F, Bergstrom R, Sparen P, Adami HO. Clinical and histopathologic predictors of survival in patients with malignant melanoma: a population-based study in Sweden. J Natl Cancer Inst. [Research Support, Non-US Gov’t]. 1994; 86: 761–769.

7.
Prade M, Sancho-Garnier H, Cesarini JP, Cochran A. Difficulties encountered in the application of Clark classification and the Breslow thickness measurement in cutaneous malignant melanoma. Int J Cancer 1980; 26: 159–163.

8.
Holman CD, James IR, Heenan PJ, Matz LR, Blackwell JB, Kelsall GR, et al. An improved method of analysis of observer variation between pathologists. Histopathology 1982; 6: 581–589.

9.
Heenan PJ, Matz LR, Blackwell JB, Kelsall GR, Singh A, ten Seldam RE, et al. Inter-observer variation between pathologists in the classification of cutaneous malignant melanoma in western Australia. Histopathology 1984; 8: 717–729.

10.
Colloby PS, West KP, Fletcher A. Observer variation in the measurement of Breslow depth and Clark’s level in thin cutaneous malignant melanoma. J Pathol 1991; 163: 245–250.

11.
Krieger N, Hiatt RA, Sagebiel RW, Clark WH Jr, Mihm MC Jr. Inter-observer variability among pathologists’ evaluation of malignant melanoma: effects upon an analytic study. J Clin Epidemiology 1994; 47: 897–902.

12.
Lock-Andersen J, Hou-Jensen K, Hansen JP, Jensen NK, Sogaard H, Andersen PK. Observer variation in histological classification of cutaneous malignant melanoma. Scand J Plast Reconstr Surg Hand Surg 1995; 29: 141–148.

13.
Corona R, Mele A, Amini M, De Rosa G, Coppola G, Piccardi P, et al. Interobserver variability on the histopathologic diagnosis of cutaneous melanoma and other pigmented skin lesions. J Clin Oncol 1996; 14: 1218–1223.

14.
Brochez L, Verhaeghe E, Grosshans E, Haneke E, Pierard G, Ruiter D, et al. Inter-observer variation in the histopathological diagnosis of clinically suspicious pigmented skin lesions. J Pathol 2002; 196: 459–466.

15.
Scolyer RA, Shaw HM, Thompson JF, Li LX, Colman MH, Lo SK, et al. Interobserver reproducibility of histopathologic prognostic variables in primary cutaneous melanomas. Am J Surg Pathol 2003; 27: 1571–1576.

16.
Murali R, Hughes MT, Fitzgerald P, Thompson JF, Scolyer RA. Interobserver variation in the histopathologic reporting of key prognostic parameters, particularly clark level, affects pathologic staging of primary cutaneous melanoma. Ann Surg 2009; 249: 641–647.

17.
Larsen TE, Little JH, Orell SR, Prade M. International patho­logists congruence survey on quantitation of malignant melanoma. Pathology 1980; 12: 245–253.

18.
Suffin SC, Waisman J, Clark WH, Jr, Morton DL. Comparison of the classification by microscopic level (stage) of malignant melanoma by three independent groups of pathologists. Cancer 1977; 40: 3112–3114.

19. Hansson J. The Swedish Melanoma Study Group – a brief history. Arch Oncol 2005; 13 Suppl 1: 72–74.

20.
Balch CM, Buzaid AC, Soong SJ, Atkins MB, Cascinelli N, Coit DG, et al. Final version of the American Joint Committee on Cancer staging system for cutaneous melanoma. J Clin Oncol 2001; 19: 3635–3648.

21.
Barlow L, Westergren K, Holmberg L, Talback M. The completeness of the Swedish Cancer Register: a sample survey for year 1998. Acta Oncol 2009; 48: 27–33.

22.
Breslow A. Thickness, cross-sectional areas and depth of invasion in the prognosis of cutaneous melanoma. Ann Surg 1970; 172: 902–908.

23.
Clark WH, Jr., From L, Bernardino EA, Mihm MC. The histogenesis and biologic behavior of primary human malignant melanomas of the skin. Cancer Res 1969; 29: 705–727.

24.
Arrington JH, 3rd, Reed RJ, Ichinose H, Krementz ET. Plantar lentiginous melanoma: a distinctive variant of human cutaneous malignant melanoma. Am J Surg Pathol 1977; 1: 131–143.

25. Santillan AA, Messina JL, Marzban SS, Crespo G, Sondak VK, Zager JS. Pathology review of thin melanoma and melanoma in situ in a multidisciplinary melanoma clinic: impact on treatment decisions. J Clin Oncol 2010; 28: 481–486.

26.
Ruiter DJ, Spatz A, van den Oord JJ, et al. Pathologic staging of melanoma. Semin Oncol 2002; 29: 370–381.

27.
Jovanovic B, Krockel D, Linden D, Nilsson B, Egyhazi S, Hansson J. Lack of cytoplasmic ERK activation is an independent adverse prognostic factor in primary cutaneous melanoma. J Invest Dermatol 2008; 128: 2696–2704.

28.
Shoo BA, Sagebiel RW, Kashani-Sabet M. Discordance in the histopathologic diagnosis of melanoma at a melanoma referral center. J Am Acad Dermatol 2010; 62: 751–756.

29.
[National quality registry for skin melanoma. Years of diagnosis 1990–2008]. http://www.lio.se/pages/26092/Melanom_NatKvalReg1990-2008.pdf (in Swedish).