Content » Vol 99, Issue 13


Outcome Measurements Used in Randomized Controlled Trials of Teledermatology: A Systematic Mapping Review

Aloysius Chow1, Charlene Soon1, Helen E. Smith1 and Christian J. Apfelbacher1–3

1Family Medicine and Primary Care, Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, 2Institute of Epidemiology and Preventive Medicine, University of Regensburg, and 3Institute of Social Medicine and Health Systems Research, Otto von Guericke University Magdeburg, Magdeburg, Germany


Assessment of the effectiveness of teledermatology has been hampered by the variety of outcome measures used, limiting the possibility for meta-analysis. This systematic mapping review classified the outcome measurement instruments used in randomized controlled trials of teledermatology conducted between 2008 and 2018 using the Core Outcome Measures in Effectiveness Trials taxonomy. Sixteen articles describing 12 studies were identified. Each trial used a mean of 3.7 outcome measurements (range 2–7), with a total of 55 different instruments employed. Most instruments mapped on the “skin and subcutaneous tissue outcomes” domain. The most frequently used instrument (Dermatology Life Quality Index) was used in only 3 studies. Over 60% of the instruments used did not cite any evidence of validation. This mapping review provides a list of outcome measurement instruments that can be used as a resource when designing teledermatology trials in the future and provides the foundation for the development of a core outcome set.

Key words: outcome measure; outcomes research; randomized controlled trial; teledermatology.

Accepted Sep 10, 2019; E-published Sep 10, 2019

Acta Derm Venereol 2020; 100: XX–XX.

Corr: Helen Elizabeth Smith, Lee Kong Chian School of Medicine, Nanyang Technological University, 11 Mandalay Road, Singapore 308232. E-mail:


Assessment of the effectiveness of teledermatology is challenging due to different outcome measurements utilized. This review mapped 55 different outcome measurements reported in clinical trials of teledermatology using the Core Outcome Measures in Effectiveness Trials taxonomy. Each trial used a mean of 3.7 measurements (range 2–7), and most measurements measured “skin and subcutaneous tissue outcomes”. The most frequently used measurement was used in only 3 studies. Over 60% of the measurements did not cite evidence of validation. This review provides a list of measurements for use in designing future teledermatology trials, and provides the foundation to develop a core outcome set.


Skin diseases are one of the most common reasons for patients to seek medical consultations (1). It is also recognized that there is a shortage of healthcare professionals with the relevant skills (2). Dermatology, because of its visual character, is well suited to telemedicine for patient consultations, referrals and triage, which has the potential to increase accessibility to dermatological expertise, maximize work-force potential, improve patient health outcomes, and reduce costs (3). Teledermatology consultations can be “store and forward”, with electronic digital images sent to review at a later time (also referred to as asynchronous), live and interactive (synchronous) or a combination of both (3). Literature reviews of teledermatology service evaluations have reported positive impacts, such as more rapid diagnoses (4), improved cost-effectiveness (5, 6), but also some negative impacts, such as increased referrals to secondary care (7). Systematic literature reviews of randomized controlled trials (RCTs) of telemedicine tend to be more reserved about potential benefits because of the heterogeneity in quality, design, conditions and outcomes of the studies, which in turn limits the ability to pool data (8–10).

The lack of standardization of outcome measurement instruments is a recurrent challenge when making evidence-based decisions to optimize patient care. However, this problem tends to persist, because within a tight project timeline, researchers may not have the resources to assess the range of outcome measurement instruments used previously, or to identify those that would enable direct comparisons with previous work. To address this issue the Core Outcomes Measures in Effectiveness Trials (COMET) Initiative ( is now encouraging researchers to develop and adopt the use of evidence-based core outcome sets (COS) (11). These are agreed standardized sets of outcomes that the COMET Initiative recommends to be measured and reported as a minimum in all clinical trials in a particular condition or context (12). They may also be used in audit or other forms of research. A taxonomy to classify outcomes has also been developed by the COMET Initiative, to standardize the classification of all outcomes reported. This taxonomy is also used in the classification of outcomes in COS, which further encourages the standardized reporting of outcomes (11). One important step in the development of a COS for a particular field is to identify outcome measurement instruments used previously in order to generate a long list of outcomes that can be considered candidates for inclusion into a particular COS (13) This is typically followed by some form of consensus-seeking process (such as an e-Delphi followed by a consensus meeting of all interested stakeholders) with the ultimate goal of agreeing on a COS (11). This study has been designed to identify and categorize the outcome measurement instruments reported in RCTs of teledermatology interventions.


Inclusion and exclusion criteria

This systematic mapping review protocol defined study inclusion criteria as RCTs, cluster randomized controlled or quasi-randomized trials of teledermatology interventions in which participants were patients presenting with dermatological problems. The study findings had to be published as peer-reviewed, full-text articles within the last 10 years. Studies with teledermatology services as an intervention and standard care as the control group were included. Articles were not limited to the English language or to any particular age group.

Systematic reviews, editorials, commentaries or letters were excluded. Similarly, articles were also excluded if they focused on the evaluation of a technology or a device without patient involvement, or if the intervention used was not teledermatology; for example, outreach consultant care or general practitioners (GPs) with a special interest in dermatology.

Search strategy

The search strategy was developed with the medical librarians at the Lee Kong Chian School of Medicine (Table SI) and conducted in November 2018. MEDLINE, EMBASE, CINAHL, PubMed, and Scopus were searched for articles published between 1 January 2008 and 31 December 2018. The search was complemented by hand-searching of trial registries (e.g., targeted journals (e.g. Journal of Telemedicine and Telecare, Telemedicine Journal and e-Health), the Cochrane Controlled Register of Trials, and the reference lists of all eligible studies.

Eligibility assessment

Two reviewers (AC and CS) independently screened the titles and abstracts for eligibility based on the above selection criteria. Where consensus of eligibility was not reached a third reviewer, (HES or ChA) was consulted. Full texts were obtained for all selected studies, and if study eligibility remained unclear it was again discussed with a third reviewer.

Data extraction

Characteristics of the studies (i.e. year published, study setting, country, skin disease studied, age and sex of participants), outcome reported, type of outcome reported (i.e. primary or secondary), outcome measurement instrument used, and the remarks about the validity of the outcome measurement instrument made by the studies authors’ were extracted from the eligible papers. Outcomes were mapped onto the taxonomy developed by the COMET initiative (12). If an outcome was composite and addressed several domains it was classified within each of the relevant domains.


Studies and study characteristics

After duplicates were removed, 460 potentially eligible records were identified and screened according to the protocol (Fig. 1). A final total of 16 articles based on 12 studies were included in this review. Data were extracted from all the articles, with one exception, an article in Dutch (14) that reported the same results from a study that had been published previously in English (15). Most of the studies included in this mapping review were conducted in the USA (64.3%) and the rest in Europe (i.e. Austria, France, Norway, Switzerland, and The Netherlands). The study characteristics of the studies were as follows: a total of 2,993 participants were recruited (ranging from 64 to 698 participants per study). The mean age of participants ranged from 2.7 to 63 years (but only 9 studies reported this). In the 10 studies that reported the sex of participants, slightly more men (54.3%) were included than women (45.7%). Full details are shown in Table I.

Fig. 1. Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram. RCT: randomized controlled trial.

Table I. Characteristics of studies

Outcome measurement instruments

The total number of outcome measurement instruments used was 55, with a mean of 3.7 in each article (range 2–7). Twenty-four of the outcome measurement instruments were categorized in the Life Impact COMET Core Area, with 2 of these outcome measurement instruments also categorized in the Resource Use COMET Core Area. Seventeen outcome measurement instruments were categorized in the Physiological/clinical COMET Core Area, and 11 were in the Resource Use COMET Core Area. The heterogeneity of outcome measurement instruments identified in this review is further exemplified by the fact that the Dermatology Life Quality Index (DLQI), which was the most commonly cited outcome measurement instrument, represented only 3 (5.5%) out of 55 outcome measurement instruments. These details are further detailed in Table II.

Table II. Outcomes, outcome measurement instruments, and Core Outcomes Measures in Effectiveness Trials (COMET) categories

Table II. Outcomes, outcome measurement instruments, and Core Outcomes Measures in Effectiveness Trials (COMET) categories (contd)

Table II. Outcomes, outcome measurement instruments, and Core Outcomes Measures in Effectiveness Trials (COMET) categories (contd)

Outcomes and outcome domains

The total number of primary and secondary outcomes reported in the 16 articles was 44; hence the mean number of outcomes reported was 2.9. Slightly over half (53.3%) of the articles differentiated between primary and secondary outcomes. From these studies, 11 primary outcomes and 23 secondary outcomes were reported. A total of 21 outcomes were reported in the studies that did not differentiate between primary or secondary outcomes.

As shown in Fig. 2, when mapped on the COMET taxonomy the “Skin and subcutaneous tissue outcomes” outcome domain had the largest number of outcome measurement instruments mapped in it (i.e. 34.5%). This domain is in the “Physiological/clinical skin” COMET Core Area, which includes physiological symptoms and functioning (12). Despite having the highest frequency, the relatively low percentage in this domain reflects the heterogeneity of the outcome measurement instruments reported in the studies. The most common outcome measurement instrument used that was mapped in this domain was the “Investigator Global Assessment”. The second most commonly mapped COMET outcome domain was “Delivery of Care” (i.e. 21.8%), and this domain is in the “Life Impact” Core Area. Only one of the outcome measurement instruments mapped in this domain was validated; this was the Dutch translation of the Patient Satisfaction Questionnaire III, which was modified and revalidated because only 20 out 43 questionnaire items were used. The third most commonly mapped COMET domain was “Economic” (i.e. 14.5%), and this is in the “Resource Use” COMET Core Area. The most common outcomes in this domain were direct costs and indirect costs.

Fig. 2. Core Outcomes Measures in Effectiveness Trials (COMET) Core Areas and Outcome Domains.


Of the 55 outcome measurement instruments, 61.8% did not have citations of validation in the study publication.

Safety outcomes

There were no specific safety outcomes measured in the studies; however, 4 studies briefly mentioned issues about safety and adverse events. One study reported that participants could report any adverse events that occurred on a standardized questionnaire used during the trial (16). Another study, which involved isotretinoin therapy for participants, collected reports of adverse reactions from clinicians during face-to-face or online consultations, depending on which experimental group the participant was allocated into (17). Two other studies mentioned safety only as part of the discussion of their results (18, 19).


To the best of our knowledge, this is the first systematic review of outcomes and outcome measurement instruments reported in teledermatology RCTs. Sixteen articles from 12 eligible studies were included in this review. It was notable that the included studies were either from Europe or from the USA.

Heterogeneity of outcome measurement instruments

There were 44 outcomes reported, and the majority of outcomes were categorized as skin and subcutaneous tissue outcomes. This finding is similar to what was found in another systematic review that identified and grouped outcomes of dermatology-related RCTs (20). Of the 55 outcome measurement instruments used to measure these outcomes, only 3 of these instruments were reported in different articles. This highlights the heterogeneity of outcome measurement instruments used in RCTs of teledermatology, and questions the comparability of these trials. The heterogeneity of outcome measurement instruments found in this review has also been reported in other systematic reviews (21, 22).

Validation of outcome measurement instruments

Over 60% of the reported outcome measurement instruments did not have any citation to a validation study. It was beyond the scope of this review to explore further the rigour of the validation of the outcome measurement instruments. Citing the validation or development references, helps the clinical and scientific community to make informed decisions about the outcome measurement instruments they can use for their own clinical use and research studies.


There were no specific safety outcomes measured in the reviewed trials, but 4 studies briefly mentioned issues about safety and adverse events. While safety may not be of great importance in studies focusing only on teledermatology referral processes, when the study includes treatment or procedures then safety is increasingly important. The Patient-Reported Outcomes Safety Event Reporting Consortium Guidance could be used to guide such a practice in the future (23).

Strengths and limitations

The results from this mapping review provide novel and valuable information about outcome measurement instruments that clinicians and researchers can use to make informed decisions about which outcome measurement instrument to use for treatment and research studies. Specifically, we have generated a list of the outcome measurement instruments used in recent RCTs of teledermatology and the reported validity of each measure. This information will provide a ready resource of outcome measurement instruments for researchers of teledermatology in the future. These data may also inform the process of developing a core outcome set in the future.

The current review has some limitations. First, the search was limited to trials published in the last decade. While this ensures an up-to-date overview of recent trials, many studies were excluded, as the rate of teledermatology trials conducted was low in the inclusion period of this review. Secondly, the current review excluded unpublished research reports and conference abstracts, in which additional outcome measurement instruments might have been found. Thirdly, an in-depth analysis of the validity of outcome measurement instruments used was not undertaken. The scope of this mapping review was constrained by the resources available, but future reviews could expand the current review to address the second and third limitations.

  1. Schofield JK, Fleming D, Grindlay D, Williams H. Skin conditions are the commonest new reason people present to general practitioners in England and Wales. Br J Dermatol 2011; 165: 1044–1050.
    View article    Google Scholar
  2. Kimball AB, Resneck Jr JS. The US dermatology workforce: a specialty remains in shortage. J Am Acad Dermatol 2008; 59: 741–745.
    View article    Google Scholar
  3. Pak HS, Datta SK, Triplett CA, Lindquist JH, Grambow SC, Whited JD. Cost minimization analysis of a store-and-forward teledermatology consult system. Telemed J E Health 2009; 15: 160–165.
    View article    Google Scholar
  4. Finnane A, Siller G, Mujcic R, Soyer HP. The growth of a skin emergency teledermatology service from 2008 to 2014. Australas J Dermatol 2016; 57: 14–18.
    View article    Google Scholar
  5. Snoswell C, Finnane A, Janda M, Soyer P, Whitty JA. Cost-effectiveness of store-and-forward teledermatology: a systematic review. JAMA Dermatol 2016; 152: 702–708.
    View article    Google Scholar
  6. Whited JD. Economic analysis of telemedicine and the teledermatology paradigm. Telemed J E Health 2010; 16: 223–228.
    View article    Google Scholar
  7. Ford JA, Pereira A. Does teledermatology reduces secondary care referrals and is it acceptable to patients and doctors?: a service evaluation. J Eval Clin Pract 2015; 21: 710–716.
    View article    Google Scholar
  8. Lee SWH, Chan CKY, Chua SS, Chaiyakunapruk N. Comparative effectiveness of telemedicine strategies on type 2 diabetes management: a systematic review and network meta-analysis. Sci Rep 2017; 7: 12680.
    View article    Google Scholar
  9. Ming WK, Mackillop LH, Farmer AJ, Loerup L, Bartlett K, Levy JC, et al. Telemedicine technologies for diabetes in pregnancy: a systematic review and meta-analysis. J Med Internet Res 2016; 18: e290.
    View article    Google Scholar
  10. Tchero H, Teguo MT, Lannuzel A, Rusch E. Telerehabilitation for stroke survivors: systematic review and meta-analysis. J Med Internet Res 2018; 20: e10867.
    View article    Google Scholar
  11. Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET handbook: version 1.0. Trials 2017; 18: 280.
    View article    Google Scholar
  12. Dodd S, Clarke M, Becker L, Mavergames C, Fish R, Williamson PR. A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery. J Clin Epidemiol 2018; 96: 84–92.
    View article    Google Scholar
  13. Schmitt J, Apfelbacher C, Spuls PI, Thomas KS, Simpson EL, Furue M, et al. The Harmonizing Outcome Measures for Eczema (HOME) roadmap: a methodological framework to develop core sets of outcome measurements in dermatology. J Invest Dermatol 2015; 135: 24–30.
    View article    Google Scholar
  14. Eminovi? NN, de Keizer NN, Wyatt JJ, ter Riet GG, Peek NN, van Weert HH, et al. [Teledermatology and fewer visits to dermatologist: A cluster sampling, randomized study]. Huisarts en Wetenschap 2010; 53: 76–83 (in Dutch).
    View article    Google Scholar
  15. Eminovi? N, de Keizer NF, Wyatt JC, ter Riet G, Peek N, van Weert HC, et al. Teledermatologic consultation and reduction in referrals to dermatologists: a cluster randomized controlled trial. Arch Dermatol 2009; 145: 558–564.
    View article    Google Scholar
  16. Armstrong AW, Johnson MA, Lin S, Maverakis E, Fazel N, Liu FT. Patient-centered, direct-access online care for management of atopic dermatitis a randomized clinical trial. JAMA Dermatology 2015; 151: 154–160.
    View article    Google Scholar
  17. Frühauf J, Kröck S, Quehenberger F, Kopera D, Fink-Puches R, Komericki P, et al. Mobile teledermatology helping patients control high-need acne: a randomized controlled trial. J Eur Acad Dermatol Venereol 2015; 29: 919–924.
    View article    Google Scholar
  18. Ford AR, Gibbons CM, Torres J, Kornmehl HA, Singh S, Young PM, et al. Access to dermatological care with an innovative online model for psoriasis management: results from a randomized controlled trial. Telemed J E Health 2019; 25: 619–627.
    View article    Google Scholar
  19. Tandjung R, Badertscher N, Kleiner N, Wensing M, Rosemann T, Braun RP, et al. Feasibility and diagnostic accuracy of teledermatology in Swiss primary care: process analysis of a randomized controlled trial. J Eval Clin Pract 2015; 21: 326–331.
    View article    Google Scholar
  20. Rönsch H, Apfelbacher C, Brans R, Matterne U, Molin S, Ofenloch R, et al. Which outcomes have been measured in hand eczema trials? A systematic review. Contact Dermatitis 2019; 80: 201–207.
    View article    Google Scholar
  21. Clark AK, Bosanac S, Ho B, Sivamani RK. Systematic review of mobile phone-based teledermatology. Arch Dermatol Res 2018; 310: 675–689.
    View article    Google Scholar
  22. Warshaw EM, Hillman YJ, Greer NL, Hagel EM, MacDonald R, Rutks IR, et al. Teledermatology for diagnosis and management of skin conditions: a systematic review. J Am Acad Dermatol 2011; 64: 759–772.
    View article    Google Scholar
  23. Banerjee AK, Okun S, Edwards IR, Wicks P, Smith MY, Mayall SJ, et al. Patient-reported outcome measures in safety event reporting: PROSPER consortium guidance. Drug Safety 2013; 36: 1129–1149.
    View article    Google Scholar
  24. Bergmo TS, Wangberg SC, Schopf TR, Solvoll T. Web-based consultations for parents of children with atopic dermatitis: results of a randomized controlled trial. Acta Paediatrica 2009; 98: 316–320.
    View article    Google Scholar
  25. Chambers CJ, Parsi KK, Schupp C, Armstrong AW. Patient-centered online management of psoriasis: a randomized controlled equivalency trial. J Am Acad Dermatol 2012; 66: 948–953.
    View article    Google Scholar
  26. Datta SK, Warshaw EM, Edison KE, Kapur K, Thottapurathu L, Moritz TE, et al. Cost and utility analysis of a store-and-forward teledermatology referral system: a randomized clinical trial. JAMA Dermatology 2015; 151: 1323–1329.
    View article    Google Scholar
  27. Kornmehl H, Singh S, Johnson MA, Armstrong AW. Direct-access online care for the management of atopic dermatitis: a randomized clinical trial examining patient quality of life. Telemed J E Health 2017; 23: 726–732.
    View article    Google Scholar
  28. Piette E, Nougairède M, Vuong V, Crickx B, Tran VT. Impact of a store-and-forward teledermatology intervention versus usual care on delay before beginning treatment: a pragmatic cluster-randomized trial in ambulatory care. J Telemed Telecare 2017; 23: 725–732.
    View article    Google Scholar
  29. van Os-Medendorp H, Koffijberg H, Eland-de Kok PC, Van der Zalm A, de Bruin-Weller MS, Pasmans SG, et al. E-health in caring for patients with atopic dermatitis: a randomized controlled cost-effectiveness study of internet-guided monitoring and online self-management training. Br J Dermatol 2012; 166: 1060–1068.
    View article    Google Scholar
  30. Watson AJ, Bergman H, Williams CM, Kvedar JC. A Randomized trial to evaluate the efficacy of online follow-up visits in the management of acne. Arch Dermatol 2010; 146: 406–411.
    View article    Google Scholar
  31. Whited JD, Warshaw EM, Edison KE, Kapur K, Thottapurathu L, Raju S, et al. Effect of store and forward teledermatology on quality of life: a randomized controlled trial. JAMA Dermatology 2013; 149: 584–591.
    View article    Google Scholar
  32. Whited JD, Warshaw EM, Kapur K, Edison KE, Thottapurathu L, Raju S, et al. Clinical course outcomes for store and forward teledermatology versus conventional consultation: a randomized trial. J Telemed Telecare 2013; 19: 197–204.
    View article    Google Scholar
Supplementary content
Table SI
Table SI