Using AI as a Diagnostic Decision Support Tool to Help the Diagnosis of Skin Disease in Primary Healthcare in Catalonia

Using AI as a Diagnostic Decision Support Tool to Help the Diagnosis of Skin Disease in Primary Healthcare in Catalonia

Using Artificial Intelligence as a Diagnostic Decision Support Tool to Help the Diagnosis of Skin Disease in Primary Healthcare in Catalonia

Background: Dermatological conditions are a relevant health problem. Machine learning models are increasingly being applied to dermatology as a diagnostic decision support tool using image analysis, specially for skin cancer detection and classification. Objective: The objective of this study is to perform a prospective validation of an image analysis ML model, which is capable of screening 44 different skin disease types, comparing its diagnostic capacity with that of General Practitioners (GPs) and dermatologists. Methods: In this prospective study 100 consecutive patients who visit a participant GP with a skin problem in central Catalonia will be recruited, data collection is planned to last 7 months. Skin diseases anonymized pictures will be taken and introduced in the ML model interface, which will return top 5 accuracy diagnosis. The same image will be also sent as a teledermatology consultation, following the current workflow. GP, ML model and dermatologist/s assessments will be compared to calculate the precision, sensitivity, specificity and accuracy of the ML model.

A secure anonymous stand alone web interface that is compatible to any mobile device will be integrated with the Autoderm API. The study conducted in this project will consist in a prospective study aimed to evaluate the ML model performance, comparing its diagnostic capacity with GPs and dermatologists. To conduct the study the following procedure will be executed until the required number of samples is reached: A suitable patient with skin concern is asked to participate and sign the patient's study agreement. GP will diagnose the skin condition. GP (or nurse) will take one good quality image of the skin condition. GP will send the photograph as a teledermatology consultation following the current workflow. The image is entered in the Autoderm ML interface. Dermatologist will diagnose the skin condition. The study will be conducted in primary care centers managed by the Catalan Health Institute. Participant PCP will be located in rural and metropolitan areas in Central Catalonia, which includes the regions of Anoia, Bages, Moianès, Berguedà and Osona. The reference population included in the study will be about 512,050. The recruitment of prospective subjects will consist on a consecutive basis. General practitioners will collect data from consecutive patients who meet the inclusion criteria after obtaining written informed consent. Collected data will be reported exclusively in case report form (attached at Annex V and VI). The GP will diagnose the skin condition and will fill the "Face-to-face assessment by GP". For each patient, the GP using a smartphone camera will take a close up good quality image of the skin problem. The image will be anonymous and it will be not possible to identify patients. The GP will use the Autoderm ML interface to upload the anonymized image and will fill the "Assessment provided by the ML model" questionnaire with the top 3 diagnoses generated by the ML model. In order to get a second opinion, the GP will incorporate the anonymized image and an accurate description of the skin lesion into the patient's medical history following the current teledermatology flow. The GP will fill "Assessment by teledermatology" questionnaire after receiving the information, being response time about 2-7 days. In case of dermatology referral, the GP will fill "Assessment by in person dermatologist", by accessing electronic health records as they become available, being the average waiting time for referral from 30 to 90 days. Questionnaire case number will be the same for all questionnaires and it will not be possible to identify the patient, as case number will be predefined before the initiation of the data collection phase. To compare the performance of the ML model with that of the GPs and dermatologists, it will be required a sample size of 100 images of skin diseases from patients who meet the inclusion criteria. The proposed sample size is based on sample size calculation used in similar research.

Machine Learning, Artificial Intelligence, Data accuracy, Computed Assisted Diagnosis, Neural Network Computer

The diagnostic capacity of the ML model will be compared with that of the general practitioners and with dermatologist.

GP using a smartphone camera will take an image of the skin problem and will use the Autoderm ML interface to upload the anonymized image. The obtained predicted diagnosis will be recorded in case report form.

Inclusion Criteria: Patients who have cutaneous disease reason-for-visit. Patients who provide written informed consent. Patients who are 18 years of age or older. Exclusion Criteria: Patients with advanced dementia. Patients with a cutaneous lesion which can't be photographed with a smartphone and images with poor quality. Patients who have conditions associated with risk of poor protocol compliance.

Lim HW, Collins SAB, Resneck JS Jr, Bolognia JL, Hodge JA, Rohrer TA, Van Beek MJ, Margolis DJ, Sober AJ, Weinstock MA, Nerenz DR, Smith Begolka W, Moyano JV. The burden of skin disease in the United States. J Am Acad Dermatol. 2017 May;76(5):958-972.e2. doi: 10.1016/j.jaad.2016.12.043. Epub 2017 Mar 1.

Schofield JK, Fleming D, Grindlay D, Williams H. Skin conditions are the commonest new reason people present to general practitioners in England and Wales. Br J Dermatol. 2011 Nov;165(5):1044-50. doi: 10.1111/j.1365-2133.2011.10464.x. Epub 2011 Sep 22.

Dokotor.se [Internet]. Survey Telemedicine statistics Dokotor.se, the % of queries that are dermatology related 2019 [cited 2019]

Activitat assistencial de la xarxa sanitària de Catalunya, any 2012: registre del conjunt mínim bàsic de dades (CMBD). Barcelona: Departament de Salut. 2013.

Lowell BA, Froelich CW, Federman DG, Kirsner RS. Dermatology in primary care: Prevalence and patient disposition. J Am Acad Dermatol. 2001 Aug;45(2):250-5. doi: 10.1067/mjd.2001.114598.

Porta N, San Juan J, Grasa MP, Simal E, Ara M, Querol MA. [Diagnostic agreement between primary care physicians and dermatologists in the health area of a referral hospital]. Actas Dermosifiliogr. 2008 Apr;99(3):207-12. Spanish.

Lopez Segui F, Franch Parella J, Girones Garcia X, Mendioroz Pena J, Garcia Cuyas F, Adroher Mas C, Garcia-Altes A, Vidal-Alaball J. A Cost-Minimization Analysis of a Medical Record-based, Store and Forward and Provider-to-provider Telemedicine Compared to Usual Care in Catalonia: More Agile and Efficient, Especially for Users. Int J Environ Res Public Health. 2020 Mar 18;17(6):2008. doi: 10.3390/ijerph17062008.

Borve A, Dahlen Gyllencreutz J, Terstappen K, Johansson Backman E, Aldenbratt A, Danielsson M, Gillstedt M, Sandberg C, Paoli J. Smartphone teledermoscopy referrals: a novel process for improved triage of skin cancer patients. Acta Derm Venereol. 2015 Feb;95(2):186-90. doi: 10.2340/00015555-1906.

Ferrer RT, Bezares AP, Manes AL, Mas AV, Gutierrez IT, Llado CN, Estaras GM. [Diagnostic reliability of an asynchronous teledermatology consultation]. Aten Primaria. 2009 Oct;41(10):552-7. doi: 10.1016/j.aprim.2008.11.012. Epub 2009 Jun 5. Spanish.

Gomolin A, Netchiporouk E, Gniadecki R, Litvinov IV. Artificial Intelligence Applications in Dermatology: Where Do We Stand? Front Med (Lausanne). 2020 Mar 31;7:100. doi: 10.3389/fmed.2020.00100. eCollection 2020.

Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017 Feb 2;542(7639):115-118. doi: 10.1038/nature21056. Epub 2017 Jan 25. Erratum In: Nature. 2017 Jun 28;546(7660):686.

Liu Y, Jain A, Eng C, Way DH, Lee K, Bui P, Kanada K, de Oliveira Marinho G, Gallegos J, Gabriele S, Gupta V, Singh N, Natarajan V, Hofmann-Wellenhof R, Corrado GS, Peng LH, Webster DR, Ai D, Huang SJ, Liu Y, Dunn RC, Coz D. A deep learning system for differential diagnosis of skin diseases. Nat Med. 2020 Jun;26(6):900-908. doi: 10.1038/s41591-020-0842-3. Epub 2020 May 18.

Kamulegeya LH, Okello M, Bwanika JM, Musinguzi D, Lubega W, Rusoke D, et al. Using artificial intelligence on dermatology conditions in Uganda: A case for diversity in training data sets for machine learning. bioRxiv [Internet]. 2019 Jan 1;826057. Available from: http://biorxiv.org/content/early/2019/10/31/826057.abstract .

Evaluation of the diagnostic accuracy of an online artificial intelligence app for skin disease diagnosis. Alexander Larson, Degree Project in Medicine, Sahlgrenska University Hospital Department of Dermatology and Venereology, Gothenburg, Sweden 2018.

Brinker TJ, Hekler A, Enk AH, Klode J, Hauschild A, Berking C, Schilling B, Haferkamp S, Schadendorf D, Holland-Letz T, Utikal JS, von Kalle C; Collaborators. Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma image classification task. Eur J Cancer. 2019 May;113:47-54. doi: 10.1016/j.ejca.2019.04.001. Epub 2019 Apr 10.

Haenssle HA, Fink C, Schneiderbauer R, Toberer F, Buhl T, Blum A, Kalloo A, Hassen ABH, Thomas L, Enk A, Uhlmann L; Reader study level-I and level-II Groups; Alt C, Arenbergerova M, Bakos R, Baltzer A, Bertlich I, Blum A, Bokor-Billmann T, Bowling J, Braghiroli N, Braun R, Buder-Bakhaya K, Buhl T, Cabo H, Cabrijan L, Cevic N, Classen A, Deltgen D, Fink C, Georgieva I, Hakim-Meibodi LE, Hanner S, Hartmann F, Hartmann J, Haus G, Hoxha E, Karls R, Koga H, Kreusch J, Lallas A, Majenka P, Marghoob A, Massone C, Mekokishvili L, Mestel D, Meyer V, Neuberger A, Nielsen K, Oliviero M, Pampena R, Paoli J, Pawlik E, Rao B, Rendon A, Russo T, Sadek A, Samhaber K, Schneiderbauer R, Schweizer A, Toberer F, Trennheuser L, Vlahova L, Wald A, Winkler J, Wolbing P, Zalaudek I. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol. 2018 Aug 1;29(8):1836-1842. doi: 10.1093/annonc/mdy166.