Performance and Evaluation for CT Colonography Screening (PERFECTS)

Colorectal cancer (CRC) is an important United Kingdom healthcare issue affecting 1 in 20 individuals, half of whom will die from the disease. Late presentation of CRC has a poor prognosis, whereas excellent cure rates (>95%) are seen in those who present early. Accurate and early diagnosis of CRC is therefore crucial. In most patients this is achieved via colonoscopy, a camera test which is widely available and allows tissue samples to be taken of any abnormalities seen during the procedure. However, a non-invasive alternative is Computed Tomography Colonography (CTC) which uses X-rays to produce images of the large bowel which are then interpreted by Radiologists. CTC has high sensitivity for the diagnosis of CRC (comparable to colonoscopy) and the cancer precursor - adenomatous polyps. Unlike colonoscopy, however, there is no accreditation process for CTC and there is no infrastructure to ensure that all reporting Radiologists are able to do so adequately and, as a result, there is a wide range of diagnostic accuracy. There are no universally-accepted standards to monitor quality or assess diagnostic performance, partly because we do not know what the quality markers are and there is currently no system to quantify them. Overall, this contributes to low cancer detection rates, missed cancers and inequity for patients across the National Health Service (NHS). This study aims to assess the impact of a structured training programme with assessment and feedback on NHS radiologist performance. If the impact is positive and results in significantly improved performance, then such a scheme could be adopted into an accreditation programme for CTC in the English Bowel Cancer Screening Programme (BCSP).

CT Colonography is a test which can can detect early cancer accurately, however, unlike colonoscopy (or mammography in breast cancer screening) there is currently no formalised program to help ensure Radiologists interpret CTC accurately. Colonoscopists are highly regulated, must be accredited and have ongoing quality assurance metrics, based on evidence linking these to clinical outcomes. Conversely, for CTC in the United Kingdom, there is no accreditation process, little regulatory oversight and no universally-accepted metrics to monitor quality or diagnostic performance. Radiologists who interpret CTC images are often inexperienced; one-third have interpreted <300 cases in total and 20% report <100 cases per annum (the minimum recommendation). Futhermore, in clinical practice across the NHS Bowel Cancer Screening Programme (NHS BCSP) CTC may have up to a 50% lower cancer and polyp detection rate than colonoscopy. Missed abnormalities at CTC are likely to underpin this difference, as centres with highly experienced radiologists (>1000 cases) using 3-dimensional interpretation have significantly higher detection rates. There is a Quality Assurance (QA) framework for the NHS BCSP (with an overseeing QA Committee for Radiology) but recommendations are largely based on opinion rather than evidence and any radiologist can report CTC, without accreditation; a fact which our patient representatives find extremely disappointing. Prior to wide-scale implementation of a possible accreditation programme and to justify both financial and human resource, Public Health England and the British Society of Gastrointestinal and Abdominal Radiologists have mandated that such a program is scientifically proven to improve Radiologist performance. Consequently, this project will investigate training and assessment methods to improve diagnostic accuracy and elucidate the factors associated with higher Radiologist sensitivity.

The control group will not receive the one-day training course or access to the online educational module. Test sets will be administered in the same manner as for the intervention group, but the control group will not receive any feedback on performance during the 12 month assessment phase. Feedback on test performance will only be provided after the 12 month period has ended.

One-day workshop of face to face training, comprising a mixture of theory and real-world CTC cases. Radiologists will complete four interpretation assessments lasting 2 to 3 hours: one set of 10 CTC cases immediately prior to training and a further test set of 10 immediately after, with tests repeated at 6 and 12 months (using different CTC cases for each set). Radiologists will be asked to identify colorectal cancer and polyps on the scans and complete an online answer form, including questions regarding diagnostic confidence, site of abnormality and management. The intervention group will receive online, confidential, individualised feedback after each test, which will include a case by case breakdown of their performance with explanatory answers and learning points.

Difference in sensitivity between intervention and control groups for detection of colorectal cancer / polyps on CT colonography at the 1 month post-intervention test.

Difference in sensitivity between intervention and controls for colorectal cancer/ polyps at the 12 month post-training test set (i.e. to test longer-term durability of one-off training supplemented by individualized feedback).

Difference in raw score between the control and intervention arms on the knowledge questions as compared to true answers.

Inclusion Criteria: National Health Service (NHS) Consultant Radiologists or final year Registrars or Fellows in the England or Wales who currently routinely report CT colonography scans. Exclusion Criteria: Radiologists who do not routinely report CT colonography scans. This study is limited to NHS staff and does not involve any patients.

Pickhardt PJ, Hassan C, Halligan S, Marmo R. Colorectal cancer: CT colonography and colonoscopy for detection--systematic review and meta-analysis. Radiology. 2011 May;259(2):393-405. doi: 10.1148/radiol.11101887. Epub 2011 Mar 17.

Pickhardt PJ, Choi JR, Hwang I, Butler JA, Puckett ML, Hildebrandt HA, Wong RK, Nugent PA, Mysliwiec PA, Schindler WR. Computed tomographic virtual colonoscopy to screen for colorectal neoplasia in asymptomatic adults. N Engl J Med. 2003 Dec 4;349(23):2191-200. doi: 10.1056/NEJMoa031618. Epub 2003 Dec 1.

Johnson CD, Chen MH, Toledano AY, Heiken JP, Dachman A, Kuo MD, Menias CO, Siewert B, Cheema JI, Obregon RG, Fidler JL, Zimmerman P, Horton KM, Coakley K, Iyer RB, Hara AK, Halvorsen RA Jr, Casola G, Yee J, Herman BA, Burgart LJ, Limburg PJ. Accuracy of CT colonography for detection of large adenomas and cancers. N Engl J Med. 2008 Sep 18;359(12):1207-17. doi: 10.1056/NEJMoa0800996. Erratum In: N Engl J Med. 2008 Dec 25;359(26):2853.

Atkin W, Dadswell E, Wooldrage K, Kralj-Hans I, von Wagner C, Edwards R, Yao G, Kay C, Burling D, Faiz O, Teare J, Lilford RJ, Morton D, Wardle J, Halligan S; SIGGAR investigators. Computed tomographic colonography versus colonoscopy for investigation of patients with symptoms suggestive of colorectal cancer (SIGGAR): a multicentre randomised trial. Lancet. 2013 Apr 6;381(9873):1194-202. doi: 10.1016/S0140-6736(12)62186-2. Epub 2013 Feb 14.

Stoop EM, de Haan MC, de Wijkerslooth TR, Bossuyt PM, van Ballegooijen M, Nio CY, van de Vijver MJ, Biermann K, Thomeer M, van Leerdam ME, Fockens P, Stoker J, Kuipers EJ, Dekker E. Participation and yield of colonoscopy versus non-cathartic CT colonography in population-based screening for colorectal cancer: a randomised controlled trial. Lancet Oncol. 2012 Jan;13(1):55-64. doi: 10.1016/S1470-2045(11)70283-2. Epub 2011 Nov 15.

Plumb AA, Halligan S, Taylor SA, Burling D, Nickerson C, Patnick J. CT colonography in the English Bowel Cancer Screening Programme: national survey of current practice. Clin Radiol. 2013 May;68(5):479-87. doi: 10.1016/j.crad.2012.10.018. Epub 2012 Dec 11.

Plumb AA, Halligan S, Nickerson C, Bassett P, Goddard AF, Taylor SA, Patnick J, Burling D. Use of CT colonography in the English Bowel Cancer Screening Programme. Gut. 2014 Jun;63(6):964-73. doi: 10.1136/gutjnl-2013-304697. Epub 2013 Aug 16.

Halligan S, Altman DG, Mallett S, Taylor SA, Burling D, Roddie M, Honeyfield L, McQuillan J, Amin H, Dehmeshki J. Computed tomographic colonography: assessment of radiologist performance with and without computer-aided detection. Gastroenterology. 2006 Dec;131(6):1690-9. doi: 10.1053/j.gastro.2006.09.051. Epub 2006 Oct 1.

Obaro AE, Plumb AA, Halligan S, Mallett S, Bassett P, McCoubrie P, Baldwin-Cleland R, Ugarte-Cano C, Lung P, Muckian J, Ilangovan R, Gupta A, Robinson C, Higginson A, Britton I, Greenhalgh R, Patel U, Mainta E, Gangi A, Taylor SA, Burling D. Colorectal Cancer: Performance and Evaluation for CT Colonography Screening- A Multicenter Cluster-randomized Controlled Trial. Radiology. 2022 May;303(2):361-370. doi: 10.1148/radiol.211456. Epub 2022 Feb 15. Erratum In: Radiology. 2022 May;303(2):E29.