The high accuracy of coronary computed tomography angiography (CCTA) compared to invasive coronary angiography (ICA) in detecting coronary stenosis has been demonstrated in several studies, using interpretation by cardiovascular experts. But which results would the same studies have yielded using less experienced readers?
For radiologists with less cardiovascular experience, the development of an automated system to aid in diagnosis holds significant potential. In this study, researchers from the Department of Radiology at Capital Medical University’s Beijing Friendship Hospital, China, set out to investigate the influence of artificial intelligence (AI) based on deep learning (DL) on the diagnostic performance and consistency of inexperienced cardiovascular radiologists.
AI rarely applied in CAD
The study highlighted the fact that AI algorithms applied in the diagnosis of a wide range of disease states - including in the detection and diagnosis of breast cancer and colon polyps - have improved inexperienced reader performance. AI algorithms are still rarely applied in coronary artery disease (CAD).
AI system used
The study used the CoronaryDoc clinical decision support platform V1.0 from Shukun (Beijing) Technology Co., Ltd. All CCTA data was transferred from a GE Advantage Workstation 4.6 or 4.7 to the AI workstation, and before the AI system extracted the centerline and automatically reconstructed multiplanar reformation (MPR) and curved MPR (cMPR) images based on the original axial image. The AI system used an automatic identification algorithm to achieve coronary artery segmentation and naming.
Study population
The study began with 252 consecutive patients with suspected or known coronary heart disease (CHD) who underwent both CCTA and ICA examinations within six months. After patients were excluded for various reasons including certain medical conditions and incomplete CCTA or ICA data, 196 patients were enrolled.
CCTA image acquisition and analysis
A 256-section, 64-section, and 128-row multidetector CT were used to capture patient image data. After patient datasets were reconstructed at a workstation to transform the data into MPR and cMPR images, the images were then transferred to a picture archiving and communication system (PACS). Patients were defined as positive for significant coronary artery disease when >50% stenosis was observed.
The readers had varying levels of experience - four readers (Readers 1-4) were general radiologists who had interpreted less than 50 cases of coronary artery stenosis via CCTA and had not been mentored, and another two (Readers 5 and 6) were cardiovascular radiologists with at least 5 years of CCTA experience.
Inexperienced Readers 1 and 2 and experienced Readers 5 and 6 evaluated all patient data on the same PACS without the AI system. Inexperienced Readers 3 and 4 evaluated the same patient data on the AI workstation and received AI assistance in coronary stenosis diagnosis.
Higher sensitivity for patient-level analysis with AI system
For patient-level analysis, it was found that Readers 3 (85.6%) and 4 (87.1%), who were aided by the AI system, had higher sensitivity than Readers 1 (70.5%) and 2 (78.4%), who did not use the AI system.
Using the area under the curve (AUC) as a measurement, the diagnostic accuracy of the inexperienced readers did not differ significantly (ranging from 0.68 to 0.71) but was lower than the experienced Readers 5 and 6 (0.77 and 0.67 respectively).
Higher sensitivity for vessel-level analysis with AI system
Sensitivity was also found to be higher in the AI-aided inexperienced readers when it came to identifying ≥ 50% stenosis at the vessel level. Readers 3 (67.1%) and 4 (69.3%), had a higher sensitivity than Readers 1 (53.2%) and 2 (61.2%).
The diagnostic accuracy for vessel-level analysis did not differ significantly between inexperienced readers, and only the AUC of Reader 2 was significantly different from those of the experienced readers.
AI shows higher sensitivity at patient level
The diagnostic accuracy of the AI system was found to be slightly higher than that of the experienced readers at the patient level, and similar at the vessel level. The AI system alone had 93.5% sensitivity, 57.9% specificity, and 80.0% accuracy at the patient level, and 78.1% sensitivity, 82.5% specificity, and 84.5% accuracy at the vessel level. The AI system missed one lesion at the vessel level and no lesions at patient level.
Diagnostic consistency of the six readers
While good interobserver consistency was found between the two inexperienced readers aided by the AI system at the patient and vessel levels, respectively, the two inexperienced readers without AI assistance had very poor interobserver consistency. The experienced cardiovascular radiologists without AI assistance showed moderate agreement.
AI can positively affect inexperienced radiologists’ performance
Before conducting the study, the researchers had recognized the potential AI has as a diagnostic aid for inexperienced radiologists. While AI is still rarely used for CAD diagnosis – a leading cause of life-threatening health problems in developing countries – the results of this study demonstrate that AI could have a positive effect on inexperienced radiologists in diagnosing coronary stenosis on CCTA.
Inexperienced readers with AI assistance performed significantly better than those without AI assistance at both the patient and vessel levels.
The increase in sensitivity witnessed in the AI training of less experienced radiologists was described as being “in line with the results of previous studies”.
AI ‘aids diagnostic sensitivity of less experienced readers’
Some limitations of the AI system were observed to be similar to those of human readers, such as an inability to accurately measure the severity of lesions and overestimate them compared to ICA or quantitative coronary angiography (QCA). Despite this consideration, and the fact that the algorithm is still undergoing optimization, the researchers are confident that AI offers “great potential for improving disease detection and excluding stenosis at the patient level for less experienced readers or novices, and it may provide an appropriate training alternative”.
Looking to the future, the researchers highlighted that “increased sensitivity of less experienced readers could improve radiologists’ abilities to detect obstructed coronary arteries and reduce missed disease diagnoses”.
The study concluded from its findings that “the AI system could effectively increase the diagnostic sensitivity of less experienced readers and significantly improve their consistency”.
Sources:
https://bmcmedimaging.biomedcentral.com/articles/10.1186/s12880-022-00756-y
https://www.sciencedirect.com/science/article/abs/pii/S016926072031484X?via%3Dihub