Classification of carotid plaque vulnerability by neurosurgical residents using ultrasonography in the clinical field

Article information

J Korean Soc Geriatr Neurosurg. 2022;18(2):38-43

Publication date (electronic) : 2022 October 18

doi : https://doi.org/10.51638/jksgn.2022.00094

Doyoung Na¹, Junmo Kim¹, Kyungmin Chung¹, Seunghun Han¹, Gihyeok Yun¹, Jinseo Yang¹, Hyukjai Choi¹, Heung Cheol Kim², Yongjun Cho¹, Jin Pyeong Jeon^,1

¹Department of Neurosurgery, Hallym University College of Medicine, Chuncheon, Korea

²Department of Radiology, Hallym University College of Medicine, Chuncheon, Korea

Corresponding Author: Jin Pyeong Jeon, MD, PhD Department of Neurosurgery, Hallym University Chuncheon Sacred Heart Hospital, Hallym University College of Medicine, 77 Sakju-ro, Chuncheon 24253, Korea Tel: +82-33-240-5171; Fax: +82-33-240-9970; E-mail: jjs6553@daum.net

Received 2022 July 14; Revised 2022 August 22; Accepted 2022 August 25.

Abstract

Introduction

Carotid plaque refers to focal wall hypertrophy that is at least 50% greater than that of the surrounding carotid wall. The prevalence of carotid plaques is approximately 40.3% and they are found more frequently in males than in females [1]. Patients with carotid plaques are more likely to have metabolic disorders with high total cholesterol and low-density lipoprotein-cholesterol levels [1]. Because carotid plaques are formed in the late stages of atherosclerosis, they are thought to be a risk factor for cerebrovascular disease including stroke [2,3]. Accordingly, screening for carotid plaques and identifying carotid plaque morphology is conducted in the clinical field. Among various radiologic tests, carotid ultrasonography (US) is performed widely in private hospitals, as well as general and university hospitals due to its relatively low cost and ease of implementation compared to magnetic resonance imaging (MRI) or computed tomography (CT) [4–8].

The variables in interpreting carotid US images are intima-media thickness (IMT), the degree of stenosis, and the presence of a carotid plaque along with an assessment of its vulnerability to rupture. Among these variables, the accurate classification of a carotid plaque vulnerable to rupture and identifying one that is more likely to cause a stroke is difficult for general practitioners who are non-radiologists. Most neurosurgeons also have difficulties in distinguishing unstable plaques from stable carotid plaques despite the increasing number of carotid stenting procedures in recent years. In addition, neurosurgical residents are not well trained in carotid US interpretation. Few studies have addressed the classification accuracy of plaque vulnerability by non-radiologists in the domestic clinical field. Here, we investigated the classification accuracy of carotid plaque vulnerability (unstable vs. stable plaque) by neurosurgical residents to obtain objective data on the interpretation accuracy in the actual medical environment.

Material and Method

Study population

The derivation cohort was obtained from the regional stroke database between May 2021 and May 2022. We selected subjects from this database who underwent carotid US for stroke or medical evaluations. Carotid US was conducted in the B mode using a 5 to 18 MHz US probe (APLIO i800; Canon Medical Systems Corporation, Tochigi, Japan) and an 11 MHz US probe (LOGIQ E9; GE Healthcare, Wauwatosa, WI, USA). The procedure was performed by a neuroradiologist with more than 20 years of experience according to the American Society of Echocardiography guidelines [9]. Patients were positioned supine on the scan bed with their heads resting flat to facilitate high-quality and reproducible images. To relieve patients, towels or pillows were used to support knees and neck. The neck was slightly extended and rotated in a direction opposite to the probe. During the scan, the sonographer adjusted the neck to optimize the images, especially in the anterior planes. Carotid US was performed by quantifying the probe angle using an external landmark such as the Meijer arc.

Carotid plaque vulnerability was classified into two groups, stable and unstable plaques (Fig. 1). Unstable plaques referred to vulnerable plaques, which are prone to rupture or thrombosis with the following features using gray-scale imaging alone: (1) an irregular plaque surface, (2) plaque with ulceration, (3) heterogeneous echogenicity within the plaque, and (4) a non-echogenic intraplaque or central core, indicating hemorrhage or a lipid-rich necrotic core covered by a thin fibrous cap [10]. Plaque stability was determined by an experienced neuroradiologist. Four neurosurgical residents, one intern, and one neuroradiologist participated in interpreting the images. All reviewers evaluated the images independently followed by a comparison of stable and unstable carotid plaques. This study was approved by the Institutional Review Board (IRB) of the Hallym University Chuncheon Sacred Heart Hospital (IRB No. 2018-11-008-004 and 2019-06-006-008) and informed consent was waived due to the retrospective nature of the study.

Fig. 1.

Examples of carotid plaque vulnerability, including stable and unstable plaques. The red and blue circles indicate unstable and stable plaques, respectively.

Results

A total of 405 patients with 995 carotid US images (axial image, n=297; and longitudinal image, n=698) were included in the analysis. The mean patient age was 70.5 years (range, 44–91 years) and 223 (55.1%) were female. The number of patients with hypertension and diabetes mellitus was 205 (50.6%) and 76 (18.8%), respectively. Cardiac disease, such as coronary artery disease or atrial fibrillation, was observed in 138 patients (34.1%). Twenty-four patients were smokers with an average of 28.25 pack-years. Among the patients who underwent carotid US, 61 (15.1%) were conducted for stroke evaluation and 344 (84.9%) were conducted for health examinations (Table 1).

Table 1.

Clinical characteristics of the enrolled patients

The classification results of the five reviewers using a neuroradiologist’s decision as the reference value are presented in Tables 2 and 3. There were 236 (24.7%) unstable and 749 (75.3%) stable carotid plaques. Reviewer 1 showed the best-balanced interpretation with a sensitivity of 83.7% (78.5%–88.1%), specificity of 69.0% (65.6%–72.3%), and overall accuracy of 72.7% (69.8%–75.4%). Compared to the overall accuracy (ranging from 69.3% to 76.1%), the sensitivity results showed severe fluctuations (ranging from 39.0% to 83.7%) between the reviewers. Pairwise comparison of the ROC of reviewer 1 also showed the best performance in classifying plaque vulnerability with an area under the curve (AUC) of 0.583 (95% confidence interval [CI], 0.552–0.614) (Fig. 2A). Reviewer 4 had the lowest AUC of 0.533 (95% CI, 0.502–0.565). The AUC difference between reviewer 1 and reviewer 4 reached statistically significance (AUC difference, 0.049; 95% CI, 0.015–0.084; P=0.005). Except for these findings, there were no significant differences in AUC values between the remaining reviewers. If the data distribution is unbalanced, like in our case (unstable plaque: n=236, 24.7%; stable plaque: n=749, 75.3%), there is a possibility that the AUC value might be biased. To overcome this potential bias, we generated a precision-recall curve, especially for reviewer 1 and reviewer 4. The results also demonstrated a significant difference in plaque vulnerability classifications (AUC difference, 0.028; 95% bootstrap CI, 0.007–0.048) (Fig. 2B).

Table 2.

Accuracy of each reviewer for the interpretation of carotid plaque characterization (stable vs. unstable plaques) with a neuroradiologist’s decision as a reference

Table 3.

Binary classification of plaque vulnerability (stable vs. unstable plaques) between the reviewers

Fig. 2.

(A) Pairwise comparison of receiver operating characteristics (ROC) curves for classifying plaque vulnerability. (B) The precision-recall curve between reviewer 1 and reviewer 4 who showed significant differences in the area under the curve (AUC) in the ROC comparisons. SE, standard error; CI, confidence interval.

Discussion

Carotid plaques form various shapes and occur in areas from the distal common carotid artery to proximal internal carotid artery [13]. It is more accurate to predict patient prognosis and choose treatment methods based on plaque vulnerability rather than only the degree of carotid stenosis. The major features used to describe carotid plaques in carotid US images include calcification, ulceration, hemorrhage, and a lipid-rich necrotic core, as well as IMT [10]. Calcification with ulceration is stimulated by prolonged shear stress and sudden arterial pressure changes on the carotid bifurcation [14]. This activates inflammation and metalloproteinase activity, causing erosion of the calcification to progress to ulceration of the surface [15]. Ulceration in the carotid plaque predisposes to thrombus formation. Plasminogen activator inhibitor-1 within plaques creates a thrombotic effect [16]. The expression of tissue factors, which have a major effect on the coagulation cascade, is determined by the extent and degree of inhibition of the tissue factors pathway within the plaque [17]. Tissue factor is localized to areas of lipid deposition and cholesterol crystals in carotid plaque [18]. Bos et al. [19] reported that intraplaque hemorrhage of the carotid artery significantly increased the risk of stroke (hazard ratio, 2.42; 95% CI, 1.30–4.50). Considering these facts, the accurate classification of carotid plaque vulnerability (unstable vs. stable plaque) is important for clinicians to determine treatment plans. Three radiologic tests, MRI, CT, and US are available to examine plaque vulnerability in the carotid artery. MRI provides excellent differentiation of soft tissue composition with high resolution, but its use can be limited due to the high medical costs and time-consuming procedures needed to obtain the images [4–6]. CT has the advantage of shorter imaging time than MRI but has the limitation that a contrast agent is required to distinguish plaque components [7,8]. In the domestic medical environment, MRI and CT is used in more than US in small and medium-sized general hospitals. And in most of these hospitals, radiologists are available to interpret plaque vulnerability. Compared to MRI and CT, carotid US is widely used in private hospitals, as well as for health examinations in general and university hospitals in Korea due to its relatively low cost, ease of using at the bedside, and non-invasiveness. Lukanova et al. [20] reported that carotid US showed high balanced accuracy for detecting unstable plaques with a sensitivity of 94% and specificity of 93% compared to multi-detector CT, with a sensitivity of 83% and specificity of 73%, and MRI with a sensitivity of 100% and specificity of 89%. However, the procedure and its interpretation may vary depending upon the examiner’s experience. In particular, considering the reality of private hospitals with many non-radiologists, it is necessary to properly classify plaque vulnerability. In this study, we evaluated the accuracy of plaque vulnerability classification, targeting neurosurgical residents. Although there were some differences in classification accuracy between the reviewers, the overall accuracy level was low (0.5<AUC≤0.7) [21]. Considering that stroke risk is largely dependent upon not only the degree of stenosis but also plaque vulnerability, accordingly, education on discriminating plaque vulnerability is necessary during resident training [22]. However, it is not easy to educate residents on carotid US interpretation given the reality of medical situations. In this case, a new computer-aided diagnostic system using machine learning can be an alternative. Tanaka et al. [23] demonstrated that a convolutional neural network (CNN) composed of VGG19 and ResNet152 showed accurate classifications of breast ultrasound images with a sensitivity of 90.9% (95% CI, 84.5%–97.3%), specificity of 87.0% (95% CI, 79.5%–94.5%), and AUC of 0.951 (95% CI, 0.916–0.987). Biswas et al. [24] proposed a two-stage artificial intelligence model to measure atherosclerotic wall thickness and total plaque burden. However, they did not distinguish plaque vulnerability. Lekadir et al. [25] also suggested a CNN for automatically characterizing plaque components, such as the lipid core, fibrous cap, and calcified tissue. However, they used only 56 cases. Accordingly, there is a need for artificial intelligence (AI) based on gray-scale carotid US images, which is often used in the field, and AI research that not only detects plaque composition but comprehensively informs on plaque vulnerability.

There were some limitations in this study. First, there were only five participants, and they were from the same hospital. Thus, a concern may arise as to whether the residents participating in the study actually represent non-radiologists working in private hospitals. Second, the correct answer was decided only by one neurointerventionist. Although he had more than 20 years of experience, selection bias may have occurred, and this should be taken into account when interpreting our results. Nevertheless, to the best of our knowledge, this was the first study to investigate the classification accuracy of plaque vulnerability by neurosurgical residents based on carotid US images in actual clinical settings.

Conclusion

The classification accuracy of neurosurgical residents to discriminate plaque vulnerability based on carotid US images was low. To address this problem, a system that helps to educate and comprehensively interpret carotid plaque morphology should be developed.

Article information Continued