Artificial Intelligence to Evaluate Radiographic Outcome following Hallux Valgus Surgery

Introduction

Angular measurements taken before corrective surgery for hallux valgus can be helpful in decision-making (Table 1) [1]. Many different methods have been described and the American Association of Orthopaedic Surgeons (AAOS) has published guidelines recommending a standard method [2-7]. The published literature on the reliability of this method in pre-operative radiographs shows reasonable consistency between observers for the hallux valgus (HV) angle and intermetatarsal (IM) angle [3,8]. But in clinical practice, the observer variation is felt to be larger. Besides, these angles have been shown to change with weightbearing and metatarsal rotation, making comparison more difficult [3,9]. Computer models using artificial intelligence and neural networks have also been developed, but they have been trained on relatively few human observations, lack a clear definition of the central axis of the bone and use pre-operative radiographs [10].

Table 1: Severity grading of Hallux Valgus based on HV and IM angles.

Ideally, the same method should be applied to the post-operative radiographs if these measures are to serve as an objective outcome measure. But it is difficult to use the same method in postoperative films due to the wide variety of osteotomy sites and configurations. Measurements are particularly complicated when a scarf osteotomy has been used to correct the deformity [11,12]. Very few studies have been conducted to test the reliability of the described methods in post-operative radiographs, none of which use the scarf osteotomy [13,14].

Good reliability has also been demonstrated using semi-automated angle measurements in weight-bearing Computed Tomography scans. These methods hold promise for the future when this modality may become more widely available and utilised routinely [15].

The aims of this study are to:

1. Assess the variation in angles measured by human observers in patients with hallux valgus surgery
2. Evaluate a computer model that calculates the angles, with the central axis of each bone defined as the first eigenvector of the segmented shape.
3. Compare computer and human measurements for reproducibility in preoperative and postoperative radiographs.

Methods

Sample size calculation was performed with G*Power (version 3.1.9.5) [16]. This indicated a minimum of 40 patients were required to detect a difference of 5 degrees between human and computer measurements (effect size of 0.6) using two-tailed paired Wilcoxon test with 80% power and a significance level of 5%. Therefore, all patients who underwent surgical correction for hallux valgus within a six-month period (between September 2023 and April 2024) were included to obtain sufficient data. Patients who had procedures other than an osteotomy (Keller’s procedure or fusion) were excluded. There were no patients lost to follow up.

The HV, IM and distal metatarsal articular (DMA) angles were measured on preoperative, intraoperative and postoperative radiographs. All pre- and postoperative radiographs were taken in a weight-bearing position, with the postoperative images taken at least five weeks following surgery. Seven observers (four specialist registrars, two medical students and one consultant) measured the angles on-screen using the built-in angle measurement function of our Picture Archiving and Communication System (PACS; Synapse® version 4.4.4, Fujifilm® healthcare, USA, 2023). Each observer measured the angles on two separate occasions; at least two weeks apart.

A semi-automated computer model (see appendix for packages) was used, built on Python version 3.10 (Python Software Foundation, https://www.python.org/). Radiographs were imported with the pydicom package (https://pydicom.github.io/, see appendix) and converted to arrays. Segment Anything (SAM, Meta AI, Toronto, https://segment-anything.com/) software was used to segment the bones (Figure 1B) [17]. Five points were selected on the image to facilitate segmentation: the proximal phalanx, the first metatarsal, the second metatarsal, the medial cuneiform and the intermediate cuneiform (Figure 1A). Using the coordinates of the segmented contours, the first eigen vectors of the proximal phalanx, first metatarsal and second metatarsal were used to define the central axes (Figure 1C). The distal metatarsal articular surface was defined using 75 pixels of the metatarsal head contour on either side of the most distal point of the first metatarsal segmentation along the first eigenvector. The tangent to this surface was taken as the distal first metatarsal joint line. The HV, IM and DMA angles were then calculated from the central axes and distal first metatarsal joint line. Implanted metalwork can interfere with the segmentation process. Consequently, the metalwork was ‘removed’ with software using

thresholding (pixel value over 200) and restored with the simple-lama-inpainting algorithm (Figure 2) before segmentation [18,19].

Figure 1: Weight bearing radiographs of left foot used for segmentation with Segment Anything (SAM) software12 (a) User selected proximal phalanx, 1st and 2nd metatarsal, medial and intermediate cuneiforms (b) Segmentation with Segment Anything (SAM) software (c) Central axes found from the 1st eigenvector of the segmentations of the proximal phalanx and 1st and 2nd metatarsals. Hallux valgus angle 34.6 degrees, intermetatarsal angle: 14.7 degrees.

Figure 2: Metalwork needs to be removed with software as it interferes with segmentation. (a) Weight bearing radiograph with metalwork in situ following surgery for hallux valgus (b) Software removal of metalwork by thresholding pixel values over 200 (white) and ‘inpainted’ with the simple-lama-inpainting algorithm [13,14].

Statistical analysis was performed with R statistical software version 4.4.3 (R Core Team) and the ggplot2 package was used to produce the plots [20,21]. Numerical variables could not be modelled with a normal distribution and were summarised with the median and interquartile range. Categorical variables were expressed as numbers or proportions. Bland-Altman plots were used to compare measurements for each angle and visualise observer error [22]. For intra-observer error, the mean of the two values recorded by the same observer was used as the true value of the measurement. For inter-observer error, this was the mean of all values recorded by all observers for the measurement. The intraclass correlation coefficient (ICC) was calculated with the irr package [23]. An ICC of less than 0.5 was regarded as poor, between 0.5 and 0.75 as moderate, between 0.75 and 0.9 as good, and more than 0.9 as excellent. Differences between groups were compared using the Wilcoxon signed-rank test at a 5% significance level. The article was structured in accordance with the STROBE checklist for cohort studies [24].

Results

51 patients underwent surgery for correction of Hallux Valgus during the study period. Three patients were excluded because they had undergone first metatarsophalangeal joint fusion (two patients) and Keller’s procedure (one patient) for correction of their deformity, leaving 48 patients who were included in the study. There were 46 female and two male patients. All patients had unilateral procedures, with the right side being operated in 44%. The median age was 58 years (IQR 51-64 years). The most common metatarsal osteotomy used was a scarf osteotomy (40 patients), followed by a chevron osteotomy (eight patients). Among these, concomitant Akin procedures were performed in 27 (68%) and four (50%) patients, respectively. The postoperative weight-bearing radiographs were obtained at a median 46 days (IQR: 38-54) after the surgical procedure.

The inter-observer and intra-observer variation in preoperative and postoperative radiographs for HV, IM and DMA angles is presented in Figures 3, 4 and 5, respectively. Table 2 presents the ICCs for inter-observer and intra-observer error on preoperative, intraoperative and postoperative radiographs.

Figure 3: Bland-Altman plots of inter-observer error in the measurement of angle in pre-operative radiographs.

Figure 4: Bland-Altman plots for intra-observer error in the measurement of angles in pre-operative radiographs.

Figure 5: Improvement in angles following surgery, measured using the computer method.

Table 2: Intraclass Correlation Coefficient for inter-observer and intra-observer variation in measurements. Upper and lower 95% confidence limits are also presented. ICCs rated excellent are in bold.

Computer and human measurements were compared with the paired Wilcoxon signed rank test. This confirms there is no significant difference in the estimated HV and IM angles between human observers and the computer. Bland-Altman plots also confirm there is no bias in the measurements (Figures 3 and 4). However, there is significantly less variation in the pre-operative and post-operative computer measurements of the HV and IM angle compared to human observations (p<0.001). The variation in the intra-operative measurement of the hallux valgus angle using the computer method was not significantly different to human observations (p=0.296), but the IM angle was (p=0.014).

Table 3 presents the median and interquartile range for measurements using the computer method. Figure 5 presents a plot showing improvement in angles measured by the computer method with surgical correction. As expected, there was a significant improvement in HV and IM angles postoperatively (p<0.001). However there was no significant difference in the DMA angle (p=0.374). There was no significant difference between intraoperative and postoperative measurements (p-values for HV angle= 0.366, IM angle= 0.918, DMA angle= 0.168).

The time taken for a set of measurements using the computer method was 23 seconds, compared to a median of 3 minutes for human observers (IQR: 1.6–4.4 minutes).

Table 3: Median and IQR of different angles measured using the computer method. All angles are in degrees.