[ad_1]
Longitudinal Image-Based AI Models for Health and Medicine
AI sees the end! Deep learning predicts all-cause mortality from single and sequential imaging of body composition
Key points, TLDR:
- A combination of body composition imaging and meta-data (eg, age, sex, strength, walking speed, etc.) resulted in the best 10-year mortality predictions.
- Overall, longitudinal or sequential models performed better than single-record models, highlighting the importance of modeling change and time dependence in health data.
- Longitudinal models have the potential to provide a more comprehensive assessment of human health
- Read the paper
Artificial intelligence (AI) and machine learning (ML) are revolutionizing healthcare and ushering in the era of precision medicine. The motivation behind the development of AI health models is to reduce mortality and morbidity, as well as to increase the quality of life. Well-trained models have the ability to more thoroughly analyze the data presented, offering a more comprehensive assessment of human health.
Image-based medical AI/ML models have already reached a maturity where they often rival or outperform human performance, adept at identifying patterns and abnormalities that easily escape the human eye. However, most of these models still work with single-point-in-time data, providing an isolated snapshot of health at a single point in time. Whether they are unimodal or multimodal models, they work with data collected over a relatively similar time frame, which forms the basis of the forecast. However, in the broader context of AI/ML for medical applications, these single time point models represent only the first step—the proverbial “low-hanging fruit.” One of the frontiers of medical AI research is longitudinal models that offer a more holistic view of human health over time.
Longitudinal models are designed to integrate data from multiple time points to capture an individual’s health trajectory rather than a single point in time. These models address the dynamic nature of human health, where physiological changes are constant. The ability to map these changes to specific outcomes or health questions could be a game changer in predictive healthcare. The concept of longitudinal data is not new to clinical practice—it is routinely used to monitor aging and predict frailty. A prime example is tracking bone mineral density (BMD), a key marker of osteoporosis and frailty. Regular BMD assessments can reveal significant declines that indicate potential health risks.
Historically, the development of longitudinal models has faced several significant challenges. Aside from the larger volume of data and the separate calculations required on an individual basis, the most critical hurdle lies in the curation of longitudinal medical data itself. Unlike data at a single point in time, longitudinal data involves tracking patients’ health information over a long period of time, often across multiple health care facilities. This requires meticulous organization and management of data, making the curation process time-consuming and expensive. A number of successful studies have been funded to prospectively collect longitudinal data. These studies note challenges with patient retention over longer follow-up periods. Thus, despite the potential advantages of longitudinal models, their development remains a complex, resource-intensive endeavor.
Changes in body composition, the proportions of lean and adipose soft tissue and bone, are known to be associated with mortality. In our study, we aimed to use body composition information to better predict all-cause mortality, in simpler terms, a person’s life expectancy. We evaluated the performance of models built on both single-time and longitudinal data, referred to as our “single-entry” and “sequential” models, respectively. Single-entry models allowed us to assess which types of information were most predictive of mortality. Successive models were developed to account for changes over time and influence mortality predictions.
Data for this study were obtained from a longitudinal study known as the Health, Aging, and Body Composition (Health ABC) study, in which more than 3,000 older, multiracial male and female adults were followed up to age 16. This study resulted in a rich and comprehensive longitudinal data set. As part of this study, patients underwent whole-body dual-energy X-ray absorptiometry (TBDXA) imaging and several pieces of metadata were collected (see Table XXX). In accordance with modeling best practices and to avoid data leakage or reduce overfitting, the data were partitioned into a carrier, validation, and suspended test set using a 70%/10%/20% split.
We quantify body composition using whole-body dual-energy X-ray absorptiometry (TBDXA) imaging, which has long been considered the gold standard imaging modality. Historically, patient metadata including variables such as age, body mass index (BMI), grip strength, gait speed, etc. The widespread use of patient meta-data and surrogate measures of body composition was due to the limited availability of DXA scanners. Access has improved significantly recently as scans have become cheaper and no longer require a doctor’s referral/order/appointment.
Three single-entry models were constructed, each with different inputs but all with the same output, which was the 10-year mortality probability. The first model was built only to receive patient metadata and is a neural network with one 32-unit ReLU activation hidden layer and a sigmoidal prediction layer. The second model used only TBDXA images as input, and it consisted of a modified Densenet121 modified for two color channels, as opposed to the three color channels (RGB) seen in most natural images. The dual energy nature of DXA results in high and low X-ray images that are fully registered and stacked in two image channels. The third model combines the metadata embedding of the first model with the TBDXA image embeddings of the second model, then passes it through a 512-unit, 64-unit fully connected ReLU layer, and finally a sigmoidal prediction layer.
Three successive models were constructed and evaluated. Single-entry model architectures served as the basis for each successive model, but the sigmoidal prediction layers were removed so that the output was a vector representing the feature embedding. Data were collected from each patient at several time points during the study. Data from each time point were entered into the corresponding models to obtain the corresponding feature vector. The feature vectors for each patient were sorted and stacked in order. A long-short-term memory (LSTM) model was trained to obtain a sequence of feature vectors and produce 10-year mortality predictions. As mentioned earlier, there are several difficulties in conducting longitudinal studies, with storage and data collection being a common problem. Our study was not without these problems, and some patients had more data than others as a result. The LSTM model was chosen as the sequence modeling approach because they are not restricted to using the same input sequence length for each patient. i.e. LSTMs can work with sequences of varying lengths, thereby eliminating the need for panel sequencing if patients are short of the full set of data points (~10).
The area under the receiver operating characteristic (AUROC) plot on the stop test set shows that metadata performs better than using TBDXA imaging alone in single-record and sequential models. However, the fusion of meta-data and TBDXA images resulted in the best AUROCs in both modeling paradigms, indicating that the image contains useful information predicting mortality not captured by the metadata. Another way to interpret this is that the meta-data are not a complete surrogate measure of body composition for predicting mortality. If they were complete surrogates, combining TBDXA images with metadata would not result in a significant increase or change in AUROC. The fact that the combination produced better AUROCs indicates that the image provides orthogonal information beyond what the metadata captures and further justifies the usefulness of the image.
Longitudinal or sequential models performed better overall than single-entry models. This applies to all modeling approaches and data input types (meta-data, image only, combined meta-data and image). These results demonstrate the importance of modeling change and time dependencies of health data.
We performed an Integrated Discrimination Improvement (IDI) analysis to assess the benefits of combining images with metadata compared to using metadata alone. This analysis was performed on sequence models that exceeded single-record models. The IDI was found to be 5.79, with an integrated sensitivity and specificity of 3.46 and 2.33, respectively. This indicates that the combination of imagery and metadata improves the model’s ability to correctly identify those who will not survive the next 10 years by 3.46% and enhances those who will survive the next 10 years by 2.33%. Overall, this indicates an improvement in model performance of approximately 5.8%.
Our study highlights the promising potential of longitudinal AI/ML models in predictive healthcare, particularly in the context of all-cause mortality. A comparative analysis of single-record models and longitudinal models showed that the latter offered superior performance, indicating the critical role of modeling change over time in the analysis of health data. The clinical implications of our findings include the ability to more accurately and comprehensively assess human health through models that take into account historical or longitudinal patient data. Although the data needed to develop longitudinal health models exist, the appropriate infrastructure and institutional support are not yet in place to enable effective data exploration and scale-up of these models. Nevertheless, many are working to overcome these obstacles, and the development of longitudinal models is one of the many exciting frontiers of artificial intelligence in medicine.
The clinical implications of these findings are far-reaching. Longitudinal models have the potential to transform care delivery through more accurate, personalized predictions of a patient’s health trajectory. Such models can inform proactive interventions, thereby enhancing care outcomes and potentially prolonging life. Moreover, using both metadata and image data sets a new precedent for future AI/ML models, offering a synergistic approach for optimal results. It reinforces the need for multidimensional, nuanced data to create an accurate and holistic picture of a patient’s health. These findings represent important steps forward in the application of AI/ML in healthcare, highlighting an exciting path forward in the pursuit of precision medicine.
[ad_2]
Source link
