Multimodal AI for cancer detection

Funded by UK Research and Innovation’s Participatory Research Fund, Professor Annalisa Occhipinti from Teesside University’s School of Computing, Engineering & Digital Technologies is working to apply Artificial Intelligence (AI) in the development of a diagnostic tool for cancer.

Dr Occhipinti worked with a Research Assistant, Suraj Verma, to analyse patient-specific data from The Cancer Imaging Archive to create an AI model that could predict how patients will react to treatment.

Response to cancer treatments can vary across different patients, and experience of adverse effects can vary. Personalised treatment is necessary to improve patients’ outcomes, optimise treatment plans, and identify patients likely to benefit from a particular therapy. A multi-modal AI approach, that integrates images with clinical data, can provide comprehensive insights into tumour progression and is therefore critical for personalised treatment planning.

The development of a multi-modal AI tool will allow for improved accuracy and efficiency of cancer treatment by helping clinicians to personalise treatment plans based on each patient’s unique characteristics. The proposed approach has the potential to improve treatment decision-making and ultimately improve patient outcomes.

The predictive results of the developed AI model were shared with two radiologists at Gateshead Health NHS Foundation Trust, and South Tyneside and Sunderland NHS Foundation Trust, who added comments on the accuracy of the model’s predictions. This feedback informed the development of the model, and the final predictions were further analysed and discussed with the radiologists. The initial results of this model were recently published in Cell Reports Methods.

Through participatory research with radiologists, the model is informing new diagnostic methods that can be applied to different cancer types. Specifically, the work started with this initial funding is now progressing into a further collaboration with the Gateshead Breast Cancer Screening Unit, where a new type of mammogram, contrast-enhanced spectral mammography (CESM) is used to improve diagnosis during breast screening. The integration of this new image-based data modality into the AI model will further develop its potential as a diagnostic tool.

To further the work that was enabled by the £2,900 funding from the Participatory Research Fund, Dr Occhipinti has been successful in obtaining £10,000 funding from The British Society of Breast Radiology. This funding has allowed for the project to continue, with the analysis of further datasets.

The aim of the research is to develop a diagnostic tool that can be used by the NHS to inform clinical decision-making. This has been made possible through collaboration with clinicians across different NHS Trusts.