ChatGPT’s Agreement with the Recommendations from the 18th St. Gallen International Consensus Conference on the Treatment of Early Breast Cancer
Nabieva N, Brucker SY, Gmeiner B (2024)
Publication Type: Journal article
Publication year: 2024
Journal
Book Volume: 16
Article Number: 4163
Journal Issue: 24
Abstract
Introduction: Organizations like the European Society for Medical Oncology and the St. Gallen Oncology Conference panel regularly review the latest research data to align on common recommendations for the treatment of breast cancer patients. In the era of artificial intelligence (AI), the question arises whether AI can enhance scientific debates by providing potential recommendations for expert discussions. Methods: We focused on the St. Gallen International Breast Cancer Conference (SGBCC) in 2023, where 71 experts from 27 countries answered 127 questions across 17 topics related to early breast cancer. OpenAI’s ChatGPT version 4.0 was employed to respond to the same set of questions. We simulated response variability and mitigated potential memory effects using several question-rounds in new chat sessions. Results: ChatGPT answered 71 questions (55.91%) in accordance with the most common answer voted by the SGBCC panel and showed a moderate overall agreement. In these cases, AI voted with an average reliability of 98.31%, compared to the panel’s average majority of 65.39% for the most common answer. A very high agreement could be observed in questions on “Genetics”, “Pathology”, “Oligometastatic disease”, “Ductal carcinoma in situ” and “Well-being for breast cancer survivors”. A very low agreement was seen in the topics “BRCA associated”, “Adjuvant endocrine therapy”, “HER2 positive”, “Local/regional recurrence” and “Bone-modifying therapy”. Conclusions: Our study demonstrates that ChatGPT shows potential in the development of breast cancer treatment recommendations, particularly in areas where high agreement with expert panel responses was observed. However, significant improvements are necessary before AI can be considered a reliable tool to support human expertise.
Authors with CRIS profile
Naiba Nabieva
Department of Obstetrics and Gynaecology
Benjamin Gmeiner
Machine Learning and Data Analytics Lab
Involved external institutions
Germany (DE)How to cite
APA:
Nabieva, N., Brucker, S.Y., & Gmeiner, B. (2024). ChatGPT’s Agreement with the Recommendations from the 18th St. Gallen International Consensus Conference on the Treatment of Early Breast Cancer. Cancers, 16(24). https://doi.org/10.3390/cancers16244163
MLA:
Nabieva, Naiba, Sara Y. Brucker, and Benjamin Gmeiner. "ChatGPT’s Agreement with the Recommendations from the 18th St. Gallen International Consensus Conference on the Treatment of Early Breast Cancer." Cancers 16.24 (2024).
BibTeX: Download