TY - JOUR
T1 - Disparities in medical recommendations from AI-based chatbots across different countries/regions
AU - Gumilar, Khanisyah E.
AU - Indraprasta, Birama R.
AU - Hsu, Yu Cheng
AU - Yu, Zih Ying
AU - Chen, Hong
AU - Irawan, Budi
AU - Tambunan, Zulkarnain
AU - Wibowo, Bagus M.
AU - Nugroho, Hari
AU - Tjokroprawiro, Brahmana A.
AU - Dachlan, Erry G.
AU - Mulawardhana, Pungky
AU - Rahestyningtyas, Eccita
AU - Pramuditya, Herlangga
AU - Putra, Very Great E.
AU - Waluyo, Setyo T.
AU - Tan, Nathan R.
AU - Folarin, Royhaan
AU - Ibrahim, Ibrahim H.
AU - Lin, Cheng Han
AU - Hung, Tai Yu
AU - Lu, Ting Fang
AU - Chen, Yen Fu
AU - Shih, Yu Hsiang
AU - Wang, Shao Jing
AU - Huang, Jingshan
AU - Yates, Clayton C.
AU - Lu, Chien Hsing
AU - Liao, Li Na
AU - Tan, Ming
N1 - Publisher Copyright:
© The Author(s) 2024.
PY - 2024/12
Y1 - 2024/12
N2 - This study explores disparities and opportunities in healthcare information provided by AI chatbots. We focused on recommendations for adjuvant therapy in endometrial cancer, analyzing responses across four regions (Indonesia, Nigeria, Taiwan, USA) and three platforms (Bard, Bing, ChatGPT-3.5). Utilizing previously published cases, we asked identical questions to chatbots from each location within a 24-h window. Responses were evaluated in a double-blinded manner on relevance, clarity, depth, focus, and coherence by ten experts in endometrial cancer. Our analysis revealed significant variations across different countries/regions (p < 0.001). Interestingly, Bing's responses in Nigeria consistently outperformed others (p < 0.05), excelling in all evaluation criteria (p < 0.001). Bard also performed better in Nigeria compared to other regions (p < 0.05), consistently surpassing them across all categories (p < 0.001, with relevance reaching p < 0.01). Notably, Bard's overall scores were significantly higher than those of ChatGPT-3.5 and Bing in all locations (p < 0.001). These findings highlight disparities and opportunities in the quality of AI-powered healthcare information based on user location and platform. This emphasizes the necessity for more research and development to guarantee equal access to trustworthy medical information through AI technologies.
AB - This study explores disparities and opportunities in healthcare information provided by AI chatbots. We focused on recommendations for adjuvant therapy in endometrial cancer, analyzing responses across four regions (Indonesia, Nigeria, Taiwan, USA) and three platforms (Bard, Bing, ChatGPT-3.5). Utilizing previously published cases, we asked identical questions to chatbots from each location within a 24-h window. Responses were evaluated in a double-blinded manner on relevance, clarity, depth, focus, and coherence by ten experts in endometrial cancer. Our analysis revealed significant variations across different countries/regions (p < 0.001). Interestingly, Bing's responses in Nigeria consistently outperformed others (p < 0.05), excelling in all evaluation criteria (p < 0.001). Bard also performed better in Nigeria compared to other regions (p < 0.05), consistently surpassing them across all categories (p < 0.001, with relevance reaching p < 0.01). Notably, Bard's overall scores were significantly higher than those of ChatGPT-3.5 and Bing in all locations (p < 0.001). These findings highlight disparities and opportunities in the quality of AI-powered healthcare information based on user location and platform. This emphasizes the necessity for more research and development to guarantee equal access to trustworthy medical information through AI technologies.
KW - Artificial intelligence
KW - Bard
KW - Bing
KW - ChatGPT
KW - Disparity
KW - Endometrial cancer
UR - http://www.scopus.com/inward/record.url?scp=85199505313&partnerID=8YFLogxK
U2 - 10.1038/s41598-024-67689-0
DO - 10.1038/s41598-024-67689-0
M3 - Article
C2 - 39048640
AN - SCOPUS:85199505313
SN - 2045-2322
VL - 14
JO - Scientific Reports
JF - Scientific Reports
IS - 1
M1 - 17052
ER -
