Reviewing the Accuracy of AI Provided Health Information

Image Credit: Adobe Stock Images/Atchariya63

In modern times, dependence on the internet when it comes to nutritional information has grown significantly. Despite this, close to half of online nutrition-related content (approximately 49%) has been found to be inaccurate or of low quality. Although artificial intelligence (AI) has the ability to transform electronic health, it is currently unknown how well it will perform when it comes to handling nutrition-related queries. To address this knowledge gap, the authors of a study published by JAMA investigated the reliability of AI in providing the energy and macronutrient content of 222 food items using different languages as inputs.¹

In order to find accurate answers, the authors compared the reliability of two AI chatbots (ChatGPT-3.5 and ChatGPT-4) in providing information on calorie and macronutrient content for eight menus designed for adults. Consistency was assessed based on the coefficient of variation, with accuracy determined by cross-referencing nutritionists' recommendations with influence from the Taiwanese Food and Drug Administration. Additionally, statistical analyses were performed using SPSS Statistics version 26. ¹

Results indicated that there were no significant differences in energy, carbohydrates, and fat estimations between nutritionists and AI; however, there was a difference observed in protein estimation. Both AI systems delivered correct energy contents for approximately 35% to 48% of the 222 food items within ±10%. ChatGPT-4 provided a better performance overall, but overestimated protein.¹

This information comes at a time where researchers believe there is great potential for AI in public well-being and nutrition. According to a 2023 Nutrients journal study on AI applications and public health, AI models have proved useful in visualizing and evaluating food surroundings and detecting places with restricted availability of nutritional foods. Furthermore, this study’s authors believe it also has the potential to predict regional nutritional needs, monitor global food trends, and harmonize nutritional guidelines internationally for consistent messaging.²

“The potential applications of AI in public health nutrition are vast, and the current body of research may only scratch the surface of what is achievable,” the Nutrients study authors wrote. “The richness of data and the evolving capabilities of AI offer a myriad of possibilities that are waiting to be explored. We encourage our fellow researchers to think beyond the traditional boundaries and explore innovative ways to harness the power of AI in promoting healthier diets and improving nutritional health at the population level.”

The authors of the JAMA study said that AI chatbots can be useful tools for obtaining energy and macronutrient information, offering real-time food analysis and the potential to transform nutritionists' patient communication. However, limitations include the general-purpose design of AI and the current inability to provide personalized dietary advice.¹

“Currently, the capability of AI-chatbots to provide personalized dietary advice, such as specific nutrition guidelines and exact portion sizes, is limited. ChatGPT is also unable to provide accurate common household units to consumers,” explained the authors. “Portion size and household units vary substantially depending on the food type, preparation method, and regional differences in measurement standards. These limitations likely stem from the nature of its training as a general-purpose design AI that is not specialized in the field of nutrition and dietetics. Future improvements in providing more accurate and practical nutrition information to customers will be important.”

References

1. Consistency and Accuracy of Artificial Intelligence for Providing Nutritional Information. JAMA Network. December 27, 2023. Accessed January 2, 2024. https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2813295

2. Artificial Intelligence Applications to Public Health Nutrition. MDPI. October 8, 2023. Accessed January 2, 2024. https://www.mdpi.com/2072-6643/15/19/4285