Asking artificial intelligence tools about medical symptoms does not help people make better health decisions than traditional methods such as standard internet searches, according to a new study published in Nature Medicine.

The authors said the findings are significant as more people increasingly turn to AI chatbots for medical guidance, despite limited evidence that these tools offer safer or more effective advice.

How The Study Was Conducted

Researchers led by the University of Oxford Internet Institute worked with a group of doctors to design 10 medical scenarios. These ranged from mild conditions such as a common cold to severe emergencies, including brain bleeding caused by haemorrhage.

The scenarios were first tested without human participants using three large language models: OpenAI’s ChatGPT 4o, Meta’s Llama 3, and Cohere’s Command R+. The models correctly identified the medical condition in 94.9% of cases, but selected the correct next step, such as seeking urgent care, in only 56.3% of cases. The companies did not respond to requests for comment.

Human Use of AI Shows No Advantage

The researchers then recruited 1,298 participants in Britain and asked them to assess symptoms using either AI tools, their own experience, standard internet searches, or the National Health Service website.

When participants made decisions themselves, relevant medical conditions were identified in fewer than 34.5% of cases. The correct course of action was chosen in less than 44.2% of cases, a result no better than participants using traditional information sources.

Poor AI to Human Interaction

Adam Mahdi, a co-author of the study and associate professor at Oxford, said the results highlighted a significant gap between AI’s technical ability and its effectiveness when used by people.

He said the knowledge exists within AI systems, but that information does not consistently translate into useful guidance during real-world interactions, indicating the need for further research.

Where AI and Humans Go Wrong

The team reviewed around 30 interactions in detail and found that users often provided incomplete or inaccurate symptom descriptions. At the same time, AI systems sometimes produced misleading or incorrect responses.

In one example, a patient describing symptoms of a subarachnoid haemorrhage was correctly advised to go to the hospital after mentioning a stiff neck, light sensitivity, and the “worst headache ever.” Another patient describing similar symptoms but using the phrase “terrible headache” was instead advised to lie down in a dark room.

Researchers plan to conduct similar studies across different countries and languages to assess whether AI performance changes over time or in different settings.

The study received support from data company Prolific, the Dieter Schwarz Stiftung, and the UK and US governments.

The Pakistan Telecommunication Authority (PTA) has decided to make calls to toll-free numbers (0800-XXXXX) completely free for mobile phone users across Pakistan.

Under the new arrangement, mobile subscribers will no longer be charged for calling 0800 numbers. Previously, toll-free calls were free only for fixed-line users, while mobile users had to pay applicable charges despite the numbers being labeled as toll-free.

PTA said it took notice of the issue in the interest of consumer protection and user facilitation. The authority initiated steps to address the difference in treatment between mobile and fixed-line subscribers.

As part of the process, PTA held consultations with key stakeholders, including Pakistan Telecommunication Company Limited (PTCL), National Telecommunication Corporation (NTC), local loop operators, and cellular mobile operators. Following these discussions, all stakeholders agreed to extend toll-free calling to mobile users as well.

PTA said the decision reflects its commitment to providing user-friendly and fair telecommunication services. The move is intended to improve access to support services and ensure uniform telecom facilities for consumers across the country.