ChatGPT generates cancer treatment plans that are full of errors — Study finds that ChatGPT provided false information when asked to design cancer treatment plans::Researchers at Brigham and Women’s Hospital found that cancer treatment plans generated by OpenAI’s revolutionary chatbot were full of errors.
I suppose most sensible people already know that ChatGPT is not the answer for medical diagnosis.
If the researcher wanted to investigate whether LLM is helpful, they should develop a model specifically using cancer treatment plans with GPT-4/3.5 before testing it thoroughly, in addition to entering prompts into the model that is available on OpenAI.
Or they could feed the current model with a reputable source of medical information.
That wouldn’t guarantee correct answers.
It’s arguably more dangerous if ChatGPT gives mostly sane specific medical advice because it makes people put more trust in it than they should.
True but it would reduce the chances of it making stuff up entirely.