AMIE has a long way to go to clinical diagnosis and treatment diseases, and it can only be a assistant to real doctors in the clinic, and cannot diagnose the disease alone.One of the most fundamental problems is that even if it is stronger than a real doctor in some ways, it has a higher error rate with other AI software.
Recently, Google Deep Thinking (DeepMind) published a paper saying that the new diagnostic dialogue artificial intelligence (AI) -AMIE defeated the doctor in the test and passed the Turing test.
In the 1024 major scientific progress released by Natural Magazine recently, artificial intelligence occupies the top of the list.
The voice did not fall, there was a major breakthrough in AI in the medical field.AMIE defeated the doctor in the test and passed the Turing test. Does AI mean that AI has entered the clinic and replaced the real doctor?
Of course impossible, because Dr. AI has no prescription so far, and people will not easily give the diagnosis and treatment right to AI.
To understand the situation, we must also look at the AI breakthrough.
Researchers have built a standard for experimental assessment clinical diagnosis based on clinical consultation and communication skills, which involving the standard of medical history collection, diagnostic accuracy, clinical management, clinical communication skills, relationship training and emotional emotional.After that, the researchers created a dialogue -based diagnostic research AI system based on the large language model (LLM), namely AMIE, and trained AMIE with real -world big data.These data include medical reasoning, medical summary, and clinical dialogue in real world.
Finally, the researchers designed a random and double -blind cross -study to allow verified patients to interact with the certified primary health doctor (PCP) and AMIE through text chatIn the form of clinical examination (OSCE), set up consultation (diagnostic) scenarios.OSCE is a practical evaluation method commonly used in real worlds to examine the skills and abilities of clinicians in a standardized and objective way.
In a random and double -blind cross -study, researchers used OSCE to evaluate the diagnosis of 20 junior health doctors, specialist chief doctors, and AMIE. Patients were simulated by professional actors.Studies include 149 cases of cases from OSCE providers from Canada, Britain and India, involving various specialties and diseases.
The results show that AMIE's performance in the simulation diagnosis dialogue is at least comparable to junior health doctors.Moreover, from the perspective of a specialist, the diagnosis of AMIE is more accurate, and it performs better on the 28 indicators of 32 indicators. From the perspective of the patient, it performs on the 24 indicators of 26 indicators.better.In addition, AMIE has surpassed primary health doctors in the evaluation set by researchers.However, this AI doctor only defeated doctors in the diagnosis of two types of diseases: respiratory system and cardiovascular disease. It did not have more accurate diagnosis than primary health doctors in all diseases, so it cannot be inferred to all disease prevention and control areas.
On the other hand, in the experiment, AMIE passed the Turing test and showed some kind of emphasis, but this does not mean that AI doctors have thought and even have similar consciousness to others.The problem is that the Turing test standard is too old.Under this test, the tester asked the tester at will. After multiple tests, if more than 30%of the testers cannot determine whether the tester is a person or a machine, the machine passed the test.The ratio of 30%is too small.Besides, Turing test confuses the result of "deceiving" and "thinking like people". Therefore, passing Turing test does not mean that AMIE can think, analyze and judge like a real doctor, and perform correct diagnosisEssence
It should be acknowledged that AMIE is indeed strong, and is better than Dr. Watson created by the International Business Machinery Company (IBM) in 2011.
Dr. Watson was the initial AI doctor. The accuracy of diagnosis of the disease reached 73%, and it was a diagnosis of various diseases.Dr. Watson stored millions of documents, including dictionary, encyclopedia, medical files, news, literature, and other reference materials that can establish knowledge bases. You can read 1 million books in 1 second.A series of calculations, including grammatical analysis, searching for each knowledge base, extracting alternative answers, searching for evidence of alternative answers, and calculating and comprehensive calculation and synthesis of evidence.
Amie has great improvements. In addition to receiving big data training, it can also obtain information and learn from real medical conditions and scenarios.Researchers have designed a self -performed simulation learning environment. The environment has an automatic feedback mechanism that is used for diagnostic medical dialogue in the virtual medical environment, so AMIE can expand the ability to learn knowledge in a variety of medical conditions and environment.
At the same time, researchers also use the hospital's real diagnosis and treatment of the disease environment to continuously change the simulation dialogue, repeatedly fine -tuning AMIE, and allowing AMIE to perform two self -performers cycles.The first is the internal self -acting cycle, and the feedback of the dialogue in the context is used to improve the learning; the second is the external self -performed cycle, that is, a complete simulation dialogue set is included in the subsequent fine -tuning iteration.
The new version of AMIE generated by this can participate in the internal cycle again, forming a benign continuous learning cycle.
Amie also designed the reasoning time chain strategy, enabling AMIE to gradually improve its answer according to the current dialogue, so as to draw the diagnosis that can best reflect the current specific situation.Therefore, the accuracy of the diagnosis of respiratory systems and cardiovascular disease is higher.
Even so, AMIE has a long way to go to clinical diagnosis and treatment of diseases, and it can only be the assistant of a real doctor to enter the clinic, and cannot diagnose the disease alone.One of the most fundamental problems is that even if it is stronger than a real doctor in some ways, it has a higher error rate with other AI software.
It is also a result of just published in a scientific magazine about AI to help doctors diagnose schizophrenia. The artificial intelligence model can predict the treatment results of the patients in the samples of the trained samples, but it should be applied to the initial initially at the beginningWhen the samples or different datasets, their performance drops to slightly better than accidental.
In other words, the use of different AI doctors has great limitations.Therefore, they can only intervene in medical treatment as assistants of real doctors, and cannot diagnose and treat diseases alone.
The author is Beijing scholar
