cool hit counter IBM oncologist "Watson" was exposed as not working well, the future of medical AI is worried?_Intefrankly

IBM oncologist "Watson" was exposed as not working well, the future of medical AI is worried?

► The first cancer patient in South Korea to receive Watson's oncology-assisted treatment is browsing information about his disease with an oncologist at Kagawa University's Yoshi Medical Center. Source: Kagawa University Yoshi Medical Center

Editor's Notes.

Recently, Watson, IBM's AI product used to assist doctors in designing cancer treatment plans, has been exposed to numerous problems, including the possibility of prescribing dangerous and incorrect cancer treatment plans. What impact will this have on the healthcare AI industry? Several experts working on medical AI research or industrialization expressed their views.

By | | | | CHIANG HOI YU

Editor Chen Xiaoxue


"This product is very bad. We were buying this product for a hospital promotion in the hope that it would deliver on IBM's stated vision. It turns out that for the most part, it doesn't work at all. "A doctor at Jupiter Medical Center (Florida, USA) said this about their Watson for Oncology in the presence of IBM executives.

Yes, the same Watson, the medically assisted artificial intelligence system that was once favored and stirred up so much controversy. Watson Oncology is very easy to use: simply enter the patient's personal information (e.g. medical history, imaging reports, etc.) into the system, and the system recommends the appropriate treatment plan for the doctor's reference based on a large amount of medical research, medical guidelines, clinical trials, and other information.

On July 25, STAT, a US health care media outlet, exposed internal documents from IBM showing that Watson, an AI product used to assist doctors in designing cancer treatment plans, is notoriously problematic: in addition to the kind of complaints from users above, it can prescribe dangerous and incorrect cancer treatment plans.

Watson Oncology is IBM's key product in the field of medical artificial intelligence, capable of providing treatment plans for a variety of cancers, including breast, lung, rectal, and colon cancers, based on patient medical records and other information, with partners such as Memorial Sloan Kettering Cancer Center, Mayo Clinic, Quest DIAGNOSTICS, and other prominent institutions in the medical field. According to the official website of its Chinese agency, nearly 80 hospitals in more than 20 provinces in China have introduced Watson Oncology Consultation Center.

IBM "family scandal" leaked

STAT received internal documents from Andrew Norden, then IBM Watson Health's deputy chief health officer, in the summer of 2017, on slides used in an internal company presentation.

According to STAT, IBM is unrelenting in its criticism of Watson in this report, with key points including.

The ∆Watson system is trained using data not from real patients, but from hypothetical patients Hypothetical data

Insufficient training data . The slide shows eight types of cancer. As of the day of the report, the highest volume of training data was only 635 cases of lung cancer, while the lowest was only 106 cases of ovarian cancer.

△ When training Watson, the treatment regimen recommended for the hypothetical patient was based on the protocols of specialists at Memorial Sloan-Kettering Cancer Center, the Not medical guidelines or real evidence

Δ In the test of a hypothetical situation. Watson prescribed an inappropriate and dangerous treatment plan . For example, it recommends concurrent chemotherapy and Avastin for a patient with lung cancer who shows the possibility of being bleeding severely. However, Avastin may cause severe bleeding and should not be recommended for patients who are already bleeding.

△ The experiments used to assess the similarity of protocols between the Watson system and cancer specialists may be biased, making the two protocols easily identical.

In an interview with STAT, Jana Eggers, CEO of artificial intelligence company Nara Logics, noted that the Watson system is clearly not using the big data in the healthcare system, "I can't figure out why they're generating a bunch of fantasy patients out of it when there's real data from real people. "

Some experts believe that, If these Hypothetical data Representative of real patient conditions, It also trains Watson very well.。 merely,“ We also need to see evidence that the data is representative”, Associate Professor, Center for Bioinformatics Research, Stanford UniversityJonathan Chen talk。

In fact, the questioning of Watson never ends. Back last year, in 2017, STAT published a survey of Watson's healthcare AI system that addressed all of these issues. Professor Claudia Perlich, formerly of IBM's Watson Research Center, said in a 2017 interview with Gizmodo that Watson Healthcare is "child's play": "Our perception in the data science community is that anything Watson can do, you should probably find free software to do it, or make one yourself. "

This internal document, more than anything else, exposes IBM's own perception of Watson's limitations.

In May of this year, IBM made cuts to its Watson Healthcare division. IBM spokesman Ian Colley told the Institute of Electrical and Electronics Engineers (IEEE) that only a small number of Watson Healthcare staff were laid off and that the layoffs were only part of a streamlining of the department. However, one of the engineers who was laid off revealed that about 80% of the employees were forced to leave. "The people who left were important technical people and people who dealt face-to-face with customers, not insignificant administrative people." This engineer said.

Recommended treatment is not mainstream

There are many application scenarios for artificial intelligence in healthcare, including medical imaging, genomic information processing, drug development, health data management, guide robots, and more. Applications in clinical care have the most significant impact on existing medical practice and therefore receive the most attention. Applications like Watson Oncology, which attempts to recommend treatment options based on patient information, are just one of many, and not the mainstream.

Currently, in the clinical context, medical AI is mostly related to medical imaging, focusing on providing doctors with various tools or optimizing existing imaging tools to help them diagnose or treat.

For example, AI diagnostic tools can identify lesions from medical images that are invisible to the human eye, or easily missed. In this regard, screening for lung cancer based on CT image identification of lung nodules is one of the most common applications today.

► Using machine learning to identify lung nodules. Photo credit: Vatsal Sodha, Medium

There are also teams that are using machine learning to improve the speed and quality of medical imaging. Those of you who have had x-rays may remember that the doctor would tell the patient to hold his or her breath. The purpose of this is to prevent breathing movements from affecting imaging. X-ray imaging takes only a few seconds, but positron emission computed tomography (PET), which usually takes more than ten minutes, is not something that can be solved by holding your breath. Many research teams are currently, through automatic gating techniques, repairing the effect of breathing on PET imaging.

► Original PET imaging (a) and PET imaging corrected with automatic gating technique (b), the (b) image has less artifacts. Image credit: Walker et al. 2018.

Artificial intelligence can also turn 2D images into 3D images to help surgeons perform surgery. The Minimally Invasive Treatment and 3D Imaging Laboratory, led by Professor Hongen Liao of Tsinghua University, applies intelligent analysis of medical images to intravascular interventions, allowing doctors to see the exact location of the surgical catheter in the vessel. Their evaluation showed that the error between the catheter shape in the images and the actual catheter was only 2.23 ± 0.87 mm (Chen et al. (2017). The study is still being conducted in animal experiments. In addition to this, they are using the technology for orthopedic grafts, allowing doctors to see the three-dimensional shape of a patient's femur and helping them match the right implant.

► Schematic diagram of the endovascular intervention

(Source: Capture Vascular, YouTube)

There are also applications, Tian Jie, a researcher at the Institute of Automation at the Chinese Academy of Sciences, told The Knowledgeable, that are helping patients avoid unnecessary surgery through artificial intelligence. Before a surgeon operates on a patient with colorectal cancer, he or she will give the patient an adjuvant chemotherapy treatment to control the progression of the cancer and then operate afterwards. Some patients are in complete remission after adjuvant chemotherapy, but doctors cannot tell if there are still cancer cells in the patient's body. Tian Jie described a collaborative study between his institute and Peking University Cancer Hospital, which, by analyzing medical images, has a 90 percent certainty of screening out those patients who have gone into remission from surgery.

The North American Congress of Radiology describes the medical imaging center of the future this way: "The medical imaging center is like the cockpit of an airplane, an amalgam of all kinds of information; and the physician of the future is the equivalent of the pilot, dealing with all kinds of information." Tianjie agrees with this statement and adds, "In my opinion, AI will not replace doctors, it will only assist them more effectively. And physicians should not be afraid of emerging technology, but actively take advantage of it and use it. "

The commercial future of medical AI is uncertain

The process of moving from AI research in the lab to commercialized products outside the lab is not a simple one. A statistic from YeoSmart shows that 7 of the 11 multi-million dollar or more medical AI companies are related to medical imaging. There are now several companies in major tertiary hospitals. These products primarily use target recognition to target a particular disease and aid in the diagnosis of the disease.

It remains to be seen how helpful the current AI products will be to physicians, as they all target only a single task. For example, Tencent Foraging targets images of the lungs to identify lung nodules. "The doctor looks at the film not only to find the nodule, but also to characterize it. In addition, we have other lesions in the lungs, such as: there may also be infarcts, there may be bronchiectasis, and fibrosis of the lungs. "If the system can combine the finding of five or six common diseases, it will meet more than 90% of the work of reading chest films before we can basically say that it helps our doctors and reduces our burden," noted Liang Changhong, director of the Department of Imaging Medicine and Director of the Department of Radiology at Guangdong Provincial People's Hospital. "

The Watson tumor problem was exposed, and Huang Feng, chief scientist of Neusoft Medical, believes that the IBM problem will not affect the related industries in China. "The role of AI is multifaceted, and IBM has chosen the riskiest applications. Domestic companies are much more pragmatic. "I'm sorry," said Huang Feng.

As of now, the commercial future of healthcare AI is unclear. "I think they're still lacking a better business model, they're basically not generating profits, and they're basically still exploring business models and using data to refine their products." Now it's basically giving the system to the hospital and using the hospital's existing data to train the system," said Changhong Liang. "In this sense, AI is currently more dependent on hospital data than hospitals need for AI systems.

"AI-assisted treatment systems have not been approved for formal clinical use. Before they can be used in the clinic, they need to be approved by the FDA for safety and efficacy. And so far, none of the systems have been approved. "It's a good idea," said Tianjie.


1. IBM’s Watson supercomputer recommended ‘unsafe and Incorrect’ cancer treatments, internal documents show,STAT,

2. Layoffs at Watson Health Reveal IBM’s Problem With AI, IEEE,

3. Why Everyone Is Hating on IBM Watson—Including the People Who Helped Make It, Gizmodo,

4. Chen, Fang, Jia Liu, and Hongen Liao. "3D Catheter Shape Determination for Endovascular Navigation Using a Two-Step Particle Filter and Ultrasound Scanning." IEEE transactions on medical imaging 36, no. 3 (2017): 685-695.

5. Walker, Matthew D., Kevin M. Bradley, and Daniel R. McGowan. "Evaluation of principal component analysis-based data-driven respiratory gating for positron emission tomography." The British journal of radiology 91, no. 1085 (2018): 20170793.

6.China Healthcare Artificial Intelligence Development Research Report 2018, Yioji, 2018

Copy Editor|Pippy Fish

2、A layer can follow the finished drawn line to move ios program fun ah
4、SpringDataRedis Usage
5、Log processing in serverless architectures

    已推荐到看一看 和朋友分享想法
    最多200字,当前共 发送