How does that work for reinforcement learning? +1 for shrinking tumor, -1 for death?
Machine-learning software has been trained to suggest the frequency and dosage of chemotherapy for patients suffering from glioblastoma, an aggressive form of brain cancer.
We know what you’re thinking: this sounds like a horrifically bad idea. It gets worse when you find out that the model was trained using reinforcement learning – the same technique used to teach bots how to play video games.
In this approach, agents are rewarded points when they do the right thing, and thus are encouraged to improve in the right direction. Shooting down an on-screen alien brings in a reward, losing a life results in a negative reward. In the end, they should develop the right sort of behaviors to master a topic.
This is all fine for fooling around with Atari games or Super Mario, but playing with chemo treatment is a no-no. The model gets a positive score when it recommends an anti-cancer drug dosage plan that manages to shrink a tumors, and presumably gets a gigantic negative score if it, well, poisons and kills you.
You may be screaming internally now. But you can relax, the whole experiment was carried out using fake simulated data for 50 patients. No humans were harmed in the making of this algorithm.
In fact, the machines were pretty conservative with their chemotherapy plans, Pratik Shah, a principal investigator at MIT’s Media Lab, said late last week . “We said [to the model], ‘Do you have to administer the same dose for all the patients? And it said, ‘No. I can give a quarter dose to this person, half to this person, and maybe we skip a dose for this person.”
But this was only after the model was threatened with a bad score during the training process if it was too aggressive in trying to shrink the tumor and suggesting full doses regularly. “If all we want to do is reduce the mean tumor diameter, and let it take whatever actions it wants, it will administer drugs irresponsibly. Instead, we said, ‘We need to reduce the harmful actions it takes to get to that outcome,’” said Shah.
The model was trained to propose the dosage for four chemotherapy drugs: temozolomide, procarbazine, lomustine, and vincristine. It ran 20,000 trial-and-error runs for each patient to come up with an optimal treatment plan during the training process.
It was then tested on a new batch of 50 patients in simulation, and, apparently, managed to reduce tumor sizes… in fake simulated humans. The researchers were interested in the potential of machine learning as computer systems can sift through more data, such as a person’s medical history, genetic profile, and particular biomarkers before making a decision. Some of these variables are not always considered in real clinical trials.
The research will be presented at the Machine Learning for Healthcare conference at Stanford University in the US this week.
“That was the most exciting part of this work, where we are able to generate precision medicine-based treatments by conducting one-person trials using unorthodox machine-learning architectures,” Shah concluded.
Exciting is one way to put it, we guess. ®
Sponsored: Following Bottomline’s journey to the Hybrid Cloud
