A Machine that Learns How to Learn – ‘Transformational’ approach to machine learning could accelerate search for new disease treatments

We humans get better and better at learning because we have learnt things in the past. A similar approach has been used by Ross D King and colleagues, to develop a new type of machine learning, called transformational machine learning. The method has been shown particularly successful in drug design and the results are now published in the latest issue of PNAS.

Machine learning is the branch of Artificial Intelligence (AI) that develops systems that learn from experience. Machine learning is behind all the recent advances in AI: self-driving cars, beating the World Champion at GO as well as predicting protein structures in biology.

In normal machine learning, individual problems are learnt one at a time. In contrast, in the new form of machine learning that Ross and colleagues have developed, the machine learning system learns information from other problems that helps it to better learn new problems. This is how human learning works: we get better at learning because we have learnt things in the past.

The researchers call the new approach ‘transformational machine learning’ (TML) as the representation for learning is transformed from the original one into one that uses the predictions of other problems. They demonstrate that this idea works on thousands of problems taken from science and engineering and show how the new approach can provide novel scientific insight.

TML is very closely related to and synergistic with stacking, multitask learning (MTL), and transfer learning (TL). It enables the utilization of knowledge previously learned from related tasks, rather than learning each new model from scratch. TML is a meta learning idea that is applicable to enhancing any non-linear ML method. It is particularly well suited when there exist many related small learning tasks.

The researchers demonstrated the effectiveness of their idea on thousands of problems from across science and engineering. They say it shows particular promise in the area of drug discovery, where this approach speeds up the process by checking what other ML models say about a particular molecule. A typical ML approach will search for drug molecules of a particular shape, for example. TML instead uses the connection of the drugs to other drug discovery problems.

“I was surprised how well it works – better than anything else we know for drug design,” said King. “It’s better at choosing drugs than humans are – and without the best science, we won’t get the best results.”

Ross D King has a professorship in Machine Intelligence and was recruited by WASP in 2019 as a Wallenberg Chair in AI. Since January 2020 his research group is situated at Chalmers University of Technology. Their new developments in TML will be published in this week’s issue of Proc. Nat. Acad. Sci. U.S.A (PNAS).

Reference

Olier, I., Orhobor, O.I., Dash, T., Davis, A., Soldatova, L.N., Vanschoren, J., King, R.D. (2021) Transformational Machine Learning: Learning How to Learn from Many Related Scientific Problems. Proc. Nat. Acad. Sci. U.S.A. (in press).