| Title: |
Achieving well-informed decision-making in drug discovery: a comprehensive calibration study using neural network-based structure-activity models. |
| Authors: |
Friesacher, Hannah Rosa1,2 (AUTHOR) rosa.friesacher@kuleuven.be, Engkvist, Ola2,3 (AUTHOR) ola.engkvist@astrazeneca.com, Mervin, Lewis4 (AUTHOR) lewis.mervin1@astrazeneca.com, Moreau, Yves1 (AUTHOR) yves.moreau@esat.kuleuven.be, Arany, Adam1 (AUTHOR) adam.arany@esat.kuleuven.be |
| Source: |
Journal of Cheminformatics. 3/5/2025, Vol. 17 Issue 1, p1-24. 24p. |
| Subject Terms: |
*DRUG discovery, *MATHEMATICAL statistics, *BAYESIAN analysis, *ARTIFICIAL intelligence, *DEEP learning, *LOGISTIC regression analysis |
| Abstract: |
In the drug discovery process, where experiments can be costly and time-consuming, computational models that predict drug-target interactions are valuable tools to accelerate the development of new therapeutic agents. Estimating the uncertainty inherent in these neural network predictions provides valuable information that facilitates optimal decision-making when risk assessment is crucial. However, such models can be poorly calibrated, which results in unreliable uncertainty estimates that do not reflect the true predictive uncertainty. In this study, we compare different metrics, including accuracy and calibration scores, used for model hyperparameter tuning to investigate which model selection strategy achieves well-calibrated models. Furthermore, we propose to use a computationally efficient Bayesian uncertainty estimation method named HMC Bayesian Last Layer (HBLL), which generates Hamiltonian Monte Carlo (HMC) trajectories to obtain samples for the parameters of a Bayesian logistic regression fitted to the hidden layer of the baseline neural network. We report that this approach improves model calibration and achieves the performance of common uncertainty quantification methods by combining the benefits of uncertainty estimation and probability calibration methods. Finally, we show that combining post hoc calibration method with well-performing uncertainty quantification approaches can boost model accuracy and calibration. Scientific contribution: In this work we provide a comprehensive probability calibration study using neural networks for drug-target interaction predictions. We report a significant effect of the hyperparameter selection strategy, as well as uncertainty estimation and probability calibration approaches on the reliability of uncertainty estimates, which is crucial for an efficient drug discovery process. [ABSTRACT FROM AUTHOR] |
|
Copyright of Journal of Cheminformatics is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) |
| Database: |
Academic Search Complete |
|
Full text is not displayed to guests. |
Login for full access.
|
