Estimation of acute oral toxicity in rat using local lazy learning
- Jing Lu†1, 2,
- Jianlong Peng†2,
- Jinan Wang2,
- Qiancheng Shen2,
- Yi Bi1,
- Likun Gong2,
- Mingyue Zheng2Email author,
- Xiaomin Luo2Email author,
- Weiliang Zhu2,
- Hualiang Jiang2, 3, 4 and
- Kaixian Chen2, 3
© Lu et al.; licensee Chemistry Central Ltd. 2014
Received: 28 March 2014
Accepted: 6 May 2014
Published: 16 May 2014
Acute toxicity means the ability of a substance to cause adverse effects within a short period following dosing or exposure, which is usually the first step in the toxicological investigations of unknown substances. The median lethal dose, LD50, is frequently used as a general indicator of a substance’s acute toxicity, and there is a high demand on developing non-animal-based prediction of LD50. Unfortunately, it is difficult to accurately predict compound LD50 using a single QSAR model, because the acute toxicity may involve complex mechanisms and multiple biochemical processes.
In this study, we reported the use of local lazy learning (LLL) methods, which could capture subtle local structure-toxicity relationships around each query compound, to develop LD50 prediction models: (a) local lazy regression (LLR): a linear regression model built using k neighbors; (b) SA: the arithmetical mean of the activities of k nearest neighbors; (c) SR: the weighted mean of the activities of k nearest neighbors; (d) GP: the projection point of the compound on the line defined by its two nearest neighbors. We defined the applicability domain (AD) to decide to what an extent and under what circumstances the prediction is reliable. In the end, we developed a consensus model based on the predicted values of individual LLL models, yielding correlation coefficients R2 of 0.712 on a test set containing 2,896 compounds.
Encouraged by the promising results, we expect that our consensus LLL model of LD50 would become a useful tool for predicting acute toxicity. All models developed in this study are available via http://www.dddc.ac.cn/admetus.
KeywordsAcute toxicity Local lazy learning Applicability domain Consensus model
Estimation of rodent acute toxicity is an important task in the safety assessment of drug candidates. Median lethal dose (LD50), a dose causing 50% death of the treated animals in a given period when administered in an acute toxicity test , is a common criterion that measures acute toxicity of compound. However, due to ethical reasons, the animal experiments on rodent acute toxicity are highly controversial. European Union Registration, Evaluation, Authorization and Restriction of Chemicals (REACH) has recommended the use of in vitro or in silico methods instead of animal testing of LD50. This proposal drives the development of quick, reliable, and non-animal predicting methods such as quantitative structure-toxicity relationships (QSTRs).
Acute toxicity involves multiple biochemical mechanisms, and a large number of compounds have been reported for their LD50 information, which covers a significant portion of chemical diversity space. These complexities pose a big challenge to the building of a single QSAR model with high prediction accuracy. Taking the acute rodent toxicity as an example, Enslein et al.[3, 4] developed multiple linear regression (MLR) models based on noncongeneric datasets, and found that the models had poor prediction power. To increase the prediction accuracy, Eldred et al. and Guo et al. built a few local models based on congeneric datasets. This type of models has improved accuracy, but their application ranges are limited. Zhu et al. introduced the applicability domain (AD) in their study, and constructed consensus model from multiple individual models using k nearest neighbors (KNN), random forest, hierarchical clustering, and so on. The consensus model showed improved results as compared to the individual constituent models, while the prediction accuracy is still limited when the model coverage increases.
Due to the complex mechanisms of acute toxicity, we explored the similarity-based local models to study the rat LD50 data by oral exposure. The basic idea of such models follow that “structurally similar molecules are likely to have similar properties”, which is suitable for modeling very complex boundaries between two classes . In light of the idea, Yuan et al. proposed a method “Clustering first, and then modeling”. It means the training set members are firstly grouped together based on their structural similarity. Then, the test set member is assigned to a specific group according to its structural resemblance to the group members, and its toxicity value is next predicted using an on-the-fly constructed model from the group. This method shows good performance for the datasets with distinct clusters, but it has the disadvantage of requiring a priori knowledge of the number of clusters. In this study, we try to use local lazy learning (LLL) to solve this problem. Given a test compound, LLL method firstly find its k nearest neighbors in the training set by using a predefined property set (molecular fingerprints or descriptors), and then build local models using these compounds to predict the value of the test compound. This method can fully consider the structural information of every test compound, while doesn’t rely on a priori knowledge of clusters. Moreover, to further improve the prediction accuracy, we try to enrich the reference data set and construct consensus models, which are critical for reducing the high variance of individual models. In the end, we analyze the application domain of the resulted models.
Results and discussion
Performance evaluation of LLL models
Performance of four LLL models using different similarity metrics on the test set using reference set I (Group I) versus the best model of the reference
Performance improvement by constructing consensus model
Performance improvement by enriching the reference set
Performance of our models on the test set using the reference set II (Group II)
The results of the test compound flocoumafen (cas: 90035-08-8) and its neighbors from the reference set I and II
The neighbors from reference set I
The neighbors from reference set II
Effects of applicability domain
As the final consensus model is constructed by taking the arithmetic average of all LLL models, its reliability to predict a test compound highly depends on its constituent models. Therefore, the “consensus prediction fraction” (i.e., the ratio of individual models being reliable to predict a new compound), is used to define the AD of the final consensus model . In the current study, the final model comprises four kinds of individual LLL models, and if one of them can reliably predict a new compound, the “consensus prediction fraction” is 25% for this compound. Only if the “consensus prediction fraction” is larger than or equals to a predefined threshold, the final consensus model is considered as reliable. When the threshold is set to be 75%, there are totally 2,378 test compounds within the AD, on which the final consensus model has significantly improved performance. For example, for the final consensus model of Group II, the MAEs of all, within, and outside are 0.422, 0.358, and 0.523, respectively. Obviously, the application of AD can tell us when the final consensus model provides a reliable and better estimation of acute oral toxicity in rat.
The rat LD50 data by oral exposure were collected from Zhu et al., United States Environmental Protection Agency dataset , and Accelrys Toxicity database 2011.4 . The final dataset included 9,617 compounds after removing the duplicated and wrong structures. Among them, there are 3,472 and 3,874 compounds identical to Zhu’s training set and test set, which will be used as the reference Set I and the test set, respectively. For comparison, we prepared three subsets representing different prediction coverage, in which the test compounds were ranked ascendingly according to their distance to their nearest neighbors, and then the first 2,896, 2,583 and 743 compounds were selected, respectively, to comprise multiunit test sets (hereafter called “Set_2896”, “Set_2583”, “Set_743”). In addition we also constructed an expanded reference set named the reference set II, which contains 5,743 compounds including the whole reference Set I, the compounds from EPA dataset and Accerlys Toxicity dataset. The original unit of LD50 was firstly converted to -log(mol/kg) to conform to the standard QSAR practice.
Feature sets and similarity measurement
For each given query compound, four sets of k nearest neighbors were retrieved from the reference set using different feature sets. Then local lazy learning strategies were applied to construct local models, from which consensus model was built. All the computation was done using in-house C/Python programs.
In this study, four kinds of local lazy learning schemes were combined with four similarity metrics to predict the acute toxicity in rat. Different from the conventional global QSAR models built upon the entire diverse data set, these LLL models were constructed “on-the-fly” by only utilizing the analogical compounds of a query. Accordingly, the detailed and subtle local structure-toxicity relationships around the query compound can be captured, which might be otherwise overshadowed by the large amount of employed training compounds in global models. As the approach relies on a priori knowledge about the toxicity profile of a query’s neighbors, its prediction accuracy can be improved by enriching the size and the structural diversity of the reference set, and the “on-the-fly” feature of LLL models also allows for a timely update and expansion. To reduce the high variance of individual LLL models, a consensus modeling scheme was employed, which further improved the accuracy of LD50 prediction. For the “Set_2896”, the R2 of the final consensus model using reference set II was enhanced from 0.545 to 0,712, and the MAE of prediction was reduced to 0.385. Moreover, by introducing the concept of AD, the reliability of a predication can be evaluated. For the compounds within AD, their toxicity can be more accurately predicted. The outstanding performance of our approach suggests that LLL models are feasible and effective for in silico prediction of acute oral toxicity in rat. We expect this method would also be a useful tool to provide inspiration for discovering novel drug candidates with favorable safety profile.
kNN-based LLL Models
Local lazy regression (LLR): For each test compound, only one most relevant descriptor was selected to build a linear equation based on k nearest neighbors. During the procedure, both the descriptor and the number of nearest neighbors were optimized. Initially let k = 5 and Q 2 = −108 (an arbitrary negative value that will be updated during the iteration), the algorithm was described as follows: (1). Select k nearest neighbors of the query compound (q) from the reference set. (2). Use one single descriptor to build a linear regression model, and perform a leave-one-out (LOO) cross-validation for the model. Note here the descriptor value calculated for the query was compared with those calculated for its neighbors. If the query’s value falls outside the range of its neighbors, the descriptor was disregarded to avoid yielding an extrapolated value. (3). After traversing all descriptors, record the descriptor D (k) that leads to the highest LOO Q 2 k . (4). If Q 2 k > Q 2, update Q 2 and D (k), then add one more neighbor, and repeat steps 1–3 until k > 20; otherwise, the iteration is terminated, and let k = k-1. Finally, the query compound was predicted by a linear model using k nearest neighbors and the descriptor D (k) . Figure 4 shows the flowchart of the LLR modeling.
- d.GP model: For a query compound C and any two neighbors A and B, a triangle can be constructed in a multidimensional descriptor space (Euclidean space). As shown in Figure 5, the value of projection point D is used as an estimate of C, and under the assumption that the y-values is linearly changed along the line AB, the value of D can be calculated as follows:(5)
where S is the similarity between a pair of neighbors, y D is obtained by Equation (5). Of note the above definitions assume that none of the edges of a triangle is degenerated. If a query compound has a neighbor with a zero distance, its LD50 is directly estimated by that neighbor. Moreover, a proper k was automatically optimized for each test compound by applying the same strategy showing in Figure 4. This procedure is similar to that of LLR except that the initial k value was set to 3.
The LLL methods in this study are all similarity based, of which the decision boundary or value largely depends on the input points and their particular positions. However, individual models using different modeling methods or similarity measurements could vary significantly and capture different part of the relationship. To reduce the high variance, consensus model can be established by combining each individual model, which has been demonstrated to be an effective means to improve the performance of similarity based methods [7, 25–29]. In this study, the strategy as used in Zhu et al. was applied to build consensus model, in which the predicted toxicity for each compound equals to the arithmetical mean of all predicted values of individual models. For each type of LLL method, a consensus model was constructed by averaging the results using different similarity metrics, named as “LLR_consensus”, “SA_consensus”, “SR_consensus”, and “GP_consensus”, respectively. Besides, a consensus model named “Final_consensus” was built by averaging all the 16 individual models.
Applicability domain (AD)
Where is the average Tanimoto or Euclidean distance between all compound and their nearest neighbor in the reference set, σ is the standard deviation of these distances, and Z is an arbitrary parameter to control the threshold level (here set to 0.5). If the distance of the compound to its nearest neighbor exceeds this threshold, this test compound is treated as an “outlier”, and the prediction result is considered to be unreliable.
Median lethal dose
Local lazy learning
Local lazy regression
Quantitative structure-toxicity relationships
- kNN k:
Mean absolute error.
This work was supported by Hi-TECH Research and Development Program of China (Grant 2012AA020308), National S&T Major Project (Grant 2012ZX09301-001-002), and National Natural Science Foundation of China (81220108025, 81001399, 2013ZX09507001). We would like to thank Dr. Hao Zhu and Todd Martin for valuable dataset of rat LD50.
- Turner R: Acute toxicity: The determination of LD50. Screening Methods in Pharmacology. 1965, New York: Academic Press, 300-Google Scholar
- European Parliament and of council: Regulation (EC) No 1907/2006 of the European Parliament and of the Council of 18 December 2006 concerning the Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH), establishing a European Chemicals Agency, amending Directive 1999/45/EC and repealing Council Regulation (EEC) No. 793/93 and Commission Regulation (EC) No. 1488/94 as well as Council Directive 76/769/EEC and Commission Directives 91/155/EEC, 93/67/EEC, 93/105/EC and 2000/21/EC. 2006, Official Journal of the European Union 396, 1-849.Google Scholar
- Enslein K: A toxicity estimation model. J Environ Pathol Toxicol. 1978, 2: 115-121.Google Scholar
- Enslein K, Lander TR, Tomb ME, Craig PN: A predictive model for estimating rat oral LD50 values. Toxicol Ind Health. 1989, 5: 261-387.Google Scholar
- Eldred DV, Jurs PC: Prediction of acute mammalian toxicity of organophosphorus pesticide compounds from molecular structure. SAR QSAR Environ Res. 1999, 10: 75-99. 10.1080/10629369908039170.View ArticleGoogle Scholar
- Guo JX, Wu JJ, Wright JB, Lushington GH: Mechanistic insight into acetylcholinesterase inhibition and acute toxicity of organophosphorus compounds: a molecular modeling study. Chem Res Toxicol. 2006, 19: 209-216. 10.1021/tx050090r.View ArticleGoogle Scholar
- Zhu H, Martin TM, Ye L, Sedykh A, Young DM, Tropsha A: Quantitative structure-activity relationship modeling of rat acute toxicity by oral exposure. Chem Res Toxicol. 2009, 22: 1913-1921. 10.1021/tx900189p.View ArticleGoogle Scholar
- Johnson MA, Maggiora GM: Concepts and Applications of Molecular Similarity. 1990, New York: John Wiley & SonsGoogle Scholar
- Yuan H, Wang Y, Cheng Y: Local and global quantitative structure-activity relationship modeling and prediction for the baseline toxicity. J Chem Inf Model. 2007, 47: 159-169. 10.1021/ci600299j.View ArticleGoogle Scholar
- Adamson GW, Bawden D, Saggers DT: Quantitative structure-activity relationship studies of acute toxicity (LD50) in a large series of herbicidal benzimidazoles. Pestic Sci. 1984, 15: 31-39. 10.1002/ps.2780150106.View ArticleGoogle Scholar
- Oral rat 50 percent lethal dose: [http://www.epa.gov/nrmrl/std/qsar/qsar.html] (accessed July 12, 2012)
- Accelrys Toxicity Database. 2011, San Diego, CA: Accelrys Software Inc, .4
- Petterino C, Paolo B: Toxicology of various anticoagulant rodenticides in animals. Vet Hum Toxicol. 2001, 43: 353-360.Google Scholar
- Sybyl. St. Louis, MO: Tripos Inc, 63144–2913
- AMPAC. Shawnee, KS: Semichem, Inc, 62216
- COmprehensive DEscriptors for Structural and Statistical Analysis (CODESSA). Shawnee, KS: Semichem, Inc, 66216
- Accelrys Discovery Studio. San Diego, CA: Accelrys Software Inc, 92121
- Willett P, Barnard JM, Downs GM: Chemical similarity searching. J Chem Inf Comput Sci. 1998, 38: 983-996. 10.1021/ci9800211.View ArticleGoogle Scholar
- Rogers D, Hahn M: Extended-connectivity fingerprints. J Chem Inf Model. 2010, 50: 742-754. 10.1021/ci100050t.View ArticleGoogle Scholar
- MDL Information Systems, Inc. 14600 Catalina Street, San Leandro, CA, 94577
- Landrum G: RDKit: Open-Source Cheminformatics. [http://www.rdkit.org] (release 2013.03.2)
- Liu J, Yang L, Li Y, Pan D, Hopfinger AJ: Prediction of plasma protein binding of drugs using Kier-Hall valence connectivity indices and 4D-fingerprint molecular similarity analyses. J Comput Aided Mol Des. 2005, 19: 567-783. 10.1007/s10822-005-9012-4.View ArticleGoogle Scholar
- Liu J, Yang L, Li Y, Pan D, Hopfinger AJ: Constructing plasma protein binding model based on a combination of cluster analysis and 4D-fingerprint molecular similarity analyses. Bioorg Med Chem. 2006, 14: 611-621. 10.1016/j.bmc.2005.08.035.View ArticleGoogle Scholar
- Lipkus A: A proof of the triangle inequality for the Tanimoto distance. J Math Chem. 1999, 26: 263-265. 10.1023/A:1019154432472.View ArticleGoogle Scholar
- Zhang S, Golbraikh A, Oloff S, Kohn H, Tropsha A: A novel automated lazy learning QSAR (ALL-QSAR) approach: method development, applications, and virtual screening of chemical databases using validated ALL-QSAR models. J Chem Inf Model. 2006, 46: 1984-1995. 10.1021/ci060132x.View ArticleGoogle Scholar
- Zhang S, Golbraikh A, Tropsha A: Development of quantitative structure-binding affinity relationship models based on novel geometrical chemical descriptors of the protein-ligand interfaces. J Med Chem. 2006, 49: 2713-2724. 10.1021/jm050260x.View ArticleGoogle Scholar
- Hewitt M, Cronin MT, Madden JC, Rowe PH, Johnson C, Obi A, Enoch SJ: Consensus QSAR models: do the benefits outweigh the complexity?. J Chem Inf Model. 2007, 47: 1460-1468. 10.1021/ci700016d.View ArticleGoogle Scholar
- Zhu H, Tropsha A, Fourches D, Varnek A, Papa E, Gramatica P, Oberg T, Dao P, Cherkasov A, Tetko IV: Combinatorial QSAR modeling of chemical toxicants tested against Tetrahymena pyriformis. J Chem Inf Model. 2008, 48: 766-784. 10.1021/ci700443v.View ArticleGoogle Scholar
- Li J, Lei B, Liu H, Li S, Yao X, Liu M, Gramatica P: QSAR study of malonyl-CoA decarboxylase inhibitors using GA-MLR and a new strategy of consensus modeling. J Comput Chem. 2008, 29: 2636-2647. 10.1002/jcc.21002.View ArticleGoogle Scholar
- Netzeva TI, Worth A, Aldenberg T, Benigni R, Cronin MT, Gramatica P, Jaworska JS, Kahn S, Klopman G, Marchant CA, Myatt G, Nikolova-Jeliazkova N, Patlewicz GY, Perkins R, Roberts D, Schultz T, Stanton DW, van de Sandt JJ, Tong W, Veith G, Yang C: Current status of methods for defining the applicability domain of (quantitative) structure-activity relationships. The report and recommendations of ECVAM Workshop 52. Altern Lab Anim. 2005, 33: 155-173.Google Scholar
- Jaworska J, Nikolova-Jeliazkova N, Aldenberg T: QSAR applicabilty domain estimation by projection of the training set descriptor space: a review. Altern Lab Anim. 2005, 33: 445-459.Google Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.