Condorcet and borda count fusion method for ligand-based virtual screening
© Ahmed et al.; licensee Chemistry Central Ltd. 2014
Received: 9 January 2014
Accepted: 23 April 2014
Published: 3 May 2014
It is known that any individual similarity measure will not always give the best recall of active molecule structure for all types of activity classes. Recently, the effectiveness of ligand-based virtual screening approaches can be enhanced by using data fusion. Data fusion can be implemented using two different approaches: group fusion and similarity fusion. Similarity fusion involves searching using multiple similarity measures. The similarity scores, or ranking, for each similarity measure are combined to obtain the final ranking of the compounds in the database.
The Condorcet fusion method was examined. This approach combines the outputs of similarity searches from eleven association and distance similarity coefficients, and then the winner measure for each class of molecules, based on Condorcet fusion, was chosen to be the best method of searching. The recall of retrieved active molecules at top 5% and significant test are used to evaluate our proposed method. The MDL drug data report (MDDR), maximum unbiased validation (MUV) and Directory of Useful Decoys (DUD) data sets were used for experiments and were represented by 2D fingerprints.
Simulated virtual screening experiments with the standard two data sets show that the use of Condorcet fusion provides a very simple way of improving the ligand-based virtual screening, especially when the active molecules being sought have a lowest degree of structural heterogeneity. However, the effectiveness of the Condorcet fusion was increased slightly when structural sets of high diversity activities were being sought.
Virtual screening refers to the use of a computer-based method to process compounds from a library or database of compounds in order to identify and select the ones that are likely to possess a desired biological activity, such as the ability to inhibit the action of a particular therapeutic target. The selection of molecules with a virtual screening algorithm should yield a higher proportion of active compounds, as assessed by experiment, relative to a random selection of the same number of molecules [1, 2].
Many virtual screening (VS) approaches have been implemented for searching chemical databases, such as substructure search, similarity, docking and QSAR. Of these, similarity searching is the simplest, and one of the most widely-used techniques, for ligand-based virtual screening (LBVS) . Similarity search aims to search and scan a chemical database to identify those molecules that are most similar to a user-defined reference structure using some quantitative measures of intermolecular structural similarity [4–8].
There are many different ways to implement the similarity searching based on different similarity models. However, as Sheridan and Kearsley  noted, it is most unlikely that a single search mechanism could be expected to perform at a consistently high level under all circumstances. Instead, a more realistic approach to enhancing the effectiveness of ligand-based virtual screening approaches is the use of data fusion  or consensus scoring in the structure-based virtual screening literature . Data fusion was first used for similarity searching in the late-Nineties [12–14]. Recently, data fusion has been used to combine the results of the structure and ligand-based approaches to virtual screening , their results outperforming any single method in ranking of activities. The latest reviews on using fusion in ligand-based virtual screening can be found in [16, 17].
There are two main approaches to data fusion: similarity fusion and group fusion [10, 18]. The first type combines ranking from single searches based on multiple similarity measures, while the second one combines ranking from multiple searches based on a single similarity measure. The basic procedure that has been developed for the fusion process is shown in algorithmic form as described below:
The basic procedure for data fusion:
for x = 1:n
for y = 1:N
Use x-th similarity or scoring measure to calculate similarity or score, Simx(qy) for y-th database-structure.
Use the fusion rule to combine the set of n score Simx(qy) for y-th database-structure to give its fused score FSimy,
Sort the database into decreasing order of fused score FSimy.
In this algorithm, there are n different similarity measures for calculating the similarity SIMx(dy) for each of the N structures in the database that is being searched (1 ≤ x ≤ n, and 1 ≤ y ≤ N).
The idea of voting algorithms emerged in the 18th century to address the shortcomings of simple majority voting when there are more than two candidates. According to Montague and Aslam  and Riker , there are two main voting algorithms: majoritarian and positional voting algorithms. Majoritarian voting algorithms are based on a series of pairwise comparisons of candidates, while positional algorithms are based on the ranking a candidate receives.
In this paper, the authors examined the use of Condorcet fusion in order to improve the effectiveness of ligand virtual screening by enhancing the recall of active compound structure. In our proposed model, for each similarity measure the top retrieved structures represent the voters; each candidate’s similarity measures received a number of points or votes depends on the similarity values of the retrieved structures. At last, Borda’s count method evaluated by summation of these points to find the winner candidate’s measure. The winner candidate got the highest number of points.
This study has compared the retrieval results obtained using two different similarity-based screening models. The first screening system was based on the Tanimoto (TAN) coefficient, which has been used in ligand-based virtual screening for many years and is now considered a reference standard. The second model, the proposed model of this study, was based on the Condorcet model proposed by Montague and Aslam . In our approach, the two groups of similarity measures were used, the first group is seven of the association coefficients: Jaccard/Tanimoto, Ochiai/Cosine, Sokal/Sneath(1), Kulczynski(2), Forbes, Fossum and Simpson; the second group is four of the distance coefficients: Mean Euclidean, Mean Canberra, Divergence and Bray/Curtis. The results from the two groups were used together and the Condocert fusion based on combining ranking from single searches for each of the eleven similarity measures is achieved. More details about the above similarity measures or metrics found in the early study proposed by Ellis et al. .
Tanimoto-based similarity model
For molecules described by continuous variables, the molecular space is defined by an M × N matrix, where entry w ji is the value of the j th fragments (1 ≤ j ≤ M) in the i th molecule (1 ≤ i ≤ N). The origins of this coefficient can be found in .
Condorcet-based fusion model
In this study we start our search using single reference structure and then the retrieved results based on different values of n represent the input of this process, which we will call a voting profile. Depending on the numbers of points, a social choice function based on Borda count that uses the positional voting procedure and Condorcet voting algorithm that uses majoritarian method will map voting profiles to a set of candidates — the winners.
The Borda count is perhaps the most sensible positional voting procedure. In the Borda count implemented here, for each voter, each candidate receives n points (n is the number of points in the retrieved structures in top-n results). The pairwise comparisons of candidates, based on the Condorcet voting algorithm that uses the majoritarian method, select the winner similarity method with the most points received. This process is repeated for each activity class.
In this method, eleven similarity measures and four different values of top retrieved structures were examined. The retrieved structures in each top retrieved represent the voter population to elect the winner similarity measures based on the Borda count method of points achieved by each candidate measure. The Condorcet-based fusion algorithm is described as follows:
for z = 1 top-n % n is number of activity classes in the data set
get the top-n ranking score for the each similarity measure
for x = 1 to m do % m is number of similarity measures
Assign value to each similarity measure equal to the a number of votes or points in retrieved topn structures in the results
- 5.find out the total Borda score for each similarity measure,
% Bi is the number of points for this activity class use the x-th similarity measure in topn for y-th database structure
Select the winner similarity measure (Fsimx) using pairwise comparisons based on Condorcet voting algorithm that used majoritarian method
The complexity of the algorithm is calculated and it processes in a worst time of O(n(2 m + top-n)). This time was calculated based on the following: (i) the outer loop (line 1) is based on the number of activity classes in the data set; thus, the maximum number of iterations is n, (ii) the first inner loop (line 3) is also based on the number of similarity measures; the maximum number of iterations is m, (iii) the second inner loop (line 5) is based on the Borda score for each similarity measure; the maximum number of iterations is top-n. Finally, for the final inner loop (line 6), on Condorcet voting algorithm, the maximum number of iterations is (m).
The searches were carried out using the most popular chemoinformatics databases, the MDL Drug Data Report (MDDR) , maximum unbiased validation (MUV)  and Directory of Useful Decoys (DUD) . All the molecules in both databases were converted to Pipeline Pilot ECFC_4 (extended connectivity fingerprints and folded to size 1024 bits) ; MDDR and MUV data sets have been used recently by our research group in this research area [26–29]. Mathworks Matlab R2012b (UTM license) was used for coding our proposed algorithms; all calculations were run on 2.80 GHz Intel(R) Xeon(R) processors.
explanation example on electing winner measure based achieved votes or points
Votes or points
MDDR1 structure activity classes
Adenosine (A1) agonists
Adenosine (A2) agonists
Renin inhibitors 1
Vitamin D analogous
MDDR2 structure activity classes
Muscarinic (M1) agonists
NMDA receptor antagonists
Nitric oxide synthase inhibitors
Aldose reductase inhibitors
Reverse transcriptase inhibitors
Phospholipase A2 inhibitors
MUV structure activity classes
S1P1 rec. (agonists)
HIV RT-RNase (inhibitors)
Eph rec. A4 (inhibitors)
HSP 90 (inhibitors) 30
ER-a-Coact. Bind. (inhibitors)
ER-b-Coact. Bind. (inhibitors)
ER-a-Coact. Bind. (potentiators)
Cathepsin G (inhibitors)
D1 rec. (allosteric modulators)
M1 rec. (allosteric inhibitors)
Number of active and inactive compounds for twelve DUD sub datasets, where N a : number of active compounds, N dec : number of decoys
Active and inactive compounds
Searches were carried out using single reference structures and a total of eleven similarity measures. Different numbers of top retrieved or nearest neighbours—10, 20, 50, and 100—were selected (as voter committee or population) for each activity class and used as input to the fusion stage to determine the winner candidate similarity measure. Finally, a search was carried out again based on the winner or fused similarity measure.
Results and discussion
Retrieval results of top 5% for data set MDDR1
Star (*) cells
Retrieval results of top 5% for data set MDDR2
Star (*) cells
Retrieval results of top 5% for data set MUV
Star (*) cells
Retrieval results of top 5% for data set DUD
Star (*) cells
A look at the recall values in Tables 6, 7, 8 and 9 enables comparisons to be made between the effectiveness of the various search models. However, a more quantitative approach is possible using the Kendall W test of concordance . This test shows whether a set of judges make comparable judgments about the ranking of a set of objects. Here, the activity classes were considered the judges and the recall rates of the various search models, the objects.
The outputs of this test are the value of the Kendall coefficient and the associated significance level, which indicates whether the value of the coefficient could have occurred by chance. If the value is significant (for which we used cutoff values of 0.05), then it is possible to give an overall ranking of the objects that have been ranked.
Rankings of similarity approaches based on Kendall W test results: MDDR1, MDDR2, MUV and DUD top 5%
Top100 > Top50 > Top20 > Top10 > TAN
Top100 > Top50 > Top20 > Top10 > TAN
Top100 > Top50 > Top20 > Top10 > TAN
Top100 > Top50 > Top20 > Top10 > TAN
where ti is the number of tied ranks in the ith group of tied ranks and gj is the number of groups of ties in the set of ranks (ranging from 1 to n) for judge j. Thus, Tj is the correction factor required for the set of ranks for judge j, i.e. the jth set of ranks.
Some of the activity classes, such as low-diversity activity classes, may contribute disproportionately to the overall value of mean recall. Therefore, using the mean recall value as the evaluation criterion could be impartial in some methods, but not in others. To avoid this bias, the effective performances of the different methods have been further investigated based on the total number of (*) cells for each method across the full set of activity classes. This is shown in the bottom rows of Tables 6, 7, 8 and 9. According to the total number of (*) cells in these tables, Condorcet fusion at Top100 was the best performing search across the three data sets.
The results of the MDDR1 search shown in Table 6 show that Condorcet fusion at Top100 produced the highest mean value compared with other measures. The value of the Kendall coefficient is 0.594. Given that the result is significant, since associated probability is < 0.01, the overall ranking of the different approaches is Top100 > Top50 > Top20 > Top10 > TAN for the cut off 5%, which shows that the proposed method has a high rank value. Similarly, For MDDR2 data set, our proposed method has the highest rank at cut off 5%. On the other hand, the MDDR2 searches are of particular interest, since they involve the most heterogeneous activity classes in the three data sets used, and thus provide a complete test of the effectiveness of a screening method. Table 7 shows that Condorcet fusion at Top100 gives the best performance out of all the methods for this data set at cut off 5%.
While the MDDR1 dataset includes highly similar activities, the MUV and DUD datasets have been carefully designed to include sets of highly dissimilar actives. Most of the similarity methods as well as our proposed method here show a very high recall rate for the low diversity dataset and very low recall for the high diversity datasets, such as MDDR2, MUV and DUD used in this study.
An ROC curve describes the trade-off between sensitivity and specificity, where the sensitivity is defined as the ability of the model to avoid false negatives, and the specificity relates to its ability to avoid false positives.
Rankings of similarity approaches based on friedman’s test results: MDDR1, MDDR2, MUV and DUD top 5%
Enrichment values of (BEDROC α = 20) and (EF 1%) using our proposed method on MDDR1, MDDR2, MUV and DUD data sets
BEDROC (α = 20)
BEDROC (α = 20)
BEDROC (α = 20)
BEDROC (α = 20)
Furthermore, our results were compared with recent similar studies such as rank- based group fusion by Chen et al.  and standard score (Z-score) by Sastry et al. . In Chen et al. study, the mean recall of their RKP method for MDDR1 data set range from 94.20 to 94.30, while in our method the minimum value of the upper band is 95.27 for Top10 and the maximum value is 99.95 for the Top100 method. Similarly, the best mean recall for the MDDR2 data set of our method is 50.63 for the activity index 31281 compared with 48.98 with their results. In addition, Sastry et al. used the top 1% of MDDR1 and the best mean recall for their method was 43.8 for the RXG combination method, when running our experiment and get 1%, the best mean recall of our method is 44.35 which slightly outperformed their findings.
In this study, we have developed a Condorcet fusion model to enhance the effectiveness of ligand-based virtual screening. The overall results of our proposed method show that the screening similarity search outperformed the Tanimoto which considered the conventional similarity methods. In addition, there was evidence to suggest that our proposed method, Condorcet fusion at Top100, was more effective for high diversity data sets.
Ahmed A: B. Comp. Eng. (Karary University, Sudan), M. Sc. Comp. Sc. (University of Khartoum, Sudan), Ph. D Comp. Sc. (UTM, Malaysia).NS: Professor Dr. B. Comp. Sc. (UTM, Malaysia), M. Sc. Comp. Sc. (W. Michigan, US) Ph. D Info. Sc. (Univ. of Sheffield, UK).
Dr. Ali Ahmed Alfakiabdalla Abdelrahim is Post-Doctoral researcher at the Research Management Centre (RMC) at Universiti Teknologi Malaysia under the Post-Doctoral Fellowship Scheme for the Project: “Enhancing Molecular Similarity Search”. This work is supported by (RMC) under Research University Grant Category (VOT Q.J130000.2528.07H89). We also would like to thank (RMC) for supporting the first author.
- Johnson MA, Maggiora GM: Concepts and Applications of Molecular Similarity. 1990, New York: Wiley New YorkGoogle Scholar
- Walters WP, Stahl MT, Murcko MA: Virtual screening–an overview. Drug Discov Today. 1998, 3 (4): 160-178. 10.1016/S1359-6446(97)01163-X.View ArticleGoogle Scholar
- Willett P, Barnard JM, Downs GM: Chemical similarity searching. J Chem Inf Comput Sci. 1998, 38 (6): 983-996. 10.1021/ci9800211.View ArticleGoogle Scholar
- Bender A, Glen RC: Molecular similarity: a key technique in molecular informatics. Org Biomol Chem. 2004, 2 (22): 3204-3218. 10.1039/b409813g.View ArticleGoogle Scholar
- Willett P: Similarity methods in chemoinformatics. Annual Rev Info Sci Technol. 2009, 43 (1): 1-117.View ArticleGoogle Scholar
- Geppert H, Vogt M, Bajorath JR: Current trends in ligand-based virtual screening: molecular representations, data mining methods, new application areas, and performance evaluation. J Chem Inf Model. 2010, 50 (2): 205-216. 10.1021/ci900419k.View ArticleGoogle Scholar
- Ripphausen P, Nisius B, Bajorath J: State-of-the-art in ligand-based virtual screening. Drug Discov Today. 2011, 16 (9): 372-376.View ArticleGoogle Scholar
- Willett P: Similarity-based data mining in files of two-dimensional chemical structures using fingerprint measures of molecular resemblance. Wiley Interdis Rev Data Mining Knowled Discov. 2011, 1 (3): 241-251. 10.1002/widm.26.View ArticleGoogle Scholar
- Sheridan RP, Kearsley SK: Why do we need so many chemical similarity search methods?. Drug Discov Today. 2002, 7 (17): 903-911. 10.1016/S1359-6446(02)02411-X.View ArticleGoogle Scholar
- Willett P: Enhancing the effectiveness of ligand-based virtual screening using data fusion. QSAR Comb Sci. 2006, 25 (12): 1143-1152. 10.1002/qsar.200610084.View ArticleGoogle Scholar
- Feher M: Consensus scoring for protein–ligand interactions. Drug Discov Today. 2006, 11 (9): 421-428.View ArticleGoogle Scholar
- Kearsley SK, Sallamack S, Fluder EM, Andose JD, Mosley RT, Sheridan RP: Chemical similarity using physiochemical property descriptors. J Chem Inf Comput Sci. 1996, 36 (1): 118-127. 10.1021/ci950274j.View ArticleGoogle Scholar
- Sheridan RP, Miller MD, Underwood DJ, Kearsley SK: Chemical similarity using geometric atom pair descriptors. J Chem Inf Comput Sci. 1996, 36 (1): 128-136. 10.1021/ci950275b.View ArticleGoogle Scholar
- Ginn CM, Willett P, Bradshaw J: Combination of molecular similarity measures using data fusion. Perspect Drug Discov Des. 2002, 20: 1-16.View ArticleGoogle Scholar
- Seal A, Yogeeswari P, Sriram D, Wild DJ: Enhanced ranking of PknB Inhibitors using data fusion methods. J Cheminfo. 2013, 5 (1): 1-11. 10.1186/1758-2946-5-1.View ArticleGoogle Scholar
- Willett P: Combination of similarity rankings using data fusion. J Chem Inf Model. 2013, 53 (1): 1-10. 10.1021/ci300547g.View ArticleGoogle Scholar
- Willett P: Fusing similarity rankings in ligand-based virtual screening. computational and structural. Comput Struct Biotechnol J. 2013, 5: e201302002-View ArticleGoogle Scholar
- Riniker S, Fechner N, Landrum G: Heterogeneous classifier fusion for ligand-based virtual screening: or, how decision making by committee can be a good thing. J Chem Inf Model. 2013, 53 (11): 2829-2836. 10.1021/ci400466r.View ArticleGoogle Scholar
- Montague M, Aslam JA: Condorcet Fusion for Improved Retrieval. Proceedings of the Eleventh International Conference on Information and Knowledge Management (CIKM’02). 2002, McLean, Virginia, USA: ACM, 538-548.Google Scholar
- Riker WH: Liberalism against Populism: a Confrontation between the Theory of Democracy and the Theory of Social Choice. 1982, San Francisco, CA: Freeman San FranciscoGoogle Scholar
- Ellis D, Furner-Hines J, Willett P: Measuring the degree of similarity between objects in text retrieval systems. Perspec Info Manag. 1993, 3 (2): 128-149.Google Scholar
- Symyx Technologies. MDL drug data report: Sci Tegic Accelrys Inc., the MDL Drug Data Report (MDDR). Database is available at http://www.accelrys.com/. (accessed 1st of April 2014)
- Rohrer SG, Baumann K: Maximum unbiased validation (MUV) data sets for virtual screening based on PubChem bioactivity data. J Chem Inf Model. 2009, 49 (2): 169-184. 10.1021/ci8002649.View ArticleGoogle Scholar
- Huang N, Shoichet BK, Irwin JJ: Benchmarking sets for molecular docking. J Med Chem. 2006, 49 (23): 6789-6801. 10.1021/jm0608356.View ArticleGoogle Scholar
- Pipeline Pilot Software: SciTegic Accelrys Inc. 2008, San Diego: Accelrys Inc, http://www.accelrys.com/,Google Scholar
- Abdo A, Salim N: New fragment weighting scheme for the Bayesian inference network in ligand-based virtual screening. J Chem Inf Model. 2011, 51: 25-32. 10.1021/ci100232h.View ArticleGoogle Scholar
- Abdo A, Saeed F, Hamza H, Ahmed A, Salim N: Ligand expansion in ligand-based virtual screening using relevance feedback. J Comput Aided Mol Des. 2012, 26 (3): 279-287. 10.1007/s10822-012-9543-4.View ArticleGoogle Scholar
- Saeed F, Salim N, Abdo A: Voting-based consensus clustering for combining multiple clusterings of chemical structures. J Cheminfo. 2012, 4 (1): 1-8. 10.1186/1758-2946-4-1.View ArticleGoogle Scholar
- Saeed F, Salim N, Abdo A: Graph-based consensus clustering for combining multiple clusterings of chemical structures. Mol Info. 2013, 32 (2): 165-178. 10.1002/minf.201200110.View ArticleGoogle Scholar
- Abdo A, Salim N: New fragment weighting scheme for the bayesian inference network in ligand-based virtual screening. J Chem Inf Model. 2010, 51 (1): 25-32.View ArticleGoogle Scholar
- Cross S, Baroni M, Carosati E, Benedetti P, Clementi S: FLAP: GRID molecular interaction fields in virtual screening. validation using the DUD data set. J Chem Inf Model. 2010, 50 (8): 1442-1450. 10.1021/ci100221g.View ArticleGoogle Scholar
- Repasky MP, Murphy RB, Banks JL, Greenwood JR, Tubert-Brohman I, Bhat S, Friesner RA: Docking performance of the glide program as evaluated on the astex and DUD datasets: a complete set of glide SP results and selected results for a new scoring function integrating WaterMap and glide. J Comput Aided Mol Des. 2012, 26 (6): 787-799. 10.1007/s10822-012-9575-9.View ArticleGoogle Scholar
- Siegel S: Nonparametric Statistics for the Behavioral Sciences. 1956, New York: McGraw-HillGoogle Scholar
- Friedman M: The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc. 1937, 32 (200): 675-701. 10.1080/01621459.1937.10503522.View ArticleGoogle Scholar
- Schlotzhauer SD: Elementary Statistics Using JMP. 2007, Cary, NC, USA: SAS Institute IncGoogle Scholar
- Swets JA: Measuring the accuracy of diagnostic systems. Science. 1988, 240 (4857): 1285-1293. 10.1126/science.3287615.View ArticleGoogle Scholar
- Triballeau N, Acher F, Brabet I, Pin J-P, Bertrand H-O: Virtual screening workflow development guided by the “receiver operating characteristic” curve approach. Application to high-throughput docking on metabotropic glutamate receptor subtype 4. J Med Chem. 2005, 48 (7): 2534-2547. 10.1021/jm049092j.View ArticleGoogle Scholar
- Swamidass SJ, Azencott CA, Daily K, Baldi P: A CROC stronger than ROC: measuring, visualizing and optimizing early retrieval. Bioinformatics. 2010, 26 (10): 1348-1356. 10.1093/bioinformatics/btq140.View ArticleGoogle Scholar
- Sastry GM, Inakollu VS, Sherman W: Boosting virtual screening enrichments with data fusion: coalescing hits from two-dimensional fingerprints, shape, and docking. J Chem Inf Model. 2013, 53 (7): 1531-1542. 10.1021/ci300463g.View ArticleGoogle Scholar
- Zhao W, Hevener KE, White SW, Lee RE, Boyett JM: A statistical framework to evaluate virtual screening. BMC Bioinformatics. 2009, 10 (1): 225-10.1186/1471-2105-10-225.View ArticleGoogle Scholar
- Truchon J-F, Bayly CI: Evaluating virtual screening methods: good and bad metrics for the “early recognition” problem. J Chem Inf Model. 2007, 47 (2): 488-508. 10.1021/ci600426e.View ArticleGoogle Scholar
- Chen B, Mueller C, Willett P: Combination rules for group fusion in similarity-based virtual screening. Mol Info. 2010, 29 (6–7): 533-541.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.