14.1.5 Classifier, Performance Evaluation, Errors, Comparisons

Chapter Contents (Back)
Evaluation, Classifiers. Comparisons.

Mattson, R.L., and Firschein, O.,
Feature Word Construction for Use with Pattern Recognition Algorithms: An Experimental Study,
JACM(10), No. 4, October 1963, pp. 458-477. BibRef 6310

Dressler, R.F., and Werner, W.,
Error Rates for Two Methods of Statistical Pattern Recognition,
JACM(11), No. 4, October 1964, pp. 471-480. BibRef 6410

Bledsoe, W.W.,
Some Results on Multicategory Pattern Recognition,
JACM(13), No. 2 April 1966, pp. 304-316. BibRef 6604

Hughes, G.F.,
On the Mean Accuracy of Statistical Pattern Recognizers,
IT(14), No. 1, January 1968, pp. 55-63. BibRef 6801

Chow, C.K.,
On Optimum Recognition Error and Reject Trade-Offs,
IT(16), No. 1, January 1970, pp. 41-46. Relationship between (False Rejection) FRR and (False Acceptance) FAR error formulations. See also Optimum Character Recognition System Using Decision Function, An. BibRef 7001

Rand, W.M.,
Objective Criteria for the Evaluation of Clustering Methods,
ASAJ(66), No. 336, 1971, pp. 846-850. BibRef 7100

Preston, K.,
A Comparison of Analog and Digital Techniques for Pattern Recognition,
PIEEE(60), No. 10, October 1972, pp. 1216-1231. BibRef 7210

Sacco, W.J.[William J.], Copes, W.S.[Wayne S.],
Reduction of the class of feature evaluation techniques in pattern analysis,
PR(4), No. 3, October 1972, pp. 331-332.
WWW Link. 0309
Letter. BibRef

Toussaint, G.T.[Godfried T.],
Bibliography on Estimation of Misclassification,
IT(20), 1974, pp. 472-479. Survey, Evaluation. Complete listing of the research up to this time. BibRef 7400

Slagle, J.R., Chang, C.L., Lee, C.T.,
Experiments with Some Cluster Analysis Algorithms,
PR(6), No. 3-4, December 1974, pp. 181-187.
WWW Link. 1-D clustering. BibRef 7412

Milligan, G.W.[Glenn W.], Isaac, P.D.[Paul D.],
The validation of four ultrametric clustering algorithms,
PR(12), No. 2, 1980, pp. 41-50.
WWW Link. 0309
BibRef

Bayne, C.K.[Charles K.], Beauchamp, J.J.[John J.], Begovich, C.L.[Connie L.], Kane, V.E.[Victor E.],
Monte Carlo comparisons of selected clustering procedures,
PR(12), No. 2, 1980, pp. 51-62.
WWW Link. 0309
BibRef

Viscolani, B.[Bruno],
Computational length in pattern recognizers,
PR(15), No. 5, 1982, pp. 413-418.
WWW Link. 0309
BibRef

Viscolani, B.[Bruno],
Optimization of computational time in pattern recognizers,
PR(15), No. 5, 1982, pp. 419-424.
WWW Link. 0309
A sequential organization of the computations arising from pattern recognizers by absolute comparison is suggested in order to reduce the mean computational time involved. BibRef

Viscolani, B.[Bruno],
Further results on optimization of recognition time,
PR(16), No. 3, 1983, pp. 337-339.
WWW Link. 0309
BibRef

Hanley, J., McNeil, B.,
The Meaning and Use of the Area under a Receiver Operating Characteristic (ROC) Curve,
Radiology(143), 1982, pp. 29-36. BibRef 8200

Goin, J.E.[James E.], Fritz, S.L.[Steven L.],
A Matrix Approach to Data Base Exploration: Analysis of Classifier Results,
PR(16), No. 2, 1983, pp. 243-252.
WWW Link. 0309
BibRef

Raveh, A.[Adi],
Preference structure analysis: A nonmetric approach,
PR(16), No. 2, 1983, pp. 253-259.
WWW Link. 0309
Suggest a nonmetric procedure in which goodness of discrimination is higher than or equal to that of Fisher's discriminant function. very different discriminant functions could yield the very same number of errors. BibRef

Fowlkes, E.B., and Mallows, C.L.,
A Method for Comparing Two Hierarchical Clusterings,
ADAJ(78), No. 383, 1983, pp. 553-569. BibRef 8300

Flick, T.E., and Jones, L.K.,
A Combinatorial Approach for Classification of Patterns with Missing Information and Random Orientation,
PAMI(8), No. 4, July 1986, pp. 482-490. BibRef 8607

Flick, T.E.[Thomas E.], Jones, L.K.[Lee K.], Priest, R.G.[Richard G.], Herman, C.[Charles],
Pattern classification using projection pursuit,
PR(23), No. 12, 1990, pp. 1367-1376.
WWW Link. 0401
BibRef

Wacker, A.G., El-Sheikh, T.S.,
Average Classification Accuracy over Collections of Gaussian Problems: Common Covariance Matrix Case,
PR(17), No. 2, 1984, pp. 259-273.
WWW Link. 0309
BibRef

Chernick, M.C., Murthy, V.K., and Nealy, C.D.,
Application of Bootstrap and Other Resampling Techniques: Evaluation of Classifier Performance,
PRL(3), 1985, pp. 167-178. BibRef 8500

Jain, A.K., Dubes, R.C., and Chen, C.C.,
Bootstrap Techniques for Error Estimation,
PAMI(9), No. 5, September 1987, pp. 628-633. BibRef 8709

Colussi, L.,
Correctness and Efficiency of Pattern Matching Algorithms,
InfoControl(95), 1991, pp. 225-251. BibRef 9100

Srivastava, A.[Anurag], Murty, M.N.,
A comparison between conceptual clustering and conventional clustering,
PR(23), No. 9, 1990, pp. 975-981.
WWW Link. 0401
BibRef

Tang, Y.Y., Qu, Y.Z., Suen, C.Y.,
Multiple-level information source and entropy-reduced transformation models,
PR(24), No. 4, 1991, pp. 341-357.
WWW Link. 0401
Analyze systematically the changes in entropy which occur in the different stages of a pattern recognizer. BibRef

Gluhchev, G., Shalev, S.,
The Systematic-Error Detection as a Classification Problem,
PRL(17), No. 12, October 25 1996, pp. 1233-1238. 9612
BibRef

Duin, R.P.W.,
A Note on Comparing Classifiers,
PRL(17), No. 5, May 1 1996, pp. 529-536. 9606
BibRef

Richards, J.A.,
Classifier Performance and MAP Accuracy,
RSE(57), No. 3, September 1996, pp. 161-166. 9609
BibRef

Nyssen, E.,
Evaluation of Pattern Classifiers: Testing the Significance of Classification Efficiency Using an Exact Probability Technique,
PRL(17), No. 11, September 16 1996, pp. 1125-1129. 9611
BibRef

Nyssen, E.,
Evaluation of Pattern Classifiers: Applying a Monte Carlo Significance Test to the Classification Efficiency,
PRL(19), No. 1, January 1998, pp. 1-6. 9807
BibRef

Barbosa, P.M., Casterad, M.A., Herrero, J.,
Performance of Several Landsat-5 Thematic Mapper (TM) Image Classification Methods for Crop Extent Estimates in an Irrigation District,
JRS(17), No. 18, December 1996, pp. 3665-3674. 9701
BibRef

SanMiguel-Ayanz, J., Biging, G.S.,
Comparison of Single-Stage and Multistage Classification Approaches for Cover Type Mapping with TM and Spot Data,
RSE(59), No. 1, January 1997, pp. 92-104. 9701
BibRef

Denoeux, T.[Thierry],
Analysis of Evidence Theoretic Decision Rules for Pattern-Classification,
PR(30), No. 7, July 1997, pp. 1095-1107.
WWW Link. 9707
BibRef

Bokka, V., Olariu, S., Schwing, J.L., Wilson, L., Zomaya, A.,
A Time-Optimal Solution to a Classification Problem in Ordered Functional Domains, with Applications,
PR(30), No. 9, September 1997, pp. 1555-1564.
WWW Link. 9708
BibRef

Zhou, J.Y.[Jiang-Ying], Lopresti, D.P.[Daniel P.],
Improving Classifier Performance Through Repeated Sampling,
PR(30), No. 10, October 1997, pp. 1637-1650.
WWW Link. 9712
BibRef

Lerner, B.[Boaz], Guterman, H.[Hugo], Aladjem, M.[Mayer], Dinsteint, I.[Its'hak], Romem, Y.[Yitzhak],
On Pattern Classification with Sammons Nonlinear Mapping: An Experimental-Study,
PR(31), No. 4, April 1998, pp. 371-381.
WWW Link. 9803
See also nonlinear mapping for data structure analysis, A. BibRef

Aladjem, M.[Mayer], Dinstein, I.[Its'hak],
A multiclass extension of discriminant mappings,
ICPR92(II:101-104).
IEEE DOI 9208
BibRef

Stehman, S.V., Czaplewski, R.L.,
Design and Analysis for Thematic Map Accuracy Assessment: Fundamental Principles,
RSE(64), No. 3, June 1998, pp. 331-344. 9806
BibRef

Adams, N.M., Hand, D.J.,
Comparing classifiers when the misallocation costs are uncertain,
PR(32), No. 7, July 1999, pp. 1139-1147.
WWW Link. BibRef 9907

Smits, P.C., Dellepiane, S.G., Schowengerdt, R.A.,
Quality assessment of image classification algorithms for land-cover mapping: a review and a proposal for a cost-based approach,
JRS(20), No. 8, May 1999, pp. 1461. BibRef 9905

Sohn, S.Y.[So Young],
Meta Analysis of Classification Algorithms for Pattern Recognition,
PAMI(21), No. 11, November 1999, pp. 1137-1144.
IEEE DOI 9912
For sample size and dimensionality. Meta model to compare different classification algorithms. Traditional statistical, neural nets, and machine learning approaches. BibRef

Srivastava, A.N., Su, R., Weigend, A.S.,
Data Mining for Features Using Scale-Sensitive Gated Experts,
PAMI(21), No. 12, December 1999, pp. 1268-1279.
IEEE DOI 0001
Data analysis to partition complex regression surface into simpler surfaces (features). See also Virtual Sensors: Using Data Mining Techniques to Efficiently Estimate Remote Sensing Spectra. BibRef

Andersson, A.[Arne], Davidsson, P.[Paul], Lindén, J.[Johan],
Measure-based classifier performance evaluation,
PRL(20), No. 11-13, November 1999, pp. 1165-1173. 0001
BibRef

Lim, T.S., Loh, W.Y., Shil, Y.S.,
A Comparison of Prediction Accuracy, Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms,
MachLearn(40), No. 3, 2000, pp. 203-228. 0003
BibRef

Ong, S.H., Zhao, X.,
On post-clustering evaluation and modification,
PRL(21), No. 5, May 2000, pp. 365-373. 0005
BibRef

Raudys, S.J.[Sarunas J.], Saudargiene, A.[Ausra],
First-Order Tree-Type Dependence between Variables and Classification Performance,
PAMI(23), No. 2, February 2001, pp. 233-239.
IEEE DOI 0102
BibRef

Hubert-Moy, L., Cotonnec, A., Le Du, L., Chardin, A., Perez, P.,
A Comparison of Parametric Classification Procedures of Remotely Sensed Data Applied on Different Landscape Units,
RSE(75), No. 2, 2001, pp. 174-187. 0102
BibRef

Tambouratzis, G.[George],
Improving the Clustering Performance of the Scanning n-Tuple Method by Using Self-Supervised Algorithms to Introduce Subclasses,
PAMI(24), No. 6, June 2002, pp. 722-733.
IEEE DOI 0206
BibRef
Earlier:
Improving the Classification Accuracy of the Scanning N-tuple Method,
ICPR00(Vol II: 1046-1049).
IEEE DOI 0009
Extend work of: See also Statistical Syntactic Methods for High-Performance OCR. Remove edge effects. BibRef

Liu, M.Q.[Ming-Qin], Samal, A.[Ashok],
Cluster validation using legacy delineations,
IVC(20), No. 7, May 2002, pp. 459-467.
WWW Link. 0206
BibRef

Berikov, V.B.[Vladimir B.],
An approach to the evaluation of the performance of a discrete classifier,
PRL(23), No. 1-3, January 2002, pp. 227-233.
Elsevier DOI 0201
BibRef

Harvey, N.R., Theiler, J., Brumby, S.P., Perkins, S., Szymanski, J.J., Bloch, J.J., Porter, R.B., Galassi, M., Young, A.C.,
Comparison of GENIE and conventional supervised classifiers for multispectral image feature extraction,
GeoRS(40), No. 2, February 2002, pp. 393-404.
IEEE Top Reference. 0205
BibRef

Muchoney, D.M.[Douglas M.], Strahler, A.H.[Alan H.],
Pixel- and site-based calibration and validation methods for evaluating supervised classification of remotely sensed data,
RSE(81), No. 2-3, August 2002, pp. 290-299.
HTML Version. 0206
BibRef

Alsing, S.G.[Stephen G.], Bauer, Jr., K.W.[Kenneth W.], Miller, J.O.[John O.],
A multinomial selection procedure for evaluating pattern recognition algorithms,
PR(35), No. 11, November 2002, pp. 2397-2412.
WWW Link. 0208
BibRef

Maulik, U.[Ujjwal], Bandyopadhyay, S.[Sanghamitra],
Performance Evaluation of Some Clustering Algorithms and Validity Indices,
PAMI(24), No. 12, December 2002, pp. 1650-1654.
IEEE Abstract. 0212
Hard K-Means, Single Linkage, Simulated annealing See also Optimization by Simulated Annealing. BibRef

Sandri, L.[Laura], Marzocchi, W.[Warner],
Testing the performance of some nonparametric pattern recognition algorithms in realistic cases,
PR(37), No. 3, March 2004, pp. 447-461.
WWW Link. 0401
BibRef

Toh, K.A.[Kar-Ann], Tran, Q.L.[Quoc-Long], Srinivasan, D.[Dipti],
Benchmarking a Reduced Multivariate Polynomial Pattern Classifier,
PAMI(26), No. 6, June 2004, pp. 740-755.
IEEE Abstract. 0404
The simplified model worked well. Analyze it. BibRef

Tran, Q.L.[Quoc-Long], Toh, K.A.[Kar-Ann], Srinivasan, D.[Dipti], Wong, K.L., Low, S.Q.C.[Shaun Qiu-Cen],
An empirical comparison of nine pattern classifiers,
SMC-B(35), No. 5, October 2005, pp. 1079-1091.
IEEE DOI 0510
Algorithm RM. Reduced Multivariate. BibRef

Attoor, S.N.[Sanju N.], Dougherty, E.R.[Edward R.],
Classifier performance as a function of distributional complexity,
PR(37), No. 8, August 2004, pp. 1641-1651.
WWW Link. 0407
BibRef

Kim, D.W.[Dae-Won], Lee, K.Y.[Ki Young], Lee, D.[Doheon], Lee, K.H.[Kwang H.],
Evaluation of the performance of clustering algorithms in kernel-induced feature space,
PR(38), No. 4, April 2005, pp. 607-611.
WWW Link. 0501
BibRef

Kim, D.W.[Dae-Won], Lee, K.Y.[Ki-Young], Lee, D.[Doheon], Lee, K.H.[Kwang H.],
A k-populations algorithm for clustering categorical data,
PR(38), No. 7, July 2005, pp. 1131-1134.
WWW Link. 0505
BibRef

Stein, A., Aryal, J., Gort, G.,
Use of the Bradley-Terry Model to Quantify Association in Remotely Sensed Images,
GeoRS(43), No. 4, April 2005, pp. 852-856.
IEEE Abstract. 0501
BibRef

Caulfield, H.J.[H. John], Heidary, K.[Kaveh],
Exploring margin setting for good generalization in multiple class discrimination,
PR(38), No. 8, August 2005, pp. 1225-1238.
WWW Link. 0505
BibRef

Salman, A.[Ayed], Omran, M.G.[Mahamed G.], Engelbrecht, A.P.[Andries P.],
SIGT: Synthetic Image Generation Tool for Clustering Algorithms,
GVIP(05), No. V2, January 2005, pp. 33-44
HTML Version. Create images to test clustering. BibRef 0501

Graaff, A.J., Engelbrecht, A.P.[Andries P.],
Clustering data in an uncertain environment using an artificial immune system,
PRL(32), No. 2, 15 January 2011, pp. 342-351.
Elsevier DOI 1101
Uncertain environments; Non-stationary data; Immune networks; Clustering performance measures BibRef

Yousef, W.A.[Waleed A.], Wagner, R.F.[Robert F.], Loew, M.H.[Murray H.],
Estimating the uncertainty in the estimated mean area under the ROC curve of a classifier,
PRL(26), No. 16, December 2005, pp. 2600-2610.
WWW Link. 0512
BibRef

Yousef, W.A.[Waleed A.], Wagner, R.F.[Robert F.], Loew, M.H.[Murray H.],
Assessing Classifiers from Two Independent Data Sets Using ROC Analysis: A Nonparametric Approach,
PAMI(28), No. 11, November 2006, pp. 1809-1817.
IEEE DOI 0609
BibRef
Earlier:
Comparison of non-parametric methods for assessing classifier performance in terms of ROC parameters,
AIPR04(190-195).
IEEE DOI 0410
3 Parameters: Conditional (an particular training set) AUC (area under RO Curve), mean and variance of AUC. BibRef

Baraldi, A., Bruzzone, L., Blonda, P., Carlin, L.,
Badly Posed Classification of Remotely Sensed Images: An Experimental Comparison of Existing Data Labeling Systems,
GeoRS(44), No. 1, January 2006, pp. 214-235.
IEEE DOI 0601
BibRef

Baraldi, A., Bruzzone, L., Blonda, P.,
A Multiscale Expectation-Maximization Semisupervised Classifier Suitable for Badly Posed Image Classification,
IP(15), No. 8, August 2006, pp. 2208-2225.
IEEE DOI 0606
BibRef

Fawcett, T.[Tom],
An introduction to ROC analysis,
PRL(27), No. 8, June 2006, pp. 861-874.
WWW Link. 0605
Survey, ROC Analysis. Classifier evaluation; Evaluation metrics BibRef

Stathakis, D., Vasilakos, A.,
Comparison of Computational Intelligence Based Classification Techniques for Remotely Sensed Optical Image Classification,
GeoRS(44), No. 8, August 2006, pp. 2305-2318.
IEEE DOI 0608
BibRef

Stathakis, D.[Demetris], Kanellopoulos, I.[Ioannis],
Global Elevation Ancillary Data for Land-use Classification Using Granular Neural Networks,
PhEngRS(74), No. 1, January 2008, pp. 55-64.
WWW Link. 0803
Initial guidelines for the construction of granular neural networks in the remote sensing context. BibRef

Stathakis, D.[Demetris], Kanellopoulos, I.[Ioannis],
Global Optimization versus Deterministic Pruning for the Classification of Remotely Sensed Imagery,
PhEngRS(74), No. 10, October 2008, pp. 1259-1266.
WWW Link. 0804
A method for optimal Multi Layer Perceptron topology determination carried out by a genetic algorithm. BibRef

Arbel, R.[Reuven], Rokach, L.[Lior],
Classifier evaluation under limited resources,
PRL(27), No. 14, 15 October 2006, pp. 1619-1631.
WWW Link. 0609
Evaluation measures; Hit-rate; Recall; Receiver operating characteristic BibRef

Nangendo, G.[Grace], Skidmore, A.K.[Andrew K.], van Oosten, H.[Henk],
Mapping East African tropical forests and woodlands: A comparison of classifiers,
PandRS(61), No. 6, February 2007, pp. 393-404.
WWW Link. 0703
Forest classification; Conventional classifiers; Expert System; Classification accuracy; East Africa BibRef

An, S.J.[Sen-Jian], Liu, W.Q.[Wan-Quan], Venkatesh, S.[Svetha],
Fast cross-validation algorithms for least squares support vector machine and kernel ridge regression,
PR(40), No. 8, August 2007, pp. 2154-2162.
WWW Link. 0704
BibRef
Earlier:
Efficient Cross-validation of the Complete Two Stages in KFD Classifier Formulation,
ICPR06(III: 240-244).
IEEE DOI 0609
Model selection; Cross-validation; Kernel methods BibRef

An, S.[Senjian], Peursum, P.[Patrick], Liu, W.Q.[Wan-Quan], Venkatesh, S.[Svetha],
Efficient subwindow search with submodular score functions,
CVPR11(1409-1416).
IEEE DOI 1106
BibRef
Earlier:
Efficient algorithms for subwindow search in object detection and localization,
CVPR09(264-271).
IEEE DOI 0906
BibRef

An, S.[Senjian], Peursum, P.[Patrick], Liu, W.Q.[Wan-Quan], Venkatesh, S.[Svetha], Chen, X.M.[Xiao-Ming],
Exploiting Monge structures in optimum subwindow search,
CVPR10(926-933).
IEEE DOI 1006
BibRef

Pham, D.S.[Duc-Son], Venkatesh, S.[Svetha],
Robust learning of discriminative projection for multicategory classification on the Stiefel manifold,
CVPR08(1-7).
IEEE DOI 0806
BibRef

Pham, D.S.[Duc-Son], Venkatesh, S.[Svetha],
Joint learning and dictionary construction for pattern recognition,
CVPR08(1-8).
IEEE DOI 0806
BibRef

Bøcher, P.K., McCloy, K.R.,
Optimizing Image Resolution to Maximize the Accuracy of Hard Classification,
PhEngRS(73), No. 8, August 2007, pp. 893-904.
WWW Link. 0709
The relationship between classification accuracy and within class variances is investigated showing that within class variances are a function of image resolution. BibRef

Devarakota, P.R.R.[Pandu Ranga Rao], Mirbach, B.[Bruno], Ottersten, B.[Bjorn],
Reliability estimation of a statistical classifier,
PRL(29), No. 3, 1 February 2008, pp. 243-253.
WWW Link. 0801
Pattern classification; Local density estimation; Confidence intervals; Binomial distribution; GMMs; Pattern rejection BibRef

Sahiner, B., Chan, H.P., Hadjiiski, L.M.,
Performance Analysis of Three-Class Classifiers: Properties of a 3-D ROC Surface and the Normalized Volume Under the Surface for the Ideal Observer,
MedImg(27), No. 2, February 2008, pp. 215-227.
IEEE DOI 0802
BibRef

Volkovich, Z., Barzily, Z., Morozensky, L.,
A statistical model of cluster stability,
PR(41), No. 7, July 2008, pp. 2174-2188.
WWW Link. 0804
Cluster validation; Negative definite functions; Statistical model BibRef

Akhbardeh, A.[Alireza], Nikhil, Koskinen, P.E.[Perttu E.], Yli-Harja, O.[Olli],
Towards the experimental evaluation of novel supervised fuzzy adaptive resonance theory for pattern classification,
PRL(29), No. 8, 1 June 2008, pp. 1082-1093.
WWW Link. 0804
Affine look-up table; Classification; Pre-classification; Post-classification; Supervised fuzzy adaptive resonance theory (SF-ART) network Iris recognition. BibRef

Ferri, C., Hernandez-Orallo, J., Modroiu, R.,
An experimental comparison of performance measures for classification,
PRL(30), No. 1, 1 January 2009, pp. 27-38.
WWW Link. 0811
Classification; Performance measures; Ranking; Calibration BibRef

Lago-Fernandez, L.F.[Luis F.], Corbacho, F.[Fernando],
Normality-based validation for crisp clustering,
PR(43), No. 3, March 2010, pp. 782-795.
Elsevier DOI 1001
Crisp clustering; Cluster validation; Negentropy BibRef

Chen, J.[Jin], Zhu, X.L.[Xiao-Lin], Imura, H.[Hidefumi], Chen, X.H.[Xue-Hong],
Consistency of accuracy assessment indices for soft classification: Simulation analysis,
PandRS(65), No. 2, March 2010, pp. 156-164.
Elsevier DOI 1003
Soft classification; Accuracy assessment; Sub-pixel confusion matrix; RMSE; Consistency BibRef

Pascual, D.[Damaris], Pla, F.[Filiberto], Salvador Sánchez, J.,
Cluster validation using information stability measures,
PRL(31), No. 6, 15 April 2010, pp. 454-461.
Elsevier DOI 1004
BibRef
Earlier:
Cluster Stability Assessment Based on Theoretic Information Measures,
CIARP08(219-226).
Springer DOI 0809
Cluster validation; Stability index; Information theory BibRef

Valverde-Albacete, F.J.[Francisco J.], Pelaez-Moreno, C.[Carmen],
Two information-theoretic tools to assess the performance of multi-class classifiers,
PRL(31), No. 12, 1 September 2010, pp. 1665-1671.
Elsevier DOI 1008
Multi-class classifier; Confusion matrix; Contingency table; Performance measure; de Finetti diagram; Entropy triangle BibRef

Lee, Y.R.[Young-Rok], Lee, J.H.[Jeong-Hwa], Jun, C.H.[Chi-Hyuck],
Stability-based validation of bicluster solutions,
PR(44), No. 2, February 2011, pp. 252-264.
Elsevier DOI 1011
Biclustering; Validation; Stability; Resampling BibRef

Fu, X.[Xin], Shen, Q.A.[Qi-Ang],
Fuzzy complex numbers and their application for classifiers performance evaluation,
PR(44), No. 7, July 2011, pp. 1403-1417.
Elsevier DOI 1103
Fuzzy complex numbers; Performance evaluation; Feature selection; Pattern classification BibRef

Wicker, N.[Nicolas],
A note on ball segment picking related to clustering,
PRL(32), No. 5, 1 April 2011, pp. 651-655.
Elsevier DOI 1103
Density of points clustering; DPC; Ball segment picking; Curse of dimensionality; Sampling BibRef

Woloszynski, T.[Tomasz], Kurzynski, M.[Marek],
A probabilistic model of classifier competence for dynamic ensemble selection,
PR(44), No. 10-11, October-November 2011, pp. 2656-2668.
Elsevier DOI 1101
BibRef
Earlier:
A Measure of Competence Based on Randomized Reference Classifier for Dynamic Ensemble Selection,
ICPR10(4194-4197).
IEEE DOI 1008
BibRef
Earlier:
On a New Measure of Classifier Competence Applied to the Design of Multiclassifier Systems,
CIAP09(995-1004).
Springer DOI 0909
Probabilistic modelling; Classifier competence; Multiple classifier system; Beta distribution BibRef

Frasch, J.V.[Janick V.], Lodwich, A.[Aleksander], Shafait, F.[Faisal], Breuel, T.M.[Thomas M.],
A Bayes-true data generator for evaluation of supervised and unsupervised learning methods,
PRL(32), No. 11, 1 August 2011, pp. 1523-1531.
Elsevier DOI 1108
Synthetic data generation; Benchmarking; Experimental proofs BibRef

Chudzian, P.[Pawel],
Evaluation measures for kernel optimization,
PRL(33), No. 9, 1 July 2012, pp. 1108-1116.
Elsevier DOI 1202
Kernel evaluation measures; Kernel optimization; Kernel methods; Radial basis function kernel; Pattern classification Transform pattern to feature space. BibRef

Gey, S.[Servane],
Risk bounds for CART classifiers under a margin condition,
PR(45), No. 9, September 2012, pp. 3523-3534.
Elsevier DOI 1206
Classification; CART; Pruning; Margin; Risk bounds. Classification And Regression Trees (CART) BibRef

Liu, J., Zhang, Z.J., Yang, Y., Wang, M.,
Comments on 'Probabilities of false alarm and detection for the NAMF operating in Gaussian clutter',
SPLetters(19), No. 10, October 2012, pp. 671.
IEEE DOI 1209
BibRef

de França, F.O., Coelho, G.P., von Zuben, F.J.,
Predicting missing values with biclustering: A coherence-based approach,
PR(46), No. 5, May 2013, pp. 1255-1266.
Elsevier DOI 1302
Biclustering; Missing data imputation; Knowledge discovery; Quadratic programming BibRef

Richards, J.A., Kingsbury, N.G.,
Is There a Preferred Classifier for Operational Thematic Mapping?,
GeoRS(52), No. 5, May 2014, pp. 2715-2725.
IEEE DOI 1403
Classification BibRef

Liu, Q.B.[Qing-Bao], Dong, G.Z.[Guo-Zhu],
CPCQ: Contrast pattern based clustering quality index for categorical data,
PR(45), No. 4, 2012, pp. 1739-1748.
Elsevier DOI 1410
Clustering validation BibRef

Wong, T.T.[Tzu-Tsung],
Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation,
PR(48), No. 9, 2015, pp. 2839-2846.
Elsevier DOI 1506
Classification BibRef

Henriques, R.[Rui], Antunes, C.[Cláudia], Madeira, S.C.[Sara C.],
A structured view on pattern mining-based biclustering,
PR(48), No. 12, 2015, pp. 3941-3958.
Elsevier DOI 1509
Biclustering BibRef

Park, N.W.[No-Wook], Kyriakidis, P.C.[Phaedon C.], Hong, S.Y.[Suk-Young],
Spatial Estimation of Classification Accuracy Using Indicator Kriging with an Image-Derived Ambiguity Index,
RS(8), No. 4, 2016, pp. 320.
DOI Link 1604
BibRef

Condessa, F.[Filipe], Bioucas-Dias, J.M.[José M.], Kovacevic, J.[Jelena],
Performance measures for classification systems with rejection,
PR(63), No. 1, 2017, pp. 437-450.
Elsevier DOI 1612
Classification with rejection BibRef

Rosales-Méndez, H.[Henry], Ramírez-Cruz, Y.[Yunior],
A new micro-objects-based evaluation measure for co-clustering algorithms,
PRL(84), No. 1, 2016, pp. 142-148.
Elsevier DOI 1612
Clustering algorithm evaluation BibRef

Lei, Y.[Yang], Bezdek, J.C.[James C.], Romano, S.[Simone], Vinh, N.X.[Nguyen Xuan], Chan, J.[Jeffrey], Bailey, J.[James],
Ground truth bias in external cluster validity indices,
PR(65), No. 1, 2017, pp. 58-70.
Elsevier DOI 1702
External cluster validity indices BibRef

Wong, T.T.[Tzu-Tsung],
Parametric methods for comparing the performance of two classification algorithms evaluated by k-fold cross validation on multiple data sets,
PR(65), No. 1, 2017, pp. 97-107.
Elsevier DOI 1702
Classification BibRef

Kautz, T.[Thomas], Eskofier, B.M.[Bjoern M.], Pasluosta, C.F.[Cristian F.],
Generic performance measure for multiclass-classifiers,
PR(68), No. 1, 2017, pp. 111-125.
Elsevier DOI 1704
Classification BibRef

Jiang, G.X.[Gao-Xia], Wang, W.J.[Wen-Jian],
Error estimation based on variance analysis of k-fold cross-validation,
PR(69), No. 1, 2017, pp. 94-106.
Elsevier DOI 1706
Error estimation BibRef


Barddal, J.P., Gomes, H.M., de Souza Britto, A., Enembreck, F.,
A benchmark of classifiers on feature drifting data streams,
ICPR16(2180-2185)
IEEE DOI 1705
Adaptation models, Benchmark testing, Context, Decision trees, Detectors, Feature extraction, Generators BibRef

Wang, D.[Dong], Tan, X.Y.[Xiao-Yang],
Label-Denoising Auto-encoder for Classification with Inaccurate Supervision Information,
ICPR14(3648-3653)
IEEE DOI 1412
Data models BibRef

Cabrera, G.F.[Guillermo F.], Miller, C.J.[Christopher J.], Schneider, J.[Jeff],
Systematic Labeling Bias: De-biasing Where Everyone is Wrong,
ICPR14(4417-4422)
IEEE DOI 1412
Accuracy; Gold; Labeling; Logistics; Spirals; Standards; Training BibRef

Hajizadeh, S.[Siamak], Li, Z.L.[Zi-Li], Dollevoet, R.P.B.J.[Rolf P.B.J.], Tax, D.M.J.[David M.J.],
Evaluating Classification Performance with only Positive and Unlabeled Samples,
SSSPR14(233-242).
Springer DOI 1408
BibRef

Santos, J.M.[Jorge M.], Embrechts, M.[Mark],
A Family of Two-Dimensional Benchmark Data Sets and Its Application to Comparing Different Cluster Validation Indices,
MCPR14(41-50).
Springer DOI 1407
BibRef

Aghazadeh, O.[Omid], Carlsson, S.[Stefan],
Properties of Datasets Predict the Performance of Classifiers,
BMVC13(xx-yy).
DOI Link 1402
BibRef

Loyola-González, O.[Octavio], Martínez-Trinidad, J.F.[José Francisco], Carrasco-Ochoa, J.A.[Jesús Ariel], García-Borroto, M.[Milton],
A Novel Contrast Pattern Selection Method for Class Imbalance Problems,
MCPR17(42-52).
Springer DOI 1706
BibRef
Earlier:
Correlation of Resampling Methods for Contrast Pattern Based Classifiers,
MCPR15(93-102).
Springer DOI 1506
BibRef

García-Borroto, M.[Milton], Loyola-Gonzalez, O.[Octavio], Martínez-Trinidad, J.F.[José Francisco],
Comparing Quality Measures for Contrast Pattern Classifiers,
CIARP13(I:311-318).
Springer DOI 1311
BibRef

Welinder, P.[Peter], Welling, M.[Max], Perona, P.[Pietro],
A Lazy Man's Approach to Benchmarking: Semisupervised Classifier Evaluation and Recalibration,
CVPR13(3262-3269)
IEEE DOI 1309
BibRef

Wei, S.M.[Si-Ming], Yu, Y.Z.[Yi-Zhou],
Subspace segmentation with a Minimal Squared Frobenius Norm Representation,
ICPR12(3509-3512).
WWW Link. 1302
performs data clustering by solving a convex optimization problem BibRef

Berrar, D.[Daniel],
Null QQ plots: A simple graphical alternative to significance testing for the comparison of classifiers,
ICPR12(1852-1855).
WWW Link. 1302
BibRef

Li, J.[James], Sonmez, A.[Abdullah], Cataltepe, Z.[Zehra], Bax, E.[Eric],
Validation of Network Classifiers,
SSSPR12(448-457).
Springer DOI 1211
probably approximately correct bounds for classifier BibRef

Kallel, M.[Manel], Naouai, M.[Mohamed], Slama, Y.[Yosr],
New Metrics to Evaluate Pattern Recognition in Remote Sensing Images,
CIARP12(664-673).
Springer DOI 1209
BibRef

Ryazanov, V.V.[Vladimir V.],
Estimations of Clustering Quality via Evaluation of Its Stability,
CIARP14(432-439).
Springer DOI 1411
BibRef
Earlier:
Clustering of Incomplete Data and Evaluation of Clustering Quality,
CIARP12(146-153).
Springer DOI 1209
BibRef

Zhou, G.J.[Gui-Jun], Wu, B.[Bo], Li, M.M.[Meng-Meng],
Improved accuracy assessment indices for object-based high resolution remotely sensed imagery classification,
IASP11(181-186).
IEEE DOI 1112
Evaluation using regions, not just pixels. BibRef

Marsetic, A., Kokalj, Z., Ostir, K.,
The effect of lossy image compression on object based image classification: WorldView-2 case study,
HighRes11(xx-yy).
PDF File. 1106
BibRef

Weih, Jr., R.C.[Robert C.], Riggan, Jr., N.D.[Norman D.],
Object-Based Classification vs. Pixel-Based Classification: Comparitive Importance of Multi-Resolution Imagery,
GEOBIA10(xx-yy).
PDF File. 1007
BibRef

Huang, H.Q.[Hai-Qiao], Mok, P.Y.[Pik-Yin], Kwok, Y.L.[Yi-Lin], Au, S.C.[Sau-Chuen],
A Parameter Free Approach for Clustering Analysis,
CAIP09(816-823).
Springer DOI 0909
BibRef

Zhang, W.[Wei], Deng, H.L.[Hong-Li],
Understanding visual dictionaries via Maximum Mutual Information curves,
ICPR08(1-4).
IEEE DOI 0812
BibRef

Zhang, X.[Xiao], Liang, L.[Lin], Tang, X.[Xiaoou], Shum, H.Y.[Heung-Yeung],
L1 regularized projection pursuit for additive model learning,
CVPR08(1-8).
IEEE DOI 0806
BibRef

Su, Y.[Yu], Shan, S.G.[Shi-Guang], Chen, X.L.[Xi-Lin], Gao, W.[Wen],
Classifiability-based Optimal Discriminatory Projection Pursuit,
CVPR08(1-7).
IEEE DOI 0806
BibRef

González-Guevara, V.I.[Víctor Iván], Godoy-Calderon, S.[Salvador], Alba-Cabrera, E.[Eduardo], Ibarra-Fiallo, J.[Julio],
A Mixed Learning Strategy for Finding Typical Testors in Large Datasets,
CIARP15(716-723).
Springer DOI 1511
BibRef

Alba-Cabrera, E.[Eduardo], Ibarra-Fiallo, J.[Julio], Godoy-Calderon, S.[Salvador], Cervantes-Alonso, F.[Fernando],
YYC: A Fast Performance Incremental Algorithm for Finding Typical Testors,
CIARP14(416-423).
Springer DOI 1411
BibRef
Earlier: A1, A2, A3, Only:
A Theoretical and Practical Framework for Assessing the Computational Behavior of Typical Testor-Finding Algorithms,
CIARP13(I:351-358).
Springer DOI 1311
BibRef

Godoy-Calderón, S.[Salvador], Martínez-Trinidad, J.F., Cortés, M.L.[Manuel Lazo],
Proposal for a Unified Methodology for Evaluating Supervised and Non-supervised Classification Algorithms,
CIARP06(674-685).
Springer DOI 0611
BibRef

Mouchere, H.[Harold], Anquetil, E.[Eric],
A Unified Strategy to Deal with Different Natures of Reject,
ICPR06(II: 792-795).
IEEE DOI 0609
BibRef

Pugliese, L.[Luca], Scarpetta, S.[Silvia], Esposito, A.[Anna], Marinaro, M.[Maria],
An Application of Neural and Probabilistic Unsupervised Methods to Environmental Factor Analysis of Multi-spectral Images,
CIAP05(1190-1197).
Springer DOI 0509
compare 2 clustering methods for LANDSAT classification. BibRef

Maillard, P.[Philippe], Clausi, D.A.[David A.],
Comparing Classification Metrics for Labeling Segmented Remote Sensing Images,
CRV05(421-428).
IEEE DOI 0505
BibRef

Kolsch, T., Keysers, D., Ney, H., Paredes, R.,
Enhancements for local feature based image classification,
ICPR04(I: 248-251).
IEEE DOI 0409
Decompose and analyze nearest neighbor search and direct voting for classification. BibRef

Mansilla, E.B., Ho, T.K.[Tin Kam],
On classifier domains of competence,
ICPR04(I: 136-139).
IEEE DOI 0409
Where are which classifiers competent. BibRef

Johnson, A.Y.[Amos Y.], Sun, J.[Jie], Bobick, A.F.[Aaron F.],
Predicting Large Population Data Cumulative Match Characteristic Performance from Small Population Data,
AVBPA03(821-829).
Springer DOI 0310
BibRef

Lucas, S.M.,
Web-based evaluation and deployment of pattern recognizers,
ICPR02(III: 419-422).
IEEE DOI 0211
BibRef

Duin, R.P.W., Pekalska, E., Tax, D.M.J.,
The characterization of classification problems by classifier disagreements,
ICPR04(I: 140-143).
IEEE DOI 0409
BibRef

de Ridder, D., Pekalska, E., Duin, R.P.W.,
The economics of classification: error vs. complexity,
ICPR02(II: 244-247).
IEEE DOI 0211
BibRef

Varma, M., Zisserman, A.P.,
Classifying materials from images: to cluster or not to cluster?,
Texture02(139-144). 0207
BibRef

Yang, M.H., Roth, D., Ahuja, N.,
A Tale of Two Classifiers: SNoW vs. SVM in Visual Recognition,
ECCV02(IV: 685 ff.).
Springer DOI 0205
BibRef

Huijsmans, N.P., Sebe, N.,
Extended Performance Graphs for Cluster Retrieval,
CVPR01(I:26-31).
IEEE DOI 0110
In classification, issue of how to measure the performance expecially when true negatives are dominant. BibRef

Bromiley, P.A., Courtney, P., Thacker, N.A.,
A Case Study in the use of ROC curves for Algorithm Design,
BMVC01(Poster Session 1).
HTML Version. University of Manchester 0110
BibRef

Michaels, R., Boult, T.E.,
A Stratified Methodology for Classifier and Recognizer Evaluation,
EEMCV01(xx-yy). 0110
BibRef

Tang, M., Xiao, J., Ma, S.D.[Song De],
Two-step Classification Based on Scale Space,
ICPR00(Vol II: 899-902).
IEEE DOI 0009
BibRef

Scott, M.J.J., Niranjan, M., Prager, R.W.,
Realisable Classifiers: Improving Operating Performance on Variable Cost Problems,
BMVC98(xx-yy). BibRef 9800

Prager, R.W.,
CART/CMAC hybrid: regression trees with interpolation,
ICPR94(B:476-478).
IEEE DOI 9410
BibRef

Kraaijveld, M.A.[Martin A.],
An Experimental Comparison of Non-Parametric Classifiers for Time-Constrained Classification Tasks,
ICPR98(Vol I: 428-435).
IEEE DOI 9808
BibRef

Lashkia, V.[Vakhtang], Aleshin, S.,
Test Feature Classifiers: Performance and Applications,
ICPR98(Vol I: 341-343).
IEEE DOI 9808
Properties BibRef

Zhao, W.Y.[Wen-Yi], Chellappa, R.[Rama], Nandhakumar, N.,
Empirical Performance Analysis of Linear Discriminant Classifiers,
CVPR98(164-169).
IEEE DOI BibRef 9800

Gorski, N.,
Optimizing Error-Reject Trade-Off in Recognition Systems,
ICDAR97(1092-1096).
IEEE DOI 9708
BibRef

Altamirano-Robles, L., Eckstein, W.,
The Importance of Feature Visibility for the Evaluation of a Matching Hypothesis,
ICPR96(I: 585-589).
IEEE DOI 9608
(Technical Univ. of Munich, D) BibRef

Yu, S.[Shan], Weigl, K.[Konrad],
Performance comparison of a deterministic and a stochastic method for image classification,
CAIP95(826-831).
Springer DOI 9509
BibRef

Ford, S.J., Hampshire, II, J.B., McKeown, D.M.,
Performance Evaluation of Multispectral Analysis for Surface Material Classification,
DARPA93(421-435). Classification. BibRef 9300

Niblack, W., Petkovic, D., Damian, D.,
Experiments and Evaluations of Rule Based Methods in Image Analysis,
CVPR88(123-128).
IEEE DOI BibRef 8800

Chapter on Pattern Recognition, Clustering, Statistics, Grammars, Learning, Neural Nets, Genetic Algorithms continues in
Error Estimation, Classification Accuracy .


Last update:Dec 15, 2017 at 20:32:53