Pattern classification using ensemble methods:
Saved in:
| Main Author: | |
|---|---|
| Format: | Book |
| Language: | English |
| Published: |
New Jersey [u.a.]
World Scientific
2010
|
| Series: | Series in machine perception and artificial intelligence
75 |
| Subjects: | |
| Online Access: | Inhaltsverzeichnis |
| Physical Description: | XV, 225 S. graph. Darst. |
| ISBN: | 9789814271066 9814271063 |
Staff View
MARC
| LEADER | 00000nam a2200000 cb4500 | ||
|---|---|---|---|
| 001 | BV036107722 | ||
| 003 | DE-604 | ||
| 005 | 20100426 | ||
| 007 | t | ||
| 008 | 100406s2010 |||| 00||| eng d | ||
| 020 | |a 9789814271066 |9 978-981-4271-06-6 | ||
| 020 | |a 9814271063 |9 981-4271-06-3 | ||
| 035 | |a (OCoLC)540206468 | ||
| 035 | |a (DE-599)BSZ307024334 | ||
| 040 | |a DE-604 |b ger | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-703 |a DE-11 | ||
| 082 | 0 | |a 621.38928 |2 22 | |
| 084 | |a ST 330 |0 (DE-625)143663: |2 rvk | ||
| 100 | 1 | |a Rokach, Lior |e Verfasser |4 aut | |
| 245 | 1 | 0 | |a Pattern classification using ensemble methods |c Lior Rokach |
| 264 | 1 | |a New Jersey [u.a.] |b World Scientific |c 2010 | |
| 300 | |a XV, 225 S. |c graph. Darst. | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 490 | 1 | |a Series in machine perception and artificial intelligence |v 75 | |
| 650 | 0 | 7 | |a Mustererkennung |0 (DE-588)4040936-3 |2 gnd |9 rswk-swf |
| 689 | 0 | 0 | |a Mustererkennung |0 (DE-588)4040936-3 |D s |
| 689 | 0 | |5 DE-604 | |
| 830 | 0 | |a Series in machine perception and artificial intelligence |v 75 |w (DE-604)BV006668231 |9 75 | |
| 856 | 4 | 2 | |m Digitalisierung UB Bayreuth |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=018997956&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 999 | |a oai:aleph.bib-bvb.de:BVB01-018997956 | ||
Record in the Search Index
| _version_ | 1804141181247946752 |
|---|---|
| adam_text | Contents
Preface
vii
1.
Introduction
to Pattern Classification
1
1.1
Pattern Classification
..................... 2
1.2
Induction Algorithms
...................... 4
1.3
Rule Induction
......................... 5
1.4
Decision Trees
.......................... 5
1.5
Bayesian Methods
....................... 8
1.5.1
Overview
......................... 8
1.5.2
Naïve Bayes.......................
9
1.5.2.1
The Basic
Naïve Bayes
Classifier
...... 9
1.5.2.2
Naïve Bayes
Induction for Numeric
Attributes
................... 12
1.5.2.3
Correction to the Probability Estimation
. . 12
1.5.2.4
Laplace Correction
.............. 13
1.5.2.5
No Match
................... 14
1.5.3
Other Bayesian Methods
................ 14
1.6
Other Induction Methods
................... 14
1.6.1
Neural Networks
.................... 14
1.6.2
Genetic Algorithms
................... 17
1.6.3
Instance-based Learning
................ 17
1.6.4
Support Vector Machines
............... 18
2.
Introduction to Ensemble Learning
19
2.1
Back to the Roots
.......................
20
2.2
The Wisdom of Crowds
....................
22
xii
Pattern Classification Using Ensemble Methods
2.3
The Bagging Algorithm
.................... 22
2.4
The Boosting Algorithm
.................... 28
2.5
The AdaBoost Algorithm
................... 28
2.6
No Free Lunch Theorem and Ensemble Learning
...... 36
2.7
Bias-Variance Decomposition and Ensemble Learning
.... 38
2.8
Occam s Razor and Ensemble Learning
............ 40
2.9
Classifier Dependency
..................... 41
2.9.1
Dependent Methods
.................. 42
2.9.1.1
Model-guided Instance Selection
....... 42
2.9.1.2
Basic Boosting Algorithms
.......... 42
2.9.1.3
Advanced Boosting Algorithms
....... 44
2.9.1.4
Incremental Batch Learning
......... 51
2.9.2
Independent Methods
................. 51
2.9.2.1
Bagging
.................... 53
2.9.2.2
Wagging
.................... 54
2.9.2.3
Random Forest and Random Subspace
Projection
................... 55
2.9.2.4
Non-Linear Boosting Projection (NLBP)
. . 56
2.9.2.5
Cross-validated Committees
......... 58
2.9.2.6
Robust Boosting
............... 59
2.10
Ensemble Methods for Advanced Classification Tasks
. ... 61
2.10.1
Cost-Sensitive Classification
.............. 61
2.10.2
Ensemble for Learning Concept Drift
......... 63
2.10.3
Reject Driven Classification
.............. 63
3.
Ensemble Classification
65
3.1
Fusions Methods
........................ 65
3.1.1
Weighting Methods
................... 65
3.1.2
Majority Voting
..................... 66
3.1.3
Performance Weighting
................ 67
3.1.4
Distribution Summation
................ 68
3.1.5
Bayesian Combination
................. 68
3.1.6
Dempster-Shafer
.................... 69
3.1.7
Vogging
......................... 69
3.1.8
Naïve Bayes.......................
69
3.1.9
Entropy Weighting
................... 70
3.1.10
Density-based Weighting
............... 70
3.1.11
DEA
Weighting Method
................ 70
3.1.12
Logarithmic Opinion Pool
............... 71
Contents xiii
3.1.13 Order
Statistics.....................
71
3.2
Selecting Classification
..................... 71
3.2.1
Partitioning the Instance Space
............ 74
3.2.1.1
The K-Means Algorithm as a Decomposition
Tool
...................... 75
3.2.1.2
Determining the Number of Subsets
..... 78
3.2.1.3
The Basic K-Classifier Algorithm
...... 78
3.2.1.4
The Heterogeneity Detecting K-Classifier
(HDK-Classifier)
............... 81
3.2.1.5
Running-Time Complexity
.......... 81
3.3
Mixture of Experts and
Meta
Learning
............ 82
3.3.1
Stacking
......................... 82
3.3.2
Arbiter Trees
...................... 85
3.3.3
Combiner Trees
..................... 88
3.3.4
Grading
......................... 88
3.3.5
Gating Network
..................... 89
4.
Ensemble Diversity
93
4.1
Overview
............................ 93
4.2
Manipulating the Inducer
................... 94
4.2.1
Manipulation of the Inducer s Parameters
...... 95
4.2.2
Starting Point in Hypothesis Space
.......... 95
4.2.3
Hypothesis Space Traversal
.............. 95
4.3
Manipulating the Training Samples
.............. 96
4.3.1
Resampling
....................... 96
4.3.2
Creation
......................... 97
4.3.3
Partitioning
....................... 100
4.4
Manipulating the Target Attribute Representation
..... 101
4.4.1
Label Switching
.................... 102
4.5
Partitioning the Search Space
................. 103
4.5.1
Divide and Conquer
.................. 104
4.5.2
Feature Subset-based Ensemble Methods
....... 105
4.5.2.1
Random-based Strategy
........... 106
4.5.2.2
Reduct-based Strategy
............ 106
4.5.2.3
Collective-Performance-based Strategy
... 107
4.5.2.4
Feature Set Partitioning
........... 108
4.5.2.5
Rotation Forest
................
Ш
4.6
Multi-Inducers
.........................
112
4.7
Measuring the Diversity
....................
H4
xiv
Pattern Classification Using Ensemble Methods
5.
Ensemble Selection
119
5.1
Ensemble Selection
....................... 119
5.2
Pre
Selection of the Ensemble Size
.............. 120
5.3
Selection of the Ensemble Size While Training
........ 120
5.4
Pruning
-
Post Selection of the Ensemble Size
........ 121
5.4.1
Ranking-based
..................... 122
5.4.2
Search based Methods
................. 123
5.4.2.1
Collective Agreement-based Ensemble
Pruning Method
................ 124
5.4.3
Clustering-based Methods
............... 129
5.4.4
Pruning Timing
..................... 129
5.4.4.1
Pre-combining Pruning
............ 129
5.4.4.2
Post-combining Pruning
........... 130
6.
Error Correcting Output Codes
133
6.1
Code-matrix Decomposition of Multiclass Problems
..... 135
6.2
Type I
-
Training an Ensemble Given a Code-Matrix
.... 136
6.2.1
Error correcting output codes
............. 138
6.2.2
Code-Matrix Framework
................ 139
6.2.3
Code-matrix Design Problem
............. 140
6.2.4
Orthogonal Arrays (OA)
................ 144
6.2.5
Hadamard
Matrix
.................... 146
6.2.6
Probabilistic Error Correcting Output Code
..... 146
6.2.7
Other ECOC Strategies
................ 147
6.3
Type II
-
Adapting Code-matrices to the Multiclass
Problems
............................ 149
7.
Evaluating Ensembles of Classifiers
153
7.1
Generalization Error
...................... 153
7.1.1
Theoretical Estimation of Generalization Error
. . . 154
7.1.2
Empirical Estimation of Generalization Error
.... 155
7.1.3
Alternatives to the Accuracy Measure
........ 157
7.1.4
TheF-Measure
..................... 158
7.1.5
Confusion Matrix
.................... 160
7.1.6
Classifier Evaluation under Limited Resources
.... 161
7.1.6.1
ROC Curves
.................. 163
7.1.6.2
Hit Rate Curve
................ 163
7.1.6.3
Qrecall (Quota Recall)
............ 164
Contents xv
7.1.6.4
Lift Curve
................... 164
7.1.6.5
Pearson Correlation Coefficient
....... 165
7.1.6.6
Area Under Curve (AUC)
.......... 166
7.1.6.7
Average Hit Rate
............... 167
7.1.6.8
Average Qrecall
................ 168
7.1.6.9
Potential Extract Measure
(РЕМ)
...... 170
7.1.7
Statistical Tests for Comparing Ensembles
...... 172
7.1.7.1
McNemar s Test
................ 173
7.1.7.2
A Test for the Difference of Two
Proportions
.................. 174
7.1.7.3
The Resampled Paired
t
Test
........ 175
7.1.7.4
The fc-fold Cross-validated Paired
t
Test
. . 176
7.2
Computational Complexity
.................. 176
7.3
Interpretability of the Resulting Ensemble
.......... 177
7.4
Scalability to Large
Datasets
................. 178
7.5
Robustness
........................... 179
7.6
Stability
............................. 180
7.7
Flexibility
............................ 180
7.8
Usability
............................. 180
7.9
Software Availability
...................... 180
7.10
Which Ensemble Method Should be Used?
.......... 181
Bibliography
185
Index
223
|
| any_adam_object | 1 |
| author | Rokach, Lior |
| author_facet | Rokach, Lior |
| author_role | aut |
| author_sort | Rokach, Lior |
| author_variant | l r lr |
| building | Verbundindex |
| bvnumber | BV036107722 |
| classification_rvk | ST 330 |
| ctrlnum | (OCoLC)540206468 (DE-599)BSZ307024334 |
| dewey-full | 621.38928 |
| dewey-hundreds | 600 - Technology (Applied sciences) |
| dewey-ones | 621 - Applied physics |
| dewey-raw | 621.38928 |
| dewey-search | 621.38928 |
| dewey-sort | 3621.38928 |
| dewey-tens | 620 - Engineering and allied operations |
| discipline | Informatik Elektrotechnik / Elektronik / Nachrichtentechnik |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01419nam a2200361 cb4500</leader><controlfield tag="001">BV036107722</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20100426 </controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">100406s2010 |||| 00||| eng d</controlfield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9789814271066</subfield><subfield code="9">978-981-4271-06-6</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9814271063</subfield><subfield code="9">981-4271-06-3</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)540206468</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BSZ307024334</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-703</subfield><subfield code="a">DE-11</subfield></datafield><datafield tag="082" ind1="0" ind2=" "><subfield code="a">621.38928</subfield><subfield code="2">22</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 330</subfield><subfield code="0">(DE-625)143663:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Rokach, Lior</subfield><subfield code="e">Verfasser</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Pattern classification using ensemble methods</subfield><subfield code="c">Lior Rokach</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">New Jersey [u.a.]</subfield><subfield code="b">World Scientific</subfield><subfield code="c">2010</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">XV, 225 S.</subfield><subfield code="c">graph. Darst.</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="490" ind1="1" ind2=" "><subfield code="a">Series in machine perception and artificial intelligence</subfield><subfield code="v">75</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Mustererkennung</subfield><subfield code="0">(DE-588)4040936-3</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Mustererkennung</subfield><subfield code="0">(DE-588)4040936-3</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="830" ind1=" " ind2="0"><subfield code="a">Series in machine perception and artificial intelligence</subfield><subfield code="v">75</subfield><subfield code="w">(DE-604)BV006668231</subfield><subfield code="9">75</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">Digitalisierung UB Bayreuth</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=018997956&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-018997956</subfield></datafield></record></collection> |
| id | DE-604.BV036107722 |
| illustrated | Not Illustrated |
| indexdate | 2024-07-09T22:11:47Z |
| institution | BVB |
| isbn | 9789814271066 9814271063 |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-018997956 |
| oclc_num | 540206468 |
| open_access_boolean | |
| owner | DE-703 DE-11 |
| owner_facet | DE-703 DE-11 |
| physical | XV, 225 S. graph. Darst. |
| publishDate | 2010 |
| publishDateSearch | 2010 |
| publishDateSort | 2010 |
| publisher | World Scientific |
| record_format | marc |
| series | Series in machine perception and artificial intelligence |
| series2 | Series in machine perception and artificial intelligence |
| spelling | Rokach, Lior Verfasser aut Pattern classification using ensemble methods Lior Rokach New Jersey [u.a.] World Scientific 2010 XV, 225 S. graph. Darst. txt rdacontent n rdamedia nc rdacarrier Series in machine perception and artificial intelligence 75 Mustererkennung (DE-588)4040936-3 gnd rswk-swf Mustererkennung (DE-588)4040936-3 s DE-604 Series in machine perception and artificial intelligence 75 (DE-604)BV006668231 75 Digitalisierung UB Bayreuth application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=018997956&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Rokach, Lior Pattern classification using ensemble methods Series in machine perception and artificial intelligence Mustererkennung (DE-588)4040936-3 gnd |
| subject_GND | (DE-588)4040936-3 |
| title | Pattern classification using ensemble methods |
| title_auth | Pattern classification using ensemble methods |
| title_exact_search | Pattern classification using ensemble methods |
| title_full | Pattern classification using ensemble methods Lior Rokach |
| title_fullStr | Pattern classification using ensemble methods Lior Rokach |
| title_full_unstemmed | Pattern classification using ensemble methods Lior Rokach |
| title_short | Pattern classification using ensemble methods |
| title_sort | pattern classification using ensemble methods |
| topic | Mustererkennung (DE-588)4040936-3 gnd |
| topic_facet | Mustererkennung |
| url | http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=018997956&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| volume_link | (DE-604)BV006668231 |
| work_keys_str_mv | AT rokachlior patternclassificationusingensemblemethods |