Learning and generalisation: with applications to neural networks
Describes the oceans of the world, their tides and coastlines, the animals and plants that live in them, ocean exploration, and more.
Gespeichert in:
| Vorheriger Titel: | Vidyasagar, Mathukumalli A theory of learning and generalization |
|---|---|
| 1. Verfasser: | |
| Format: | Buch |
| Sprache: | English |
| Veröffentlicht: |
London
Springer
2003
|
| Ausgabe: | 2. ed. |
| Schriftenreihe: | Communications and control engineering
|
| Schlagworte: | |
| Online-Zugang: | Inhaltsverzeichnis |
| Zusammenfassung: | Describes the oceans of the world, their tides and coastlines, the animals and plants that live in them, ocean exploration, and more. |
| Beschreibung: | Includes bibliographical references (p. [475]-484) and index |
| Beschreibung: | XXI, 488 S. Ill. |
| ISBN: | 1852333731 |
Internformat
MARC
| LEADER | 00000nam a2200000zc 4500 | ||
|---|---|---|---|
| 001 | BV014515864 | ||
| 003 | DE-604 | ||
| 005 | 20030218 | ||
| 007 | t | ||
| 008 | 020603s2003 xxka||| |||| 00||| eng d | ||
| 010 | |a 2002070674 | ||
| 020 | |a 1852333731 |9 1-85233-373-1 | ||
| 035 | |a (OCoLC)49859869 | ||
| 035 | |a (DE-599)BVBBV014515864 | ||
| 040 | |a DE-604 |b ger |e aacr | ||
| 041 | 0 | |a eng | |
| 044 | |a xxk |c GB | ||
| 049 | |a DE-91G |a DE-634 | ||
| 050 | 0 | |a Q325.5 | |
| 082 | 0 | |a 006.3/1 |2 21 | |
| 084 | |a ST 301 |0 (DE-625)143651: |2 rvk | ||
| 084 | |a MAT 914f |2 stub | ||
| 084 | |a DAT 708f |2 stub | ||
| 100 | 1 | |a Vidyasagar, Mathukumalli |d 1947- |e Verfasser |0 (DE-588)124243584 |4 aut | |
| 245 | 1 | 0 | |a Learning and generalisation |b with applications to neural networks |c M. Vidyasagar |
| 246 | 1 | 3 | |a Learning and generalization |
| 250 | |a 2. ed. | ||
| 264 | 1 | |a London |b Springer |c 2003 | |
| 300 | |a XXI, 488 S. |b Ill. | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 490 | 0 | |a Communications and control engineering | |
| 500 | |a Includes bibliographical references (p. [475]-484) and index | ||
| 520 | 3 | |a Describes the oceans of the world, their tides and coastlines, the animals and plants that live in them, ocean exploration, and more. | |
| 650 | 7 | |a Aprendizado computacional |2 larpcal | |
| 650 | 7 | |a Machine Learning Summer School |2 ram | |
| 650 | 7 | |a Redes neurais |2 larpcal | |
| 650 | 4 | |a Control theory | |
| 650 | 4 | |a Machine learning | |
| 650 | 4 | |a Neural networks (Computer science) | |
| 650 | 0 | 7 | |a Mathematische Lerntheorie |0 (DE-588)4169103-9 |2 gnd |9 rswk-swf |
| 689 | 0 | 0 | |a Mathematische Lerntheorie |0 (DE-588)4169103-9 |D s |
| 689 | 0 | |5 DE-604 | |
| 780 | 0 | 0 | |i 1. Auflage |a Vidyasagar, Mathukumalli |t A theory of learning and generalization |
| 856 | 4 | 2 | |m HBZ Datenaustausch |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=009886428&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 999 | |a oai:aleph.bib-bvb.de:BVB01-009886428 | ||
Datensatz im Suchindex
| _version_ | 1804129316953391104 |
|---|---|
| adam_text | Table of Contents
Preface to the Second Edition xiii
Preface to the First Edition xvii
1. Introduction 1
2. Preliminaries 13
2.1 Pseudometric Spaces, Packing and Covering Numbers 13
2.1.1 Pseudometric Spaces 13
2.1.2 Packing and Covering Numbers 14
2.1.3 Compact and Totally Bounded Sets 16
2.2 Probability Measures 17
2.2.1 Definition of a Probability Space 17
2.2.2 A Pseudometric Induced by a Probability Measure ... 18
2.2.3 A Metric on the Set of Probability Measures 19
2.2.4 Random Variables 21
2.2.5 Conditional Expectations 23
2.3 Large Deviation Type Inequalities 24
2.3.1 ChernofF Bounds 24
2.3.2 ChernofF Okamoto Bound 26
2.3.3 Hoeffding s Inequality 26
2.4 Stochastic Processes, Almost Sure Convergence 29
2.4.1 Probability Measures on Infinite Cartesian Products . . 29
2.4.2 Stochastic Processes 29
2.4.3 The Borel Cantelli Lemma and Almost Sure Conver¬
gence 30
2.5 Mixing Properties of Stochastic Processes 33
2.5.1 Definitions of Various Kinds of Mixing Coefficients ... 34
2.5.2 Inequalities for Mixing Processes 36
3. Problem Formulations 43
3.1 Uniform Convergence of Empirical Means 43
3.1.1 The UCEM Property 43
3.1.2 The UCEMUP Property 52
viii Table of Contents
3.1.3 Extension to Dependent Input Sequences 54
3.2 Learning Concepts and Functions 55
3.2.1 Concept Learning 55
3.2.2 Function Learning 64
3.2.3 Extension to Dependent Input Sequences 65
3.2.4 Assumptions Underlying the Model of Learning 66
3.2.5 Alternate Notions of Learnability 70
3.3 Model Free Learning 76
3.3.1 Problem Formulation 76
3.3.2 Relationship to the Uniform Convergence of Empirical
Means 81
3.4 Preservation of UCEMUP and PAC Properties 83
3.4.1 Preservation of UCEMUP Property with Beta Mixing
Inputs 84
3.4.2 Law of Large Numbers Under Alpha Mixing Inputs ... 89
3.4.3 Preservation of PAC Learning Property with Beta
Mixing Inputs 94
3.4.4 Preservation of PAC Learning Property with Beta
Mixing Inputs: Continued 95
3.4.5 Replacing V by its Closure 97
3.5 Markov Chains and Beta Mixing 100
3.5.1 Geometric Ergodicity and Beta Mixing 100
3.5.2 Beta Mixing Properties of Markov Sequences 105
3.5.3 Mixing Properties of Hidden Markov Models 110
4. Vapnik Chervonenkis, Pseudo and Fat Shattering Dimen¬
sions 115
4.1 Definitions 115
4.1.1 The Vapnik Chervonenkis Dimension 115
4.1.2 The Pseudo Dimension 120
4.1.3 The Fat Shattering Dimension 122
4.2 Bounds on Growth Functions 123
4.2.1 Growth Functions of Collections of Sets 123
4.2.2 Bounds on Covering Numbers Based on the Pseudo
Dimension 128
4.2.3 Metric Entropy Bounds for Families of Functions 132
4.2.4 Bounds on Covering Numbers Based on the Fat Shattering
Dimension 139
4.3 Growth Functions of Iterated Families 141
5. Uniform Convergence of Empirical Means 149
5.1 Restatement of the Problems Under Study 149
5.2 Equivalence of the UCEM and ASCEM Properties 153
5.3 Main Theorems 155
5.4 Preliminary Lemmas 161
Table of Contents ix
5.5 Theorem 5.1: Proof of Necessity 173
5.6 Theorem 5.1: Proof of Sufficiency 178
5.7 Proofs of the Remaining Theorems 190
5.8 Uniform Convergence Properties of Iterated Families 194
5.8.1 Boolean Operations on Collections of Sets 195
5.8.2 Uniformly Continuous Mappings on Families of Func¬
tions 196
5.8.3 Families of Loss Functions 200
6. Learning Under a Fixed Probability Measure 207
6.1 Introduction 207
6.2 UCEM Property Implies ASEC Learnability 209
6.3 Finite Metric Entropy Implies Learnability 216
6.4 Consistent Learnability 224
6.4.1 Consistent PAC Learnability 224
6.4.2 Consistent PUAC Learnability 226
6.5 Examples 230
6.6 Learnable Concept Classes Have Finite Metric Entropy 236
6.7 Model Free Learning 242
6.7.1 A Sufficient Condition for Learnability 244
6.7.2 A Necessary Condition 248
6.8 Dependent Inputs 250
6.8.1 Finite Metric Entropy and Alpha Mixing Input Se¬
quences 250
6.8.2 Consistent Learnability and Beta Mixing Input Se¬
quences 251
7. Distribution Free Learning 255
7.1 Uniform Convergence of Empirical Means 255
7.1.1 Function Classes 256
7.1.2 Concept Classes 258
7.1.3 Loss Functions 261
7.2 Function Learning 263
7.2.1 Finite P Dimension Implies PAC and PUAC Learn¬
ability 264
7.2.2 Finite P Dimension is not Necessary for PAC Learn¬
ability 267
7.3 Concept Learning 269
7.3.1 Improved Upper Bound for the Sample Complexity . . . 269
7.3.2 A Universal Lower Bound for the Sample Complexity . 273
7.3.3 Learnability Implies Finite VC Dimension 278
7.4 Learnability of Functions with a Finite Range 280
x Table of Contents
8. Learning Under an Intermediate Family of Probabilities . . 285
8.1 General Families of Probabilities 287
8.1.1 Uniform Convergence of Empirical Means 287
8.1.2 Function Learning 288
8.1.3 Concept Learning 292
8.2 Totally Bounded Families of Probabilities 297
8.3 Families of Probabilities with a Nonempty Interior 308
9. Alternate Models of Learning 311
9.1 Efficient Learning 312
9.1.1 Definition of Efficient Learnability 313
9.1.2 The Complexity of Finding a Consistent Hypothesis .. 317
9.2 Active Learning 326
9.2.1 Fixed Distribution Learning 329
9.2.2 Distribution Free Learning 332
9.3 Learning with Prior Information: Necessary and Sufficient
Conditions 335
9.3.1 Definition of Learnability with Prior Information 335
9.3.2 Some Simple Sufficient Conditions 337
9.3.3 Dispersability of Function Classes 341
9.3.4 Connections Between Dispersability and Learnability
WPI 344
9.3.5 Distribution Free Learning with Prior Information .... 348
9.4 Learning with Prior Information: Bounds on Learning Rates . 352
10. Applications to Neural Networks 365
10.1 What is a Neural Network? 366
10.2 Learning in Neural Networks 369
10.2.1 Problem Formulation 369
10.2.2 Reprise of Sample Complexity Estimates 372
10.2.3 Complexity Theoretic Limits to Learnability 377
10.3 Estimates of VC Dimensions of Families of Networks 381
10.3.1 Multi Layer Perceptron Networks 382
10.3.2 A Network with Infinite VC Dimension 388
10.3.3 Neural Networks as Verifiers of Formulas 390
10.3.4 Neural Networks with Piecewise Polynomial Activa¬
tion Functions 396
10.3.5 A General Approach 402
10.3.6 An Improved Bound 406
10.3.7 Networks with Pfaffian Activation Functions 410
10.3.8 Results Based on Order Minimality 413
10.4 Structural Risk Minimization 415
Table of Contents xi
11. Applications to Control Systems 421
11.1 Randomized Algorithms for Robustness Analysis 421
11.1.1 Introduction to Robust Control 421
11.1.2 Some NP Hard Problems in Robust Control 424
11.1.3 Randomized Algorithms for Robustness Analysis 426
11.2 Randomized Algorithms for Robust Controller Synthesis: Gen¬
eral Approach 429
11.2.1 Paradigm of Robust Controller Synthesis Problem .... 429
11.2.2 Various Types of Near Minima 432
11.2.3 A General Approach to Randomized Algorithms 435
11.2.4 Two Algorithms for Finding Probably Approximate
Near Minima 436
11.3 VC Dimension Estimates for Problems in Robust Controller
Synthesis 438
11.3.1 A General Result 438
11.3.2 Robust Stabilization 438
11.3.3 Weighted floo Norm Minimization 441
11.3.4 Weighted ff2 Norm Minimization 444
11.3.5 Sample Complexity Considerations 445
11.3.6 Robust Controller Design Using Randomized Algo¬
rithms: An Example 449
11.4 A Learning Theory Approach to System Identification 453
11.4.1 Problem Formulation 453
11.4.2 A General Result 455
11.4.3 Sufficient Conditions for the UCEM Property 458
11.4.4 Bounds on the P Dimension 461
12. Some Open Problems 465
|
| any_adam_object | 1 |
| author | Vidyasagar, Mathukumalli 1947- |
| author_GND | (DE-588)124243584 |
| author_facet | Vidyasagar, Mathukumalli 1947- |
| author_role | aut |
| author_sort | Vidyasagar, Mathukumalli 1947- |
| author_variant | m v mv |
| building | Verbundindex |
| bvnumber | BV014515864 |
| callnumber-first | Q - Science |
| callnumber-label | Q325 |
| callnumber-raw | Q325.5 |
| callnumber-search | Q325.5 |
| callnumber-sort | Q 3325.5 |
| callnumber-subject | Q - General Science |
| classification_rvk | ST 301 |
| classification_tum | MAT 914f DAT 708f |
| ctrlnum | (OCoLC)49859869 (DE-599)BVBBV014515864 |
| dewey-full | 006.3/1 |
| dewey-hundreds | 000 - Computer science, information, general works |
| dewey-ones | 006 - Special computer methods |
| dewey-raw | 006.3/1 |
| dewey-search | 006.3/1 |
| dewey-sort | 16.3 11 |
| dewey-tens | 000 - Computer science, information, general works |
| discipline | Informatik Mathematik |
| edition | 2. ed. |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>02077nam a2200529zc 4500</leader><controlfield tag="001">BV014515864</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20030218 </controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">020603s2003 xxka||| |||| 00||| eng d</controlfield><datafield tag="010" ind1=" " ind2=" "><subfield code="a">2002070674</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">1852333731</subfield><subfield code="9">1-85233-373-1</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)49859869</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BVBBV014515864</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">aacr</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="044" ind1=" " ind2=" "><subfield code="a">xxk</subfield><subfield code="c">GB</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-91G</subfield><subfield code="a">DE-634</subfield></datafield><datafield tag="050" ind1=" " ind2="0"><subfield code="a">Q325.5</subfield></datafield><datafield tag="082" ind1="0" ind2=" "><subfield code="a">006.3/1</subfield><subfield code="2">21</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 301</subfield><subfield code="0">(DE-625)143651:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">MAT 914f</subfield><subfield code="2">stub</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">DAT 708f</subfield><subfield code="2">stub</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Vidyasagar, Mathukumalli</subfield><subfield code="d">1947-</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)124243584</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Learning and generalisation</subfield><subfield code="b">with applications to neural networks</subfield><subfield code="c">M. Vidyasagar</subfield></datafield><datafield tag="246" ind1="1" ind2="3"><subfield code="a">Learning and generalization</subfield></datafield><datafield tag="250" ind1=" " ind2=" "><subfield code="a">2. ed.</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">London</subfield><subfield code="b">Springer</subfield><subfield code="c">2003</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">XXI, 488 S.</subfield><subfield code="b">Ill.</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="490" ind1="0" ind2=" "><subfield code="a">Communications and control engineering</subfield></datafield><datafield tag="500" ind1=" " ind2=" "><subfield code="a">Includes bibliographical references (p. [475]-484) and index</subfield></datafield><datafield tag="520" ind1="3" ind2=" "><subfield code="a">Describes the oceans of the world, their tides and coastlines, the animals and plants that live in them, ocean exploration, and more.</subfield></datafield><datafield tag="650" ind1=" " ind2="7"><subfield code="a">Aprendizado computacional</subfield><subfield code="2">larpcal</subfield></datafield><datafield tag="650" ind1=" " ind2="7"><subfield code="a">Machine Learning Summer School</subfield><subfield code="2">ram</subfield></datafield><datafield tag="650" ind1=" " ind2="7"><subfield code="a">Redes neurais</subfield><subfield code="2">larpcal</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Control theory</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Machine learning</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Neural networks (Computer science)</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Mathematische Lerntheorie</subfield><subfield code="0">(DE-588)4169103-9</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Mathematische Lerntheorie</subfield><subfield code="0">(DE-588)4169103-9</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="780" ind1="0" ind2="0"><subfield code="i">1. Auflage</subfield><subfield code="a">Vidyasagar, Mathukumalli</subfield><subfield code="t">A theory of learning and generalization</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">HBZ Datenaustausch</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=009886428&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-009886428</subfield></datafield></record></collection> |
| id | DE-604.BV014515864 |
| illustrated | Illustrated |
| indexdate | 2024-07-09T19:03:13Z |
| institution | BVB |
| isbn | 1852333731 |
| language | English |
| lccn | 2002070674 |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-009886428 |
| oclc_num | 49859869 |
| open_access_boolean | |
| owner | DE-91G DE-BY-TUM DE-634 |
| owner_facet | DE-91G DE-BY-TUM DE-634 |
| physical | XXI, 488 S. Ill. |
| publishDate | 2003 |
| publishDateSearch | 2003 |
| publishDateSort | 2003 |
| publisher | Springer |
| record_format | marc |
| series2 | Communications and control engineering |
| spelling | Vidyasagar, Mathukumalli 1947- Verfasser (DE-588)124243584 aut Learning and generalisation with applications to neural networks M. Vidyasagar Learning and generalization 2. ed. London Springer 2003 XXI, 488 S. Ill. txt rdacontent n rdamedia nc rdacarrier Communications and control engineering Includes bibliographical references (p. [475]-484) and index Describes the oceans of the world, their tides and coastlines, the animals and plants that live in them, ocean exploration, and more. Aprendizado computacional larpcal Machine Learning Summer School ram Redes neurais larpcal Control theory Machine learning Neural networks (Computer science) Mathematische Lerntheorie (DE-588)4169103-9 gnd rswk-swf Mathematische Lerntheorie (DE-588)4169103-9 s DE-604 1. Auflage Vidyasagar, Mathukumalli A theory of learning and generalization HBZ Datenaustausch application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=009886428&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Vidyasagar, Mathukumalli 1947- Learning and generalisation with applications to neural networks Aprendizado computacional larpcal Machine Learning Summer School ram Redes neurais larpcal Control theory Machine learning Neural networks (Computer science) Mathematische Lerntheorie (DE-588)4169103-9 gnd |
| subject_GND | (DE-588)4169103-9 |
| title | Learning and generalisation with applications to neural networks |
| title_alt | Learning and generalization |
| title_auth | Learning and generalisation with applications to neural networks |
| title_exact_search | Learning and generalisation with applications to neural networks |
| title_full | Learning and generalisation with applications to neural networks M. Vidyasagar |
| title_fullStr | Learning and generalisation with applications to neural networks M. Vidyasagar |
| title_full_unstemmed | Learning and generalisation with applications to neural networks M. Vidyasagar |
| title_old | Vidyasagar, Mathukumalli A theory of learning and generalization |
| title_short | Learning and generalisation |
| title_sort | learning and generalisation with applications to neural networks |
| title_sub | with applications to neural networks |
| topic | Aprendizado computacional larpcal Machine Learning Summer School ram Redes neurais larpcal Control theory Machine learning Neural networks (Computer science) Mathematische Lerntheorie (DE-588)4169103-9 gnd |
| topic_facet | Aprendizado computacional Machine Learning Summer School Redes neurais Control theory Machine learning Neural networks (Computer science) Mathematische Lerntheorie |
| url | http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=009886428&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| work_keys_str_mv | AT vidyasagarmathukumalli learningandgeneralisationwithapplicationstoneuralnetworks AT vidyasagarmathukumalli learningandgeneralization |