Theory of neural information processing systems:
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Book |
| Language: | English |
| Published: |
Oxford [u.a.]
Oxford Univ. Press
2005
|
| Edition: | 1. publ. |
| Subjects: | |
| Online Access: | Inhaltsverzeichnis |
| Physical Description: | XVI, 569 S graph. Darst. |
| ISBN: | 0198530242 0198530234 9780198530237 9780198530244 |
Staff View
MARC
| LEADER | 00000nam a2200000 c 4500 | ||
|---|---|---|---|
| 001 | BV020042577 | ||
| 003 | DE-604 | ||
| 005 | 20130114 | ||
| 007 | t | ||
| 008 | 050923s2005 d||| |||| 00||| eng d | ||
| 020 | |a 0198530242 |c (pbk) : £30.00 |9 0-19-853024-2 | ||
| 020 | |a 0198530234 |c (cased) : £75.00 |9 0-19-853023-4 | ||
| 020 | |a 9780198530237 |9 978-0-19-853023-7 | ||
| 020 | |a 9780198530244 |9 978-0-19-853024-4 | ||
| 035 | |a (OCoLC)61301413 | ||
| 035 | |a (DE-599)BVBBV020042577 | ||
| 040 | |a DE-604 |b ger |e aacr | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-473 |a DE-355 |a DE-91G |a DE-634 |a DE-19 |a DE-11 | ||
| 050 | 0 | |a QA76.87 | |
| 082 | 0 | |a 006.32 |2 22 | |
| 084 | |a ST 301 |0 (DE-625)143651: |2 rvk | ||
| 084 | |a DAT 717f |2 stub | ||
| 100 | 1 | |a Coolen, Anthony C. C. |e Verfasser |4 aut | |
| 245 | 1 | 0 | |a Theory of neural information processing systems |c A. C. C. Coolen, R. Kühn, P. Sollich |
| 250 | |a 1. publ. | ||
| 264 | 1 | |a Oxford [u.a.] |b Oxford Univ. Press |c 2005 | |
| 300 | |a XVI, 569 S |b graph. Darst. | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 650 | 7 | |a Informatieverwerking (computer) |2 gtt | |
| 650 | 7 | |a Zenuwstelsel |2 gtt | |
| 650 | 4 | |a Neural networks (Computer science) | |
| 650 | 0 | 7 | |a Neuronales Netz |0 (DE-588)4226127-2 |2 gnd |9 rswk-swf |
| 689 | 0 | 0 | |a Neuronales Netz |0 (DE-588)4226127-2 |D s |
| 689 | 0 | |5 DE-604 | |
| 700 | 1 | |a Kühn, Reimer |e Verfasser |4 aut | |
| 700 | 1 | |a Sollich, Peter |e Verfasser |4 aut | |
| 856 | 4 | 2 | |m HBZ Datenaustausch |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=013363632&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 999 | |a oai:aleph.bib-bvb.de:BVB01-013363632 | ||
Record in the Search Index
| _version_ | 1804133615258304512 |
|---|---|
| adam_text | Contents
Part I Introduction to neural networks
1 General introduction 3
1.1 Principles of neural information processing 1.2 Biological neurons and model neurons 1.3 Universality of McCulloch-Pitts neurons 1.4 Exercises 3 7 21 25
2 Layered networks 29
2.1 Linear separability 2.2 Multilayer networks 2.3 The perceptron 2.4 Learning in layered networks: error backpropagation 2.5 Learning dynamics in small learning rate perceptrons 2.6 Numerical simulations 2.7 Exercises 29 33 36 46 52 58 64
3 Recurrent networks with binary neurons 69
3.1 Noiseless recurrent networks 3.2 Synaptic symmetry and Lyapunov functions 3.3 Information processing in recurrent networks 3.4 Exercises 70 77 81 87
4 Notes and suggestions for further reading 89
Part II Advanced neural networks 93
5 Competitive unsupervised learning processes 95
5.1 Vector quantization 95
5.2 Soft vector quantization 98
xH Contents
5.3 Time-dependent learning rates 110
5.4 Self-organizing maps 114
5.5 Exercises 122
6 Bayesian techniques in supervised learning 127
6.1 Preliminaries and introduction 127
6.2 Bayesian learning of network weights 136
6.3 Predictions with error bars: real-valued functions 144
6.4 Predictions with error bars: binary classification 152
6.5 Bayesian model selection 156
6.6 Practicalities: measuring curvature 163
6.7 Exercises 164
7 Gaussian processes 169
7.1 The underlying idea 169
7.2 Examples of networks reducing to Gaussian processes 172
7.3 The priors over functions point of view 176
7.4 Stationary covariance functions 177
7.5 Learning and prediction with Gaussian processes 180
7.6 Exercises 182
8 Support vector machines for binary classification 185
8.1 Optimal separating plane for linearly separable tasks 185
8.2 Representation in terms of support vectors 189
8.3 Preprocessing and SVM kernels 197
8.4 Exercises 202
9 Notes and suggestions for further reading 205
Part HI Information theory and neural networks 207
10 Measuring information 209
10.1 Brute force: counting messages 209
10.2 Exploiting message likelihood differences via coding 212
10.3 Proposal for a measure of information 218
Contents xüi
11 Identification of entropy as an information measure 223
11.1 Coding theory and the Kraft inequality 223
11.2 Entropy and optimal coding 229
11.3 Shannon s original proof 233
12 Building blocks of Shannon s information theory 235
12.1 Entropy 235
12.2 Joint and conditional entropy 240
12.3 Relative entropy and mutual information 245
12.4 Information measures for continuous random variables 251
12.5 Exercises 258
13 Information theory and statistical inference 261
13.1 Maximum likelihood estimation 261
13.2 The maximum entropy principle 264
13.3 Exercises 270
14 Applications to neural networks 273
14.1 Supervised learning: Boltzmann machines 273
14.2 Maximum information preservation 281
14.3 Neuronal specialization 285
14.4 Detection of coherent features 294
14.5 The effect of non-linearities 297
14.6 Introduction to Amari s information geometry 299
14.7 Simple applications of information geometry 306
14.8 Exercises 311
15 Notes and suggestions for further reading 313
Part IV Macroscopic analysis of dynamics 315
16 Network operation: macroscopic dynamics 317
16.1 Microscopic dynamics in probabilistic form 318
16.2 Sequential dynamics 324
16.3 Parallel dynamics 338
16.4 Exercises 345
xîv Contents
17 Dynamics of online learning in binary perceptrons 349
17.1 Probabilistic definitions, performance measures 349
17.2 Explicit learning rules 353
17.3 Optimized learning rules 368
17.4 Exercises 382
18 Dynamics of online gradient descent learning 385
18.1 Online gradient descent 385
18.2 Learning from noisy examples 392
18.3 Exercises 394
19 Notes and suggestions for further reading 397
Part V Equilibrium statistical mechanics of neural networks 399
20 Basics of equilibrium statistical mechanics 401
20.1 Stationary distributions and ergodicity 401
20.2 Detailed balance and interaction symmetry 408
20.3 Equilibrium statistical mechanics: concepts, definitions 413
20.4 A simple example: storing a single pattern 419
20.5 Phase transitions and ergodicity breaking 425
20.6 Exercises 430
21 Network operation: equilibrium analysis 437
21.1 Hopfield model with finite number of patterns 438
21.2 Introduction to replica theory: the SK model 447
21.3 Hopfield model with an extensive number of patterns 460
21.4 Exercises 480
22 Gardner theory of task readability 489
22.1 The space of interactions 489
22.2 Capacity of perceptrons—definition and toy example 494
22.3 Capacity of perceptrons—random inputs 498
23 Notes and suggestions for further reading 507
Contents xv
Appendix A: Probability theory in a nutshell 511
A.1 Discrete event sets A.2 Continuous event sets A.3 Averages of specific random variables 511 513 515
Appendix B: Conditions for the central limit theorem to apply 517
B.1 Moment condition B.2 Lindeberg s theorem 517 518
Appendix C: Some simple summation identities 521
Appendix D: Gaussian integrals and probability distributions 523
D.1 General properties of Gaussian integrals D.2 Gaussian probability distributions D.3 A list of specific Gaussian integrals 523 527 531
Appendix E: Matrix identities 537
E.1 Block inverses E.2 The Woodbury formula 537 538
Appendix F: The ¿-distribution 539
F.1 Definition F.2 {x) as solution of the Liouville equation F.3 Representations, relations, and generalizations 539 540 541
Appendix G: Inequalities based on convexity 543
Appendix H: Metrics for parametrized probability distributions 549
H.1 Local distance definitions 549
H.2 The triangular inequality 550
H.3 Global distance definitions 551
xvi Contents
Appendix 1: Saddle-point integration 553
References 555
Index 563
|
| any_adam_object | 1 |
| author | Coolen, Anthony C. C. Kühn, Reimer Sollich, Peter |
| author_facet | Coolen, Anthony C. C. Kühn, Reimer Sollich, Peter |
| author_role | aut aut aut |
| author_sort | Coolen, Anthony C. C. |
| author_variant | a c c c acc accc r k rk p s ps |
| building | Verbundindex |
| bvnumber | BV020042577 |
| callnumber-first | Q - Science |
| callnumber-label | QA76 |
| callnumber-raw | QA76.87 |
| callnumber-search | QA76.87 |
| callnumber-sort | QA 276.87 |
| callnumber-subject | QA - Mathematics |
| classification_rvk | ST 301 |
| classification_tum | DAT 717f |
| ctrlnum | (OCoLC)61301413 (DE-599)BVBBV020042577 |
| dewey-full | 006.32 |
| dewey-hundreds | 000 - Computer science, information, general works |
| dewey-ones | 006 - Special computer methods |
| dewey-raw | 006.32 |
| dewey-search | 006.32 |
| dewey-sort | 16.32 |
| dewey-tens | 000 - Computer science, information, general works |
| discipline | Informatik |
| edition | 1. publ. |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01739nam a2200457 c 4500</leader><controlfield tag="001">BV020042577</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20130114 </controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">050923s2005 d||| |||| 00||| eng d</controlfield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">0198530242</subfield><subfield code="c">(pbk) : £30.00</subfield><subfield code="9">0-19-853024-2</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">0198530234</subfield><subfield code="c">(cased) : £75.00</subfield><subfield code="9">0-19-853023-4</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9780198530237</subfield><subfield code="9">978-0-19-853023-7</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9780198530244</subfield><subfield code="9">978-0-19-853024-4</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)61301413</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BVBBV020042577</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">aacr</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-473</subfield><subfield code="a">DE-355</subfield><subfield code="a">DE-91G</subfield><subfield code="a">DE-634</subfield><subfield code="a">DE-19</subfield><subfield code="a">DE-11</subfield></datafield><datafield tag="050" ind1=" " ind2="0"><subfield code="a">QA76.87</subfield></datafield><datafield tag="082" ind1="0" ind2=" "><subfield code="a">006.32</subfield><subfield code="2">22</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 301</subfield><subfield code="0">(DE-625)143651:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">DAT 717f</subfield><subfield code="2">stub</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Coolen, Anthony C. C.</subfield><subfield code="e">Verfasser</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Theory of neural information processing systems</subfield><subfield code="c">A. C. C. Coolen, R. Kühn, P. Sollich</subfield></datafield><datafield tag="250" ind1=" " ind2=" "><subfield code="a">1. publ.</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">Oxford [u.a.]</subfield><subfield code="b">Oxford Univ. Press</subfield><subfield code="c">2005</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">XVI, 569 S</subfield><subfield code="b">graph. Darst.</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="650" ind1=" " ind2="7"><subfield code="a">Informatieverwerking (computer)</subfield><subfield code="2">gtt</subfield></datafield><datafield tag="650" ind1=" " ind2="7"><subfield code="a">Zenuwstelsel</subfield><subfield code="2">gtt</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Neural networks (Computer science)</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Neuronales Netz</subfield><subfield code="0">(DE-588)4226127-2</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Neuronales Netz</subfield><subfield code="0">(DE-588)4226127-2</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Kühn, Reimer</subfield><subfield code="e">Verfasser</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Sollich, Peter</subfield><subfield code="e">Verfasser</subfield><subfield code="4">aut</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">HBZ Datenaustausch</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=013363632&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-013363632</subfield></datafield></record></collection> |
| id | DE-604.BV020042577 |
| illustrated | Illustrated |
| indexdate | 2024-07-09T20:11:32Z |
| institution | BVB |
| isbn | 0198530242 0198530234 9780198530237 9780198530244 |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-013363632 |
| oclc_num | 61301413 |
| open_access_boolean | |
| owner | DE-473 DE-BY-UBG DE-355 DE-BY-UBR DE-91G DE-BY-TUM DE-634 DE-19 DE-BY-UBM DE-11 |
| owner_facet | DE-473 DE-BY-UBG DE-355 DE-BY-UBR DE-91G DE-BY-TUM DE-634 DE-19 DE-BY-UBM DE-11 |
| physical | XVI, 569 S graph. Darst. |
| publishDate | 2005 |
| publishDateSearch | 2005 |
| publishDateSort | 2005 |
| publisher | Oxford Univ. Press |
| record_format | marc |
| spelling | Coolen, Anthony C. C. Verfasser aut Theory of neural information processing systems A. C. C. Coolen, R. Kühn, P. Sollich 1. publ. Oxford [u.a.] Oxford Univ. Press 2005 XVI, 569 S graph. Darst. txt rdacontent n rdamedia nc rdacarrier Informatieverwerking (computer) gtt Zenuwstelsel gtt Neural networks (Computer science) Neuronales Netz (DE-588)4226127-2 gnd rswk-swf Neuronales Netz (DE-588)4226127-2 s DE-604 Kühn, Reimer Verfasser aut Sollich, Peter Verfasser aut HBZ Datenaustausch application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=013363632&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Coolen, Anthony C. C. Kühn, Reimer Sollich, Peter Theory of neural information processing systems Informatieverwerking (computer) gtt Zenuwstelsel gtt Neural networks (Computer science) Neuronales Netz (DE-588)4226127-2 gnd |
| subject_GND | (DE-588)4226127-2 |
| title | Theory of neural information processing systems |
| title_auth | Theory of neural information processing systems |
| title_exact_search | Theory of neural information processing systems |
| title_full | Theory of neural information processing systems A. C. C. Coolen, R. Kühn, P. Sollich |
| title_fullStr | Theory of neural information processing systems A. C. C. Coolen, R. Kühn, P. Sollich |
| title_full_unstemmed | Theory of neural information processing systems A. C. C. Coolen, R. Kühn, P. Sollich |
| title_short | Theory of neural information processing systems |
| title_sort | theory of neural information processing systems |
| topic | Informatieverwerking (computer) gtt Zenuwstelsel gtt Neural networks (Computer science) Neuronales Netz (DE-588)4226127-2 gnd |
| topic_facet | Informatieverwerking (computer) Zenuwstelsel Neural networks (Computer science) Neuronales Netz |
| url | http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=013363632&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| work_keys_str_mv | AT coolenanthonycc theoryofneuralinformationprocessingsystems AT kuhnreimer theoryofneuralinformationprocessingsystems AT sollichpeter theoryofneuralinformationprocessingsystems |