Mathematical aspects of deep learning:
In recent years the development of new classification and regression algorithms based on deep learning has led to a revolution in the fields of artificial intelligence, machine learning, and data analysis. The development of a theoretical foundation to guarantee the success of these algorithms const...
Saved in:
| Other Authors: | , |
|---|---|
| Format: | Book |
| Language: | English |
| Published: |
Cambridge
Cambridge University Press
2023
|
| Edition: | First published |
| Subjects: | |
| Online Access: | Inhaltsverzeichnis |
| Summary: | In recent years the development of new classification and regression algorithms based on deep learning has led to a revolution in the fields of artificial intelligence, machine learning, and data analysis. The development of a theoretical foundation to guarantee the success of these algorithms constitutes one of the most active and exciting research topics in applied mathematics. This book presents the current mathematical understanding of deep learning methods from the point of view of the leading experts in the field. It serves both as a starting point for researchers and graduate students in computer science, mathematics, and statistics trying to get into the field and as an invaluable reference for future research |
| Item Description: | Hier auch später erschienene, unveränderte Nachdrucke |
| Physical Description: | xviii, 473 Seiten Illustrationen, Diagramme |
| ISBN: | 9781316516782 |
Staff View
MARC
| LEADER | 00000nam a2200000zc 4500 | ||
|---|---|---|---|
| 001 | BV048685391 | ||
| 003 | DE-604 | ||
| 005 | 20240711 | ||
| 007 | t| | ||
| 008 | 230201s2023 xx a||| |||| 00||| eng d | ||
| 020 | |a 9781316516782 |9 978-1-316-51678-2 | ||
| 035 | |a (OCoLC)1371315696 | ||
| 035 | |a (DE-599)BVBBV048685391 | ||
| 040 | |a DE-604 |b ger |e rda | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-19 |a DE-83 |a DE-739 |a DE-M468 |a DE-91G |a DE-4325 | ||
| 082 | 0 | |a 006.31 | |
| 084 | |a ST 300 |0 (DE-625)143650: |2 rvk | ||
| 084 | |a ST 304 |0 (DE-625)143653: |2 rvk | ||
| 084 | |a DAT 708 |2 stub | ||
| 084 | |a 68T07 |2 msc | ||
| 245 | 1 | 0 | |a Mathematical aspects of deep learning |c edited by Philipp Grohs ; Universität Wien, Austria ; Gitta Kutyniok ; Ludwig-Maximilians-Universität München |
| 250 | |a First published | ||
| 264 | 1 | |a Cambridge |b Cambridge University Press |c 2023 | |
| 300 | |a xviii, 473 Seiten |b Illustrationen, Diagramme | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 500 | |a Hier auch später erschienene, unveränderte Nachdrucke | ||
| 520 | |a In recent years the development of new classification and regression algorithms based on deep learning has led to a revolution in the fields of artificial intelligence, machine learning, and data analysis. The development of a theoretical foundation to guarantee the success of these algorithms constitutes one of the most active and exciting research topics in applied mathematics. This book presents the current mathematical understanding of deep learning methods from the point of view of the leading experts in the field. It serves both as a starting point for researchers and graduate students in computer science, mathematics, and statistics trying to get into the field and as an invaluable reference for future research | ||
| 650 | 4 | |a Deep learning (Machine learning) | |
| 650 | 4 | |a Mathematics | |
| 650 | 0 | 7 | |a Deep Learning |0 (DE-588)1135597375 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Mathematische Methode |0 (DE-588)4155620-3 |2 gnd |9 rswk-swf |
| 689 | 0 | 0 | |a Deep Learning |0 (DE-588)1135597375 |D s |
| 689 | 0 | |5 DE-604 | |
| 689 | 1 | 0 | |a Mathematische Methode |0 (DE-588)4155620-3 |D s |
| 689 | 1 | |5 DE-604 | |
| 700 | 1 | |a Grohs, Philipp |d 1981- |0 (DE-588)1036788318 |4 edt | |
| 700 | 1 | |a Kutyniok, Gitta |d 1972- |0 (DE-588)1043849793 |4 edt | |
| 776 | 0 | 8 | |i Erscheint auch als |n Online-Ausgabe |z 978-1-00-902509-6 |
| 856 | 4 | 2 | |m Digitalisierung UB Passau - ADAM Catalogue Enrichment |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=034059685&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 943 | 1 | |a oai:aleph.bib-bvb.de:BVB01-034059685 | |
Record in the Search Index
| _version_ | 1824928748110086144 |
|---|---|
| adam_text |
Contents page xiii Contributors xv Preface 1 The Modern Mathematics of Deep Learning Julius Berner, Philipp Grohs, Gitta Kutyniok and Philipp Petersen 1.1 1.2 1.3 1.4 1.5 1.6 Introduction l.i.l Notation 1.1.2 Foundations of Learning Theory 1.1.3 Do We Need a New Theory? Generalization of Large Neural Networks 1.2.1 Kernel Regime 1.2.2 Norm-Based Bounds and Margin Theory 1.2.3 Optimization and Implicit Regularization 1.2.4 Limits of Classical Theory and Double Descent The Role of Depth in the Expressivity of NeuralNetworks 1.3.1 Approximation of Radial Functions 1.3.2 Deep ReLU Networks 1.3.3 Alternative Notions of Expressivity Deep Neural Networks Overcome the Curse of Dimensionality 1.4.1 Manifold Assumption 1.4.2 Random Sampling 1.4.3 PDE Assumption Optimization of Deep Neural Networks 1.5.1 Loss Landscape Analysis 1.5.2 Lazy Training and Provable Convergence of Stochas tic Gradient Descent Tangible Effects of Special Architectures 1.6.1 Convolutional Neural Networks v I I 4 5 23 31 31 33 35 38 41 41 44 47 49 49 51 53 57 57 61 65 66
Contents vi 1.7 1.8 2 1.6.2 Residual Neural Networks 1.6.3 Framelets and U-Nets 1.6.4 Batch Normalization 1.6.5 Sparse Neural Networks and Pruning 1.6.6 Recurrent Neural Networks Describing the Features that a Deep Neural Network Learns 1.7.1 Invariances and the Scattering Transform 1.7.2 Hierarchical Sparse Representations Effectiveness in Natural Sciences 1.8.1 Deep Neural Networks Meet InverseProblems 1.8.2 PDE-Based Models Generalization in Deep Learning K. Kawaguchi, У. Bengio, and L. Kaelbling Introduction Background Rethinking Generalization 2.3.1 Consistency of Theory 2.3.2 Differences in Assumptions and Problem Settings 2.3.3 Practical Role of Generalization Theory 2.4 Generalization Bounds via Validation 2.5 Direct Analyses of Neural Networks 2.5.1 Model Description via Deep Paths 2.5.2 Theoretical Insights via Tight Theory for Every Pair (P,5) 125 2.5.3 Probabilistic Bounds over Random Datasets 2.5.4 Probabilistic Bound for 0-1 Loss with Multi-Labels 2.6 Discussions and Open Problems Appendix A Additional Discussions Al Simple Regularization Algorithm A2 Relationship to Other Fields A3 SGD Chooses Direction in Terms of w A4 Simple Implementation of Two-PhaseTraining Procedure A5 On Proposition 2.3 A6 On Extensions Appendix В Experimental Details AppendixC Proofs Cl Proof of Theorem 2.1 C2 Proof of Corollary 2.2 2.1 2.2 2.3 68 70 73 75 76 78 78 79 81 82 84 112 112 113 116 118 119 121 121 122 123 127 130 131 133 133 135 135 136 136 137 137 138 139 139
Contents СЗ C4 C5 C6 3 4 Proof of Theorem 2.7 Proof of Theorem 2.9 Proof of Theorem 2.10 Proof of Proposition 2.5 Expressivity of Deep Neural Networks Ingo Giihring, Mones Rasian, and Gitta Kutyniok 3.1 Introduction 3.1.1 Neural Networks 3.1.2 Goal and Outline of this Chapter 3.1.3 Notation 3.2 Shallow Neural Networks 3.2.1 Universality of Shallow NeuralNetworks 3.2.2 Lower Complexity Bounds 3.2.3 Upper Complexity Bounds 3.3 Universality of Deep Neural Networks 3.4 Approximation of Classes of Smooth Functions 3.5 Approximation of Piecewise Smooth Functions 3.6 Assuming More Structure 3.6.1 Hierachical Structure 3.6.2 Assumptions on the Data Manifold 3.6.3 Expressivity of Deep Neural Networks for Solutions ofPDEs 175 3.7 Deep Versus Shallow Neural Networks 3.8 Special Neural Network Architectures and Activation Functions 3.8.1 Convolutional Neural Networks 3.8.2 Residual Neural Networks 3.8.3 Recurrent Neural Networks Optimization Landscape of Neural Networks René Vidal, Zhihui Zhu, and Benjamin D. Haeffele 4.1 Introduction 4.2 Basics of Statistical Learning 4.3 Optimization Landscape of Linear Networks 4.3.1 Single-Hidden-Layer Linear Networks with Squared Loss and Fixed Size Regularization 4.3.2 Deep Linear Networks with Squared Loss 4.4 Optimization Landscape of Nonlinear Networks 4.4.1 Motivating Example 4.4.2 Positively Homogeneous Networks 4.5 Conclusions vii 140 141 142 143 149 149 151 154 154 155 156 159 160 161 163 167 172 172 174 177 180 180 184 185 200 201 205 206 207 212 214 215 221 225
viii 5 6 Contents Explaining the Decisions of Convolutional and Recurrent Neural Networks Wojciech Samek, Leila Arras, Ahmed Osman, Grégoire Montavon, Klaus-Robert Müller 5.1 Introduction 5.2 Why Explainability? 5.2.1 Practical Advantages of Expkunability 5.2.2 Social and Legal Role of Explainability 5.2.3 Theoretical Insights Through Explainability 5.3 From Explaining Linear Models to General Model Explain ability 5.3.1 Explainability of Linear Models 5.3.2 Generalizing Explainability to Nonlinear Models 5.3.3 Short Survey on Explanation Methods 5.4 Layer-Wise Relevance Propagation 5.4.1 LRP in Convolutional Neural Networks 5.4.2 Theoretical Interpretation of the LRP Redistribution Process 5.4.3 Extending LRP to LSTM Networks 5.5 Explaining a Visual Question Answering Model 5.6 Discussion 229 229 231 231 232 232 233 233 235 236 238 239 242 248 251 258 Stochastic Feedforward Neural Networks: Universal Approxima tion Thomas Merkh and Guido Montúfar 267 6.1 Introduction 268 6.2 Overview of Previous Works and Results 271 6.3 Markov Kernels and Stochastic Networks 273 6.3.1 Binary Probability Distributions and MarkovKernels 273 6.3.2 Stochastic Feedforward Networks 274 6.4 Results for Shallow Networks 276 6.4.1 Fixed Weights in the Output Layer 277 6.4.2 Trainable Weights in the Output Layer 278 6.5 Proofs for Shallow Networks 278 6.5.1 Fixed Weights in the Output Layer 279 6.5.2 Trainable Weights in the Second Layer 283 6.5.3 Discussion of the Proofs for Shallow Networks 285 6.6 Results for Deep Networks 286 6.6.1 Parameter Count 288 6.6.2 Approximation with Finite Weights
and Biases 288
Contents Proofs for Deep Networks 6.7.1 Notation 6.7.2 Probability Mass Sharing 6.7.3 Universal Approximation 6.7.4 Error Analysis for Finite Weights and Biases 6.7.5 Discussion of the Proofs for Deep Networks 6.8 Lower Bounds for Shallow and Deep Networks 6.8.1 Parameter Counting Lower Bounds 6.8.2 Minimum Width 6.9 A Numerical Example 6.10 Conclusion 6.11 Open Problems 289 289 290 293 296 298 299 299 301 302 306 307 Deep Learning as Sparsity-Enforcing Algorithms A. Aberdam and J. Sulam 7.1 Introduction 7.2 Related Work 7.3 Background 7.4 Multilayer Sparse Coding 7.4.1 ML-SC Pursuit and the Forward Pass 7.4.2 ML-SC: A Projection Approach 7.5 The Holistic Way 7.6 Multilayer Iterative Shrinkage Algorithms 7.6.1 Towards Principled Recurrent Neural Networks 7.7 Final Remarks and Outlook 314 314 316 317 320 321 323 324 327 329 332 6.7 7 8 ix The Scattering Transform Joan Bruna 8.1 Introduction 8.2 Geometric Stability 8.2.1 Euclidean Geometric Stability 8.2.2 Representations with Euclidean Geometric Stability 8.2.3 Non-Euclidean Geometric Stability 8.2.4 Examples 8.3 Scattering on the Translation Group 8.3.1 Windowed Scattering Transform 8.3.2 Scattering Metric and Energy Conservation 8.3.3 Local Translation Invariance and Lipschitz Continu ity with Respect to Deformations 351 8.3.4 Algorithms 8.3.5 Empirical Analysis of Scattering Properties 338 338 339 340 341 342 343 346 346 349 354 357
Contents x 8.3.6 Scattering in Modern Computer Vision Scattering Representations of Stochastic Processes 8.4.1 Expected Scattering 8.4.2 Analysis of Stationary Textures with Scattering 8.4.3 Multifractal Analysis with Scattering Moments Non-Euclidean Scattering 8.5.1 Joint versus Separable Scattering 8.5.2 Scattering on Global Symmetry Groups 8.5.3 Graph Scattering 8.5.4 Manifold Scattering Generative Modeling with Scattering 8.6.1 Sufficient Statistics 8.6.2 Microcanonical Scattering Models 8.6.3 Gradient Descent Scattering Reconstruction 8.6.4 Regularising Inverse Problems with Scattering 8.6.5 Texture Synthesis with Microcanonical Scattering Final Remarks 362 363 363 367 369 371 372 372 375 383 384 384 385 387 389 391 393 Deep Generative Models and Inverse Problems Alexandros G. Dimakis 9.1 Introduction 9.2 How to Tame High Dimensions 9.2.1 Sparsity 9.2.2 Conditional Independence 9.2.3 Deep Generative Models 9.2.4 GANs and VAEs 9.2.5 Invertible Generative Models 9.2.6 Untrained Generative Models 9.3 Linear Inverse Problems Using Deep Generative Models 9.3.1 Reconstruction from Gaussian Measurements 9.3.2 Optimization Challenges 9.3.3 Extending the Range of the Generator 9.3.4 Non-Linear Inverse Problems 9.3.5 Inverse Problems with UntrainedGenerative Priors 9.4 Supervised Methods for Inverse Problems 400 400 401 401 402 403 404 405 405 406 407 409 410 410 412 414 Dynamical Systems and Optimal Control Approach to Deep Learn ing Weinan E, Jiequn Han, and Qianxiao Li 10.1 Introduction 10.1.1 The Problem of Supervised Learning 422 422 423 8.4 8.5 8.6 8.7 9 10
Contents 10.2 10.3 ODE Formulation Mean-Field Optimal Control and Pontryagin’s Maximum Principle 10.3.1 Pontryagin’s Maximum Principle 10.4 Method of Successive Approximations 10.4.1 Extended Pontryagin Maximum Principle 10.4.2 The Basic Method of Successive Approximation 10.4.3 Extended Method of Successive Approximation 10.4.4 Discrete PMP and Discrete MSA 10.5 Future Work 11 xi 424 425 426 428 428 428 431 433 435 Bridging Many-Body Quantum Physics and Deep Learning via Tensor Networks Yoav Levine, Or Sharir, Nadav Cohen and Amnon Shashua 439 11.1 Introduction 440 11.2 Preliminaries ֊ Many-Body Quantum Physics 442 11.2.1 The Many-Body Quantum Wave Function 443 11.2.2 Quantum Entanglement Measures 444 11.2.3 Tensor Networks 447 I 1.3 Quantum Wave Functions and Deep Learning Architectures 450 11.3.1 Convolutional and Recurrent Networks as Wave Functions 450 11.3.2 Tensor Network Representations of Convolutional and Recurrent Networks 453 11.4 Deep Learning Architecture Design via Entanglement Measures 453 11.4.1 Dependencies via Entanglement Measures 454 11.4.2 Quantum-Physics-Inspired Control of Inductive Bias 456 11.5 Power of Deep Learning for Wave Function Representations 460 11.5.1 Entanglement Scaling of Deep Recurrent Networks 461 11.5.2 Entanglement Scaling of Overlapping Convolutional Networks 463 11.6 Discussion 467 |
| adam_txt |
Contents page xiii Contributors xv Preface 1 The Modern Mathematics of Deep Learning Julius Berner, Philipp Grohs, Gitta Kutyniok and Philipp Petersen 1.1 1.2 1.3 1.4 1.5 1.6 Introduction l.i.l Notation 1.1.2 Foundations of Learning Theory 1.1.3 Do We Need a New Theory? Generalization of Large Neural Networks 1.2.1 Kernel Regime 1.2.2 Norm-Based Bounds and Margin Theory 1.2.3 Optimization and Implicit Regularization 1.2.4 Limits of Classical Theory and Double Descent The Role of Depth in the Expressivity of NeuralNetworks 1.3.1 Approximation of Radial Functions 1.3.2 Deep ReLU Networks 1.3.3 Alternative Notions of Expressivity Deep Neural Networks Overcome the Curse of Dimensionality 1.4.1 Manifold Assumption 1.4.2 Random Sampling 1.4.3 PDE Assumption Optimization of Deep Neural Networks 1.5.1 Loss Landscape Analysis 1.5.2 Lazy Training and Provable Convergence of Stochas tic Gradient Descent Tangible Effects of Special Architectures 1.6.1 Convolutional Neural Networks v I I 4 5 23 31 31 33 35 38 41 41 44 47 49 49 51 53 57 57 61 65 66
Contents vi 1.7 1.8 2 1.6.2 Residual Neural Networks 1.6.3 Framelets and U-Nets 1.6.4 Batch Normalization 1.6.5 Sparse Neural Networks and Pruning 1.6.6 Recurrent Neural Networks Describing the Features that a Deep Neural Network Learns 1.7.1 Invariances and the Scattering Transform 1.7.2 Hierarchical Sparse Representations Effectiveness in Natural Sciences 1.8.1 Deep Neural Networks Meet InverseProblems 1.8.2 PDE-Based Models Generalization in Deep Learning K. Kawaguchi, У. Bengio, and L. Kaelbling Introduction Background Rethinking Generalization 2.3.1 Consistency of Theory 2.3.2 Differences in Assumptions and Problem Settings 2.3.3 Practical Role of Generalization Theory 2.4 Generalization Bounds via Validation 2.5 Direct Analyses of Neural Networks 2.5.1 Model Description via Deep Paths 2.5.2 Theoretical Insights via Tight Theory for Every Pair (P,5) 125 2.5.3 Probabilistic Bounds over Random Datasets 2.5.4 Probabilistic Bound for 0-1 Loss with Multi-Labels 2.6 Discussions and Open Problems Appendix A Additional Discussions Al Simple Regularization Algorithm A2 Relationship to Other Fields A3 SGD Chooses Direction in Terms of w A4 Simple Implementation of Two-PhaseTraining Procedure A5 On Proposition 2.3 A6 On Extensions Appendix В Experimental Details AppendixC Proofs Cl Proof of Theorem 2.1 C2 Proof of Corollary 2.2 2.1 2.2 2.3 68 70 73 75 76 78 78 79 81 82 84 112 112 113 116 118 119 121 121 122 123 127 130 131 133 133 135 135 136 136 137 137 138 139 139
Contents СЗ C4 C5 C6 3 4 Proof of Theorem 2.7 Proof of Theorem 2.9 Proof of Theorem 2.10 Proof of Proposition 2.5 Expressivity of Deep Neural Networks Ingo Giihring, Mones Rasian, and Gitta Kutyniok 3.1 Introduction 3.1.1 Neural Networks 3.1.2 Goal and Outline of this Chapter 3.1.3 Notation 3.2 Shallow Neural Networks 3.2.1 Universality of Shallow NeuralNetworks 3.2.2 Lower Complexity Bounds 3.2.3 Upper Complexity Bounds 3.3 Universality of Deep Neural Networks 3.4 Approximation of Classes of Smooth Functions 3.5 Approximation of Piecewise Smooth Functions 3.6 Assuming More Structure 3.6.1 Hierachical Structure 3.6.2 Assumptions on the Data Manifold 3.6.3 Expressivity of Deep Neural Networks for Solutions ofPDEs 175 3.7 Deep Versus Shallow Neural Networks 3.8 Special Neural Network Architectures and Activation Functions 3.8.1 Convolutional Neural Networks 3.8.2 Residual Neural Networks 3.8.3 Recurrent Neural Networks Optimization Landscape of Neural Networks René Vidal, Zhihui Zhu, and Benjamin D. Haeffele 4.1 Introduction 4.2 Basics of Statistical Learning 4.3 Optimization Landscape of Linear Networks 4.3.1 Single-Hidden-Layer Linear Networks with Squared Loss and Fixed Size Regularization 4.3.2 Deep Linear Networks with Squared Loss 4.4 Optimization Landscape of Nonlinear Networks 4.4.1 Motivating Example 4.4.2 Positively Homogeneous Networks 4.5 Conclusions vii 140 141 142 143 149 149 151 154 154 155 156 159 160 161 163 167 172 172 174 177 180 180 184 185 200 201 205 206 207 212 214 215 221 225
viii 5 6 Contents Explaining the Decisions of Convolutional and Recurrent Neural Networks Wojciech Samek, Leila Arras, Ahmed Osman, Grégoire Montavon, Klaus-Robert Müller 5.1 Introduction 5.2 Why Explainability? 5.2.1 Practical Advantages of Expkunability 5.2.2 Social and Legal Role of Explainability 5.2.3 Theoretical Insights Through Explainability 5.3 From Explaining Linear Models to General Model Explain ability 5.3.1 Explainability of Linear Models 5.3.2 Generalizing Explainability to Nonlinear Models 5.3.3 Short Survey on Explanation Methods 5.4 Layer-Wise Relevance Propagation 5.4.1 LRP in Convolutional Neural Networks 5.4.2 Theoretical Interpretation of the LRP Redistribution Process 5.4.3 Extending LRP to LSTM Networks 5.5 Explaining a Visual Question Answering Model 5.6 Discussion 229 229 231 231 232 232 233 233 235 236 238 239 242 248 251 258 Stochastic Feedforward Neural Networks: Universal Approxima tion Thomas Merkh and Guido Montúfar 267 6.1 Introduction 268 6.2 Overview of Previous Works and Results 271 6.3 Markov Kernels and Stochastic Networks 273 6.3.1 Binary Probability Distributions and MarkovKernels 273 6.3.2 Stochastic Feedforward Networks 274 6.4 Results for Shallow Networks 276 6.4.1 Fixed Weights in the Output Layer 277 6.4.2 Trainable Weights in the Output Layer 278 6.5 Proofs for Shallow Networks 278 6.5.1 Fixed Weights in the Output Layer 279 6.5.2 Trainable Weights in the Second Layer 283 6.5.3 Discussion of the Proofs for Shallow Networks 285 6.6 Results for Deep Networks 286 6.6.1 Parameter Count 288 6.6.2 Approximation with Finite Weights
and Biases 288
Contents Proofs for Deep Networks 6.7.1 Notation 6.7.2 Probability Mass Sharing 6.7.3 Universal Approximation 6.7.4 Error Analysis for Finite Weights and Biases 6.7.5 Discussion of the Proofs for Deep Networks 6.8 Lower Bounds for Shallow and Deep Networks 6.8.1 Parameter Counting Lower Bounds 6.8.2 Minimum Width 6.9 A Numerical Example 6.10 Conclusion 6.11 Open Problems 289 289 290 293 296 298 299 299 301 302 306 307 Deep Learning as Sparsity-Enforcing Algorithms A. Aberdam and J. Sulam 7.1 Introduction 7.2 Related Work 7.3 Background 7.4 Multilayer Sparse Coding 7.4.1 ML-SC Pursuit and the Forward Pass 7.4.2 ML-SC: A Projection Approach 7.5 The Holistic Way 7.6 Multilayer Iterative Shrinkage Algorithms 7.6.1 Towards Principled Recurrent Neural Networks 7.7 Final Remarks and Outlook 314 314 316 317 320 321 323 324 327 329 332 6.7 7 8 ix The Scattering Transform Joan Bruna 8.1 Introduction 8.2 Geometric Stability 8.2.1 Euclidean Geometric Stability 8.2.2 Representations with Euclidean Geometric Stability 8.2.3 Non-Euclidean Geometric Stability 8.2.4 Examples 8.3 Scattering on the Translation Group 8.3.1 Windowed Scattering Transform 8.3.2 Scattering Metric and Energy Conservation 8.3.3 Local Translation Invariance and Lipschitz Continu ity with Respect to Deformations 351 8.3.4 Algorithms 8.3.5 Empirical Analysis of Scattering Properties 338 338 339 340 341 342 343 346 346 349 354 357
Contents x 8.3.6 Scattering in Modern Computer Vision Scattering Representations of Stochastic Processes 8.4.1 Expected Scattering 8.4.2 Analysis of Stationary Textures with Scattering 8.4.3 Multifractal Analysis with Scattering Moments Non-Euclidean Scattering 8.5.1 Joint versus Separable Scattering 8.5.2 Scattering on Global Symmetry Groups 8.5.3 Graph Scattering 8.5.4 Manifold Scattering Generative Modeling with Scattering 8.6.1 Sufficient Statistics 8.6.2 Microcanonical Scattering Models 8.6.3 Gradient Descent Scattering Reconstruction 8.6.4 Regularising Inverse Problems with Scattering 8.6.5 Texture Synthesis with Microcanonical Scattering Final Remarks 362 363 363 367 369 371 372 372 375 383 384 384 385 387 389 391 393 Deep Generative Models and Inverse Problems Alexandros G. Dimakis 9.1 Introduction 9.2 How to Tame High Dimensions 9.2.1 Sparsity 9.2.2 Conditional Independence 9.2.3 Deep Generative Models 9.2.4 GANs and VAEs 9.2.5 Invertible Generative Models 9.2.6 Untrained Generative Models 9.3 Linear Inverse Problems Using Deep Generative Models 9.3.1 Reconstruction from Gaussian Measurements 9.3.2 Optimization Challenges 9.3.3 Extending the Range of the Generator 9.3.4 Non-Linear Inverse Problems 9.3.5 Inverse Problems with UntrainedGenerative Priors 9.4 Supervised Methods for Inverse Problems 400 400 401 401 402 403 404 405 405 406 407 409 410 410 412 414 Dynamical Systems and Optimal Control Approach to Deep Learn ing Weinan E, Jiequn Han, and Qianxiao Li 10.1 Introduction 10.1.1 The Problem of Supervised Learning 422 422 423 8.4 8.5 8.6 8.7 9 10
Contents 10.2 10.3 ODE Formulation Mean-Field Optimal Control and Pontryagin’s Maximum Principle 10.3.1 Pontryagin’s Maximum Principle 10.4 Method of Successive Approximations 10.4.1 Extended Pontryagin Maximum Principle 10.4.2 The Basic Method of Successive Approximation 10.4.3 Extended Method of Successive Approximation 10.4.4 Discrete PMP and Discrete MSA 10.5 Future Work 11 xi 424 425 426 428 428 428 431 433 435 Bridging Many-Body Quantum Physics and Deep Learning via Tensor Networks Yoav Levine, Or Sharir, Nadav Cohen and Amnon Shashua 439 11.1 Introduction 440 11.2 Preliminaries ֊ Many-Body Quantum Physics 442 11.2.1 The Many-Body Quantum Wave Function 443 11.2.2 Quantum Entanglement Measures 444 11.2.3 Tensor Networks 447 I 1.3 Quantum Wave Functions and Deep Learning Architectures 450 11.3.1 Convolutional and Recurrent Networks as Wave Functions 450 11.3.2 Tensor Network Representations of Convolutional and Recurrent Networks 453 11.4 Deep Learning Architecture Design via Entanglement Measures 453 11.4.1 Dependencies via Entanglement Measures 454 11.4.2 Quantum-Physics-Inspired Control of Inductive Bias 456 11.5 Power of Deep Learning for Wave Function Representations 460 11.5.1 Entanglement Scaling of Deep Recurrent Networks 461 11.5.2 Entanglement Scaling of Overlapping Convolutional Networks 463 11.6 Discussion 467 |
| any_adam_object | 1 |
| any_adam_object_boolean | 1 |
| author2 | Grohs, Philipp 1981- Kutyniok, Gitta 1972- |
| author2_role | edt edt |
| author2_variant | p g pg g k gk |
| author_GND | (DE-588)1036788318 (DE-588)1043849793 |
| author_facet | Grohs, Philipp 1981- Kutyniok, Gitta 1972- |
| building | Verbundindex |
| bvnumber | BV048685391 |
| classification_rvk | ST 300 ST 304 |
| classification_tum | DAT 708 |
| ctrlnum | (OCoLC)1371315696 (DE-599)BVBBV048685391 |
| dewey-full | 006.31 |
| dewey-hundreds | 000 - Computer science, information, general works |
| dewey-ones | 006 - Special computer methods |
| dewey-raw | 006.31 |
| dewey-search | 006.31 |
| dewey-sort | 16.31 |
| dewey-tens | 000 - Computer science, information, general works |
| discipline | Informatik |
| discipline_str_mv | Informatik |
| edition | First published |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>00000nam a2200000zc 4500</leader><controlfield tag="001">BV048685391</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20240711</controlfield><controlfield tag="007">t|</controlfield><controlfield tag="008">230201s2023 xx a||| |||| 00||| eng d</controlfield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9781316516782</subfield><subfield code="9">978-1-316-51678-2</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)1371315696</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BVBBV048685391</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">rda</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-19</subfield><subfield code="a">DE-83</subfield><subfield code="a">DE-739</subfield><subfield code="a">DE-M468</subfield><subfield code="a">DE-91G</subfield><subfield code="a">DE-4325</subfield></datafield><datafield tag="082" ind1="0" ind2=" "><subfield code="a">006.31</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 300</subfield><subfield code="0">(DE-625)143650:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 304</subfield><subfield code="0">(DE-625)143653:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">DAT 708</subfield><subfield code="2">stub</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">68T07</subfield><subfield code="2">msc</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Mathematical aspects of deep learning</subfield><subfield code="c">edited by Philipp Grohs ; Universität Wien, Austria ; Gitta Kutyniok ; Ludwig-Maximilians-Universität München</subfield></datafield><datafield tag="250" ind1=" " ind2=" "><subfield code="a">First published</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">Cambridge</subfield><subfield code="b">Cambridge University Press</subfield><subfield code="c">2023</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">xviii, 473 Seiten</subfield><subfield code="b">Illustrationen, Diagramme</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="500" ind1=" " ind2=" "><subfield code="a">Hier auch später erschienene, unveränderte Nachdrucke</subfield></datafield><datafield tag="520" ind1=" " ind2=" "><subfield code="a">In recent years the development of new classification and regression algorithms based on deep learning has led to a revolution in the fields of artificial intelligence, machine learning, and data analysis. The development of a theoretical foundation to guarantee the success of these algorithms constitutes one of the most active and exciting research topics in applied mathematics. This book presents the current mathematical understanding of deep learning methods from the point of view of the leading experts in the field. It serves both as a starting point for researchers and graduate students in computer science, mathematics, and statistics trying to get into the field and as an invaluable reference for future research</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Deep learning (Machine learning)</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Mathematics</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Deep Learning</subfield><subfield code="0">(DE-588)1135597375</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Mathematische Methode</subfield><subfield code="0">(DE-588)4155620-3</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Deep Learning</subfield><subfield code="0">(DE-588)1135597375</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="689" ind1="1" ind2="0"><subfield code="a">Mathematische Methode</subfield><subfield code="0">(DE-588)4155620-3</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="1" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Grohs, Philipp</subfield><subfield code="d">1981-</subfield><subfield code="0">(DE-588)1036788318</subfield><subfield code="4">edt</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Kutyniok, Gitta</subfield><subfield code="d">1972-</subfield><subfield code="0">(DE-588)1043849793</subfield><subfield code="4">edt</subfield></datafield><datafield tag="776" ind1="0" ind2="8"><subfield code="i">Erscheint auch als</subfield><subfield code="n">Online-Ausgabe</subfield><subfield code="z">978-1-00-902509-6</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">Digitalisierung UB Passau - ADAM Catalogue Enrichment</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=034059685&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="943" ind1="1" ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-034059685</subfield></datafield></record></collection> |
| id | DE-604.BV048685391 |
| illustrated | Illustrated |
| index_date | 2024-07-03T21:26:23Z |
| indexdate | 2025-02-24T09:01:15Z |
| institution | BVB |
| isbn | 9781316516782 |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-034059685 |
| oclc_num | 1371315696 |
| open_access_boolean | |
| owner | DE-19 DE-BY-UBM DE-83 DE-739 DE-M468 DE-91G DE-BY-TUM DE-4325 |
| owner_facet | DE-19 DE-BY-UBM DE-83 DE-739 DE-M468 DE-91G DE-BY-TUM DE-4325 |
| physical | xviii, 473 Seiten Illustrationen, Diagramme |
| publishDate | 2023 |
| publishDateSearch | 2023 |
| publishDateSort | 2023 |
| publisher | Cambridge University Press |
| record_format | marc |
| spelling | Mathematical aspects of deep learning edited by Philipp Grohs ; Universität Wien, Austria ; Gitta Kutyniok ; Ludwig-Maximilians-Universität München First published Cambridge Cambridge University Press 2023 xviii, 473 Seiten Illustrationen, Diagramme txt rdacontent n rdamedia nc rdacarrier Hier auch später erschienene, unveränderte Nachdrucke In recent years the development of new classification and regression algorithms based on deep learning has led to a revolution in the fields of artificial intelligence, machine learning, and data analysis. The development of a theoretical foundation to guarantee the success of these algorithms constitutes one of the most active and exciting research topics in applied mathematics. This book presents the current mathematical understanding of deep learning methods from the point of view of the leading experts in the field. It serves both as a starting point for researchers and graduate students in computer science, mathematics, and statistics trying to get into the field and as an invaluable reference for future research Deep learning (Machine learning) Mathematics Deep Learning (DE-588)1135597375 gnd rswk-swf Mathematische Methode (DE-588)4155620-3 gnd rswk-swf Deep Learning (DE-588)1135597375 s DE-604 Mathematische Methode (DE-588)4155620-3 s Grohs, Philipp 1981- (DE-588)1036788318 edt Kutyniok, Gitta 1972- (DE-588)1043849793 edt Erscheint auch als Online-Ausgabe 978-1-00-902509-6 Digitalisierung UB Passau - ADAM Catalogue Enrichment application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=034059685&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Mathematical aspects of deep learning Deep learning (Machine learning) Mathematics Deep Learning (DE-588)1135597375 gnd Mathematische Methode (DE-588)4155620-3 gnd |
| subject_GND | (DE-588)1135597375 (DE-588)4155620-3 |
| title | Mathematical aspects of deep learning |
| title_auth | Mathematical aspects of deep learning |
| title_exact_search | Mathematical aspects of deep learning |
| title_exact_search_txtP | Mathematical aspects of deep learning |
| title_full | Mathematical aspects of deep learning edited by Philipp Grohs ; Universität Wien, Austria ; Gitta Kutyniok ; Ludwig-Maximilians-Universität München |
| title_fullStr | Mathematical aspects of deep learning edited by Philipp Grohs ; Universität Wien, Austria ; Gitta Kutyniok ; Ludwig-Maximilians-Universität München |
| title_full_unstemmed | Mathematical aspects of deep learning edited by Philipp Grohs ; Universität Wien, Austria ; Gitta Kutyniok ; Ludwig-Maximilians-Universität München |
| title_short | Mathematical aspects of deep learning |
| title_sort | mathematical aspects of deep learning |
| topic | Deep learning (Machine learning) Mathematics Deep Learning (DE-588)1135597375 gnd Mathematische Methode (DE-588)4155620-3 gnd |
| topic_facet | Deep learning (Machine learning) Mathematics Deep Learning Mathematische Methode |
| url | http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=034059685&sequence=000001&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| work_keys_str_mv | AT grohsphilipp mathematicalaspectsofdeeplearning AT kutyniokgitta mathematicalaspectsofdeeplearning |