Approximate dynamic programming: solving the curses of dimensionality
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Buch |
| Sprache: | English |
| Veröffentlicht: |
Hoboken, New Jersey
Wiley
[2011]
|
| Ausgabe: | Second edition |
| Schriftenreihe: | Wiley series in probability and statistics
|
| Schlagworte: | |
| Online-Zugang: | Inhaltsverzeichnis |
| Beschreibung: | Includes bibliographical references and index |
| Beschreibung: | XVIII, 627 Seiten Illustrationen, Diagramme |
| ISBN: | 047060445X 9780470604458 9781118029176 |
Internformat
MARC
| LEADER | 00000nam a2200000 c 4500 | ||
|---|---|---|---|
| 001 | BV039835544 | ||
| 003 | DE-604 | ||
| 005 | 20190218 | ||
| 007 | t | ||
| 008 | 120124s2011 a||| |||| 00||| eng d | ||
| 020 | |a 047060445X |c (hbk.) £83.50 |9 0-470-60445-X | ||
| 020 | |a 9780470604458 |c (hbk.) £83.50 |9 978-0-470-60445-8 | ||
| 020 | |a 9781118029176 |c oBook |9 978-1-118-02917-6 | ||
| 035 | |a (OCoLC)681536496 | ||
| 035 | |a (DE-599)BSZ353876828 | ||
| 040 | |a DE-604 |b ger |e rda | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-11 |a DE-384 |a DE-91 |a DE-20 |a DE-706 |a DE-945 |a DE-521 |a DE-19 |a DE-739 | ||
| 082 | 0 | |a 519.7/03 |2 22 | |
| 084 | |a QH 423 |0 (DE-625)141577: |2 rvk | ||
| 084 | |a SK 880 |0 (DE-625)143266: |2 rvk | ||
| 084 | |a SK 970 |0 (DE-625)143276: |2 rvk | ||
| 084 | |a ST 230 |0 (DE-625)143617: |2 rvk | ||
| 084 | |a DAT 917f |2 stub | ||
| 100 | 1 | |a Powell, Warren B. |d 1955- |e Verfasser |0 (DE-588)170298175 |4 aut | |
| 245 | 1 | 0 | |a Approximate dynamic programming |b solving the curses of dimensionality |c Warren B. Powell (Princeton University, The Department of operations research and financial engineering, Princeton, NJ) |
| 250 | |a Second edition | ||
| 264 | 1 | |a Hoboken, New Jersey |b Wiley |c [2011] | |
| 300 | |a XVIII, 627 Seiten |b Illustrationen, Diagramme | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 490 | 0 | |a Wiley series in probability and statistics | |
| 500 | |a Includes bibliographical references and index | ||
| 650 | 0 | 7 | |a Dynamische Optimierung |0 (DE-588)4125677-3 |2 gnd |9 rswk-swf |
| 689 | 0 | 0 | |a Dynamische Optimierung |0 (DE-588)4125677-3 |D s |
| 689 | 0 | |5 DE-604 | |
| 776 | 0 | 8 | |i Erscheint auch als |n Online-Ausgabe, EPUB |z 978-1-118-02916-9 |
| 776 | 0 | 8 | |i Erscheint auch als |n Online-Ausgabe, PDF |z 978-1-118-02915-2 |
| 856 | 4 | 2 | |m HBZ Datenaustausch |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=024695545&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 999 | |a oai:aleph.bib-bvb.de:BVB01-024695545 | ||
Datensatz im Suchindex
| _version_ | 1804148765345447936 |
|---|---|
| adam_text | Titel: Approximate dynamic programming
Autor: Powell, Warren B
Jahr: 2011
Contents Preface to the Second Edition xi Preface to the First Edition xv Acknowledgments xvii The Challenges of Dynamic Programming 1 2 1.1 A Dynamic Programming Example: A Shortest Path Problem, 2 1.2 The Three Curses of Dimensionality, 3 1.3 Some Real Applications, 6 1.4 Problem Classes, 11 1.5 The Many Dialects of Dynamic Programming, 15 1.6 What Is New in This Book?, 17 1.7 Pedagogy, 19 1.8 Bibliographic Notes, 22 Some Illustrative Models 25 2.1 Deterministic Problems, 26 2.2 Stochastic Problems, 31 2.3 Information Acquisition Problems, 47 2.4 A Simple Modeling Framework for Dynamic Programs, 50 2.5 Bibliographic Notes, 54 Problems, 54 Introduction to Markov Decision Processes 57 3.1 The Optimality Equations, 58 3.2 Finite Horizon Problems, 65 v
vi CONTENTS 3.3 Infinite Horizon Problems, 66 3.4 Value Iteration, 68 3.5 Policy Iteration, 74 3.6 Hybrid Value-Policy Iteration, 75 3.7 Average Reward Dynamic Programming, 76 3.8 The Linear Programming Method for Dynamic Programs, 77 3.9 Monotone Policies*, 78 3.10 Why Does It Work?**, 84 3.11 Bibliographic Notes, 103 Problems, 103 4 Introduction to Approximate Dynamic Programming 111 4.1 The Three Curses of Dimensionality (Revisited), 112 4.2 The Basic Idea, 114 4.3 Q-Learning and SARSA, 122 4.4 Real-Time Dynamic Programming, 126 4.5 Approximate Value Iteration, 127 4.6 The Post-Decision State Variable, 129 4.7 Low-Dimensional Representations of Value Functions, 144 4.8 So Just What Is Approximate Dynamic Programming?, 146 4.9 Experimental Issues, 149 4.10 But Does It Work?, 155 4.11 Bibliographic Notes, 156 Problems, 158 5 Modeling Dynamic Programs 167 5.1 Notational Style, 169 5.2 Modeling Time, 170 5.3 Modeling Resources, 174 5.4 The States of Our System, 178 5.5 Modeling Decisions, 187 5.6 The Exogenous Information Process, 189 5.7 The Transition Function, 198 5.8 The Objective Function, 206 5.9 A Measure-Theoretic View of Information**, 211 5.10 Bibliographic Notes, 213 Problems, 214
CONTENTS vu 6 Policies 221 6.1 Myopic Policies, 224 6.2 Lookahead Policies, 224 6.3 Policy Function Approximations, 232 6.4 Value Function Approximations, 235 6.5 Hybrid Strategies, 239 6.6 Randomized Policies, 242 6.7 How to Choose a Policy?, 244 6.8 Bibliographic Notes, 247 Problems, 247 7 Policy Search 249 7.1 Background, 250 7.2 Gradient Search, 253 7.3 Direct Policy Search for Finite Alternatives, 256 7.4 The Knowledge Gradient Algorithm for Discrete Alternatives, 262 7.5 Simulation Optimization, 270 7.6 Why Does It Work?**, 274 7.7 Bibliographic Notes, 285 Problems, 286 8 Approximating Value Functions 289 8.1 Lookup Tables and Aggregation, 290 8.2 Parametric Models, 304 8.3 Regression Variations, 314 8.4 Nonparametric Models, 316 8.5 Approximations and the Curse of Dimensionality, 325 8.6 Why Does It Work?**, 328 8.7 Bibliographic Notes, 333 Problems, 334 9 Learning Value Function Approximations 337 9.1 Sampling the Value of a Policy, 337 9.2 Stochastic Approximation Methods, 347 9.3 Recursive Least Squares for Linear Models, 349 9.4 Temporal Difference Learning with a Linear Model, 356 9.5 Bellman’s Equation Using a Linear Model, 358
viii CONTENTS 9.6 Analysis of TD(0), LSTD, and LSPE Using a Single State, 364 9.7 Gradient-Based Methods for Approximate Value Iteration*, 366 9.8 Least Squares Temporal Differencing with Kernel Regression*, 371 9.9 Value Function Approximations Based on Bayesian Learning*, 373 9.10 Why Does It Work*, 376 9.11 Bibliographic Notes, 379 Problems, 381 10 Optimizing While Learning 383 10.1 Overview of Algorithmic Strategies, 385 10.2 Approximate Value Iteration and Q- Learning Using Lookup Tables, 386 10.3 Statistical Bias in the Max Operator, 397 10.4 Approximate Value Iteration and Q-Learning Using Linear Models, 400 10.5 Approximate Policy Iteration, 402 10.6 The Actor-Critic Paradigm, 408 10.7 Policy Gradient Methods, 410 10.8 The Linear Programming Method Using Basis Functions, 411 10.9 Approximate Policy Iteration Using Kernel Regression*, 413 10.10 Finite Horizon Approximations for Steady-State Applications, 415 10.11 Bibliographic Notes, 416 Problems, 418 11 Adaptive Estimation and Stepsizes 419 11.1 Learning Algorithms and Stepsizes, 420 11.2 Deterministic Stepsize Recipes, 425 11.3 Stochastic Stepsizes, 433 11.4 Optimal Stepsizes for Nonstationary Time Series, 437 11.5 Optimal Stepsizes for Approximate Value Iteration, 447 11.6 Convergence, 449 11.7 Guidelines for Choosing Stepsize Formulas, 451 11.8 Bibliographic Notes, 452 Problems, 453
CONTENTS 12 Exploration Versus Exploitation 12.1 A Learning Exercise: The Nomadic Trucker, 457 12.2 An Introduction to Learning, 460 12.3 Heuristic Learning Policies, 464 12.4 Gittins Indexes for Online Learning, 470 12.5 The Knowledge Gradient Policy, 477 12.6 Learning with a Physical State, 482 12.7 Bibliographic Notes, 492 Problems, 493 13 Value Function Approximations for Resource Allocation Problems 13.1 Value Functions versus Gradients, 498 13.2 Linear Approximations, 499 13.3 Piecewise-Linear Approximations, 501 13.4 Solving a Resource Allocation Problem Using Piecewise-Linear Functions, 505 13.5 The SHAPE Algorithm, 509 13.6 Regression Methods, 513 13.7 Cutting Planes*, 516 13.8 Why Does It Work?**, 528 13.9 Bibliographic Notes, 535 Problems, 536 14 Dynamic Resource Allocation Problems 14.1 An Asset Acquisition Problem, 541 14.2 The Blood Management Problem, 547 14.3 A Portfolio Optimization Problem, 557 14.4 A General Resource Allocation Problem, 560 14.5 A Fleet Management Problem, 573 14.6 A Driver Management Problem, 580 14.7 Bibliographic Notes, 585 Problems, 586 15 Implementation Challenges 15.1 Will ADP Work for Your Problem?, 593 15.2 Designing an ADP Algorithm for Complex Problems, 594 15.3 Debugging an ADP Algorithm, 596
X CONTENTS 15.4 Practical Issues, 597 15.5 Modeling Your Problem, 602 15.6 Online versus Offline Models, 604 15.7 If It Works, Patent It!, 606 Bibliography 607 Index 623
|
| any_adam_object | 1 |
| author | Powell, Warren B. 1955- |
| author_GND | (DE-588)170298175 |
| author_facet | Powell, Warren B. 1955- |
| author_role | aut |
| author_sort | Powell, Warren B. 1955- |
| author_variant | w b p wb wbp |
| building | Verbundindex |
| bvnumber | BV039835544 |
| classification_rvk | QH 423 SK 880 SK 970 ST 230 |
| classification_tum | DAT 917f |
| ctrlnum | (OCoLC)681536496 (DE-599)BSZ353876828 |
| dewey-full | 519.7/03 |
| dewey-hundreds | 500 - Natural sciences and mathematics |
| dewey-ones | 519 - Probabilities and applied mathematics |
| dewey-raw | 519.7/03 |
| dewey-search | 519.7/03 |
| dewey-sort | 3519.7 13 |
| dewey-tens | 510 - Mathematics |
| discipline | Informatik Mathematik Wirtschaftswissenschaften |
| edition | Second edition |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>02034nam a2200457 c 4500</leader><controlfield tag="001">BV039835544</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20190218 </controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">120124s2011 a||| |||| 00||| eng d</controlfield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">047060445X</subfield><subfield code="c">(hbk.) £83.50</subfield><subfield code="9">0-470-60445-X</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9780470604458</subfield><subfield code="c">(hbk.) £83.50</subfield><subfield code="9">978-0-470-60445-8</subfield></datafield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">9781118029176</subfield><subfield code="c">oBook</subfield><subfield code="9">978-1-118-02917-6</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)681536496</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BSZ353876828</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">rda</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-11</subfield><subfield code="a">DE-384</subfield><subfield code="a">DE-91</subfield><subfield code="a">DE-20</subfield><subfield code="a">DE-706</subfield><subfield code="a">DE-945</subfield><subfield code="a">DE-521</subfield><subfield code="a">DE-19</subfield><subfield code="a">DE-739</subfield></datafield><datafield tag="082" ind1="0" ind2=" "><subfield code="a">519.7/03</subfield><subfield code="2">22</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">QH 423</subfield><subfield code="0">(DE-625)141577:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">SK 880</subfield><subfield code="0">(DE-625)143266:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">SK 970</subfield><subfield code="0">(DE-625)143276:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ST 230</subfield><subfield code="0">(DE-625)143617:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">DAT 917f</subfield><subfield code="2">stub</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Powell, Warren B.</subfield><subfield code="d">1955-</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)170298175</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Approximate dynamic programming</subfield><subfield code="b">solving the curses of dimensionality</subfield><subfield code="c">Warren B. Powell (Princeton University, The Department of operations research and financial engineering, Princeton, NJ)</subfield></datafield><datafield tag="250" ind1=" " ind2=" "><subfield code="a">Second edition</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">Hoboken, New Jersey</subfield><subfield code="b">Wiley</subfield><subfield code="c">[2011]</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">XVIII, 627 Seiten</subfield><subfield code="b">Illustrationen, Diagramme</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="490" ind1="0" ind2=" "><subfield code="a">Wiley series in probability and statistics</subfield></datafield><datafield tag="500" ind1=" " ind2=" "><subfield code="a">Includes bibliographical references and index</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Dynamische Optimierung</subfield><subfield code="0">(DE-588)4125677-3</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Dynamische Optimierung</subfield><subfield code="0">(DE-588)4125677-3</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="776" ind1="0" ind2="8"><subfield code="i">Erscheint auch als</subfield><subfield code="n">Online-Ausgabe, EPUB</subfield><subfield code="z">978-1-118-02916-9</subfield></datafield><datafield tag="776" ind1="0" ind2="8"><subfield code="i">Erscheint auch als</subfield><subfield code="n">Online-Ausgabe, PDF</subfield><subfield code="z">978-1-118-02915-2</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">HBZ Datenaustausch</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=024695545&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-024695545</subfield></datafield></record></collection> |
| id | DE-604.BV039835544 |
| illustrated | Illustrated |
| indexdate | 2024-07-10T00:12:20Z |
| institution | BVB |
| isbn | 047060445X 9780470604458 9781118029176 |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-024695545 |
| oclc_num | 681536496 |
| open_access_boolean | |
| owner | DE-11 DE-384 DE-91 DE-BY-TUM DE-20 DE-706 DE-945 DE-521 DE-19 DE-BY-UBM DE-739 |
| owner_facet | DE-11 DE-384 DE-91 DE-BY-TUM DE-20 DE-706 DE-945 DE-521 DE-19 DE-BY-UBM DE-739 |
| physical | XVIII, 627 Seiten Illustrationen, Diagramme |
| publishDate | 2011 |
| publishDateSearch | 2011 |
| publishDateSort | 2011 |
| publisher | Wiley |
| record_format | marc |
| series2 | Wiley series in probability and statistics |
| spelling | Powell, Warren B. 1955- Verfasser (DE-588)170298175 aut Approximate dynamic programming solving the curses of dimensionality Warren B. Powell (Princeton University, The Department of operations research and financial engineering, Princeton, NJ) Second edition Hoboken, New Jersey Wiley [2011] XVIII, 627 Seiten Illustrationen, Diagramme txt rdacontent n rdamedia nc rdacarrier Wiley series in probability and statistics Includes bibliographical references and index Dynamische Optimierung (DE-588)4125677-3 gnd rswk-swf Dynamische Optimierung (DE-588)4125677-3 s DE-604 Erscheint auch als Online-Ausgabe, EPUB 978-1-118-02916-9 Erscheint auch als Online-Ausgabe, PDF 978-1-118-02915-2 HBZ Datenaustausch application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=024695545&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Powell, Warren B. 1955- Approximate dynamic programming solving the curses of dimensionality Dynamische Optimierung (DE-588)4125677-3 gnd |
| subject_GND | (DE-588)4125677-3 |
| title | Approximate dynamic programming solving the curses of dimensionality |
| title_auth | Approximate dynamic programming solving the curses of dimensionality |
| title_exact_search | Approximate dynamic programming solving the curses of dimensionality |
| title_full | Approximate dynamic programming solving the curses of dimensionality Warren B. Powell (Princeton University, The Department of operations research and financial engineering, Princeton, NJ) |
| title_fullStr | Approximate dynamic programming solving the curses of dimensionality Warren B. Powell (Princeton University, The Department of operations research and financial engineering, Princeton, NJ) |
| title_full_unstemmed | Approximate dynamic programming solving the curses of dimensionality Warren B. Powell (Princeton University, The Department of operations research and financial engineering, Princeton, NJ) |
| title_short | Approximate dynamic programming |
| title_sort | approximate dynamic programming solving the curses of dimensionality |
| title_sub | solving the curses of dimensionality |
| topic | Dynamische Optimierung (DE-588)4125677-3 gnd |
| topic_facet | Dynamische Optimierung |
| url | http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=024695545&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| work_keys_str_mv | AT powellwarrenb approximatedynamicprogrammingsolvingthecursesofdimensionality |