Decision processes in dynamic probabilistic systems:
Saved in:
| Main Author: | |
|---|---|
| Format: | Book |
| Language: | English |
| Published: |
Dordrecht [u.a.]
Kluwer Acad. Publ.
1990
|
| Series: | Mathematics and its applications / East European series
42 |
| Subjects: | |
| Online Access: | Inhaltsverzeichnis |
| Physical Description: | XVII, 354 S. Illustrationen |
| ISBN: | 0792305442 |
Staff View
MARC
| LEADER | 00000nam a2200000 cb4500 | ||
|---|---|---|---|
| 001 | BV004239699 | ||
| 003 | DE-604 | ||
| 005 | 20230303 | ||
| 007 | t | ||
| 008 | 910218s1990 a||| |||| 00||| engod | ||
| 020 | |a 0792305442 |9 0-7923-0544-2 | ||
| 035 | |a (OCoLC)21599383 | ||
| 035 | |a (DE-599)BVBBV004239699 | ||
| 040 | |a DE-604 |b ger |e rakddb | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-12 |a DE-91 |a DE-739 |a DE-824 |a DE-706 |a DE-11 |a DE-188 | ||
| 050 | 0 | |a T57.95 | |
| 082 | 0 | |a 003/.56 |2 20 | |
| 084 | |a SK 800 |0 (DE-625)143256: |2 rvk | ||
| 084 | |a SK 820 |0 (DE-625)143258: |2 rvk | ||
| 084 | |a MAT 607f |2 stub | ||
| 100 | 1 | |a Gheorghe, Adrian V. |d 1945- |e Verfasser |0 (DE-588)170056694 |4 aut | |
| 245 | 1 | 0 | |a Decision processes in dynamic probabilistic systems |c by Adrian V. Gheorghe |
| 264 | 1 | |a Dordrecht [u.a.] |b Kluwer Acad. Publ. |c 1990 | |
| 300 | |a XVII, 354 S. |b Illustrationen | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 490 | 1 | |a Mathematics and its applications / East European series |v 42 | |
| 650 | 7 | |a Markov, Processus de |2 ram | |
| 650 | 7 | |a Prise de décision |2 ram | |
| 650 | 4 | |a Decision making | |
| 650 | 4 | |a Markov processes | |
| 650 | 0 | 7 | |a Markov-Entscheidungsprozess |0 (DE-588)4168927-6 |2 gnd |9 rswk-swf |
| 689 | 0 | 0 | |a Markov-Entscheidungsprozess |0 (DE-588)4168927-6 |D s |
| 689 | 0 | |5 DE-604 | |
| 810 | 2 | |a East European series |t Mathematics and its applications |v 42 |w (DE-604)BV000006709 |9 42 | |
| 856 | 4 | 2 | |m HBZ Datenaustausch |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=002637542&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 940 | 1 | |n oe | |
| 999 | |a oai:aleph.bib-bvb.de:BVB01-002637542 | ||
Record in the Search Index
| _version_ | 1804118435530014720 |
|---|---|
| adam_text | CONTENTS
Series Editor s Preface vii
Introduction xv
Chapter 1 Semi Markov and Markov Chains 1
1.1 Definitions and basic properties 1
1.1.1 Discrete time semi Markov and Markov behaviour of systems 3
1.1.2 Multi step transition probability 6
1.1.3 Semi Markov processes 8
1.1.4 State occupancy and waiting time statistics 10
1.1.5 Non homogeneous Markov processes 14
1.1.6 The limit theorem 16
1.1.7 Effect of small deviations in the transition probability matrix 18
1.1.7.1 Limits of some important characteristics for Markov
chains 22
1.2 Algebraic and analytical methods in the study of Markovian systems 31
1.2.1 Eigenvalues and eigenvectors 31
1.2.2 Stochastic matrices 33
1.2.3 Perron Frobenius theorem 34
1.2.4 The Geometric transformation (the z transform) 35
1.2.5 Exponential transformation (Laplace transform) 37
1.3 Transient and recurrent processes 38
1.3.1 Transient processes 38
1.3.2 The study of recurrent state occupancy in Markov
processes 41
1.4 Markovian populations 42
1.4.1 Vectorial processes with a Markovian structure 42
1.4.2 General branching processes 46
1.5 Partially observable Markov chains 48
1.5.1 The core process 50
1.5.2 The observation process 52
1.5.3 The state of knowledge and its dynamics 53
1.5.4 Examples 57
1.6 Rewards and discounting 67
1.6.1 Rewards for sequential decision processes 67
1.6.2 Rewards in decision processes with Markov structure 67
1.6.3 Markovian decision processes with and without discounting 68
x CONTENTS
1.7 Models and applications 70
1.7.1 Real systems with a Markovian structure 70
1.7.2 Model formulation and practical results 70
1.7.2.1 A semi Markov model for hospital planning 70
1.7.2.2 System reliability 82
1.7.2.3 A Markovian interpretation for PERT networks 95
1.8 Dynamic decision models for clinical diagnosis 102
1.8.1 Pattern recognition 102
1.8.2 Model optimization 103
Chapter 2 Dynamic and Linear Programming 105
2.1 Discrete dynamic programming 105
2.2 A linear programming formulation and an algorithm for computation 109
2.2.1 A general formulation for the LP problem and the
Simplex method 109
2.2.2 Linear programming a matrix formulation 113
Chapter 3 Utility Functions and Decisions under Risk 115
3.1 Informational lotteries and axioms for utility functions 115
3.2 Exponential utility functions 121
3.3 Decisions under risk and uncertainty; event trees 124
3.4 Probability encoding 126
Chapter 4 Markovian Decision Processes (Semi Markov and Markov)
with Complete Information (Completely Observable) 129
4.1 Value iteration algorithm (the finite horizon case) 129
4.1.1 Semi Markov decision processes 129
4.1.2 Markov decision processes 131
4.2 Policy iteration algorithm (the finite horizon optimization) 133
4.2.1 Semi Markov decision processes 133
4.2.2 Markov decision processes 140
4.3 Policy iteration with discounting 143
4.3.1 Semi Markov decision processes 143
4.3.2 Markov decision processes 149
4.4 Optimization algorithm using linear programming 153
4.4.1 Semi Markov decision process 153
4.4.2 Markov decision processes 158
CONTENTS xi
45 Risk sensitive decision processes 163
4.5.1 Risk sensitive finite horizon Markov decision processes 163
4.5.2 Risk sensitive infinite horizon Markov decision processes 168
4.5.3 Risk sensitive finite horizon semi Markov
decision processes 175
4.5.4 Risk sensitive infinite horizon semi Markov
decision processes 177
4.6 On eliminating sub optimal decision alternatives in Markov
and semi Markov decision processes 181
4.6.1 Markov decision processes 181
4.6.2 Semi Markov decision processes with finite horizon 183
Chapter 5 Partially Observable Markovian Decision Processes 187
5.1 Finite horizon partially observable Markov decision processes 187
5.2 The infinite horizon with discounting for partially observable
Markov decision processes 202
5.2.1 Model formulation 202
5.2.2 The concept of finitely transient policies 207
5.2.3 The function C(tc|8) approximated as a Markov process
with a finite number of states 209
5.3 A useful policy iteration algorithm, for discounted (p 1)
partially observable Markov decision processes 210
5.3.1 ThecasevforiV = 2 210
5.3.2 The case v for W 2 213
5.4 The infinite horizon without discounting for partially observable
Markov processes 219
5.4.1 Model formulations 219
5.4.2 Cost of a stationary policy 219
5.4.3 Policy improvement phase 222
5.4.4 Policy iteration algorithm 223
5.5 Partially observable semi Markov decision processes 228
5.5.1 Model formulation 229
5.5.2 State dynamics 230
5.5.3 The observation space 232
5.5.4 Overall system dynamics 233
5.5.5 Decision alternatives in clinical disorders 237
xii CONTENTS
5.6 Risk sensitive partially observable Markov decision processes 238
5.6.1 Model formulation and practical examples 238
5.6.1.1 Maintenance policies for a nuclear reactor
pressure vessel 239
5.6.1.2 Medical diagnosis and treatment as applied to
physiological systems 240
5.6.2 The stationary Markov decision process with
probabilistic observations of states 240
5.6.3 A branch and bound algorithm 250
5.6.4 A Fibonacci search method for a branch and bound algo¬
rithm for a partially observable Markov decision process 254
5.6.5 A numerical example 255
Chapter 6 Policy Constraints in Markov Decision Processes 261
6.1 Methods of investigating policy costraints in Markov
decision processes 261
6.2 Markov decision processes with policy constraints 263
6.2.1 A Lagrange multiplier formulation 266
6.2.2 Development and convergence of the algorithm 277
6.2.3 The case of transient states and periodic processes 284
6.3 Risk sensitive Markov decision process with policy constraints 285
6.3.1 A Lagrange multiplier formulation 285
6.3.2 Development and convergence of the algorithm 292
Chapter 7 Applications 296
7.1 The emergency repair control for electrical power systems 296
7.1.1 Reliability and system effectiveness 296
7.1.2 Reward structure 297
7.1.3 The Markovian decision process for emergency repair 298
7.1.4 Linear programming formulation for repair optimization 301
7.1.5 The investment problem 303
7.2 Stochastic models for evaluation of inspection and repair schedules
[2] 304
7.2.1 Inspection actions 308
7.2.1.1 Complete inspection 309
7.2.1.2 Control limit inspection 309
7.2.1.3 Inspection 310
CONTENTS xiii
7.2.2 Markov chain models 310
7.2.3 Cost structures and operating requirements 316
7.2.3.1 Inspection costs 316
7.2.3.2 Repair costs 316
7.2.3.3 Operating costs and requirements 317
7.2.3.4 Inspection and repair policies 318
7.2.3.5 Closed loop policies 318
7.2.3.6 Updating state probabilities after an inspection 321
7.2.3.7 Obtaining next time state probabilities using
transition matrix 321
7.2.3.8 Open loop policies 323
7.3 A Markovian decision model for clinical diagnosis and treatment
applied to the respiratory system 324
7.3.1 Concept of state in the respiratory system 326
7.3.2 The clinical observation space 329
7.3.3 Computing probabilities in cause effect models and
overall system dynamics 330
7.3.4 Decision alternatives in respiratory disorders 334
7.3.4.1 Branch and bound algorithm 336
7.3.4.2 Steps in the branch and bound algorithm 338
7.3.5 A numerical example for the respiratory system 339
7.3.6 Conclusions 343
Bibliography 344
Index 352
|
| any_adam_object | 1 |
| author | Gheorghe, Adrian V. 1945- |
| author_GND | (DE-588)170056694 |
| author_facet | Gheorghe, Adrian V. 1945- |
| author_role | aut |
| author_sort | Gheorghe, Adrian V. 1945- |
| author_variant | a v g av avg |
| building | Verbundindex |
| bvnumber | BV004239699 |
| callnumber-first | T - Technology |
| callnumber-label | T57 |
| callnumber-raw | T57.95 |
| callnumber-search | T57.95 |
| callnumber-sort | T 257.95 |
| callnumber-subject | T - General Technology |
| classification_rvk | SK 800 SK 820 |
| classification_tum | MAT 607f |
| ctrlnum | (OCoLC)21599383 (DE-599)BVBBV004239699 |
| dewey-full | 003/.56 |
| dewey-hundreds | 000 - Computer science, information, general works |
| dewey-ones | 003 - Systems |
| dewey-raw | 003/.56 |
| dewey-search | 003/.56 |
| dewey-sort | 13 256 |
| dewey-tens | 000 - Computer science, information, general works |
| discipline | Informatik Mathematik |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01745nam a2200445 cb4500</leader><controlfield tag="001">BV004239699</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20230303 </controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">910218s1990 a||| |||| 00||| engod</controlfield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">0792305442</subfield><subfield code="9">0-7923-0544-2</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)21599383</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BVBBV004239699</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">rakddb</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-12</subfield><subfield code="a">DE-91</subfield><subfield code="a">DE-739</subfield><subfield code="a">DE-824</subfield><subfield code="a">DE-706</subfield><subfield code="a">DE-11</subfield><subfield code="a">DE-188</subfield></datafield><datafield tag="050" ind1=" " ind2="0"><subfield code="a">T57.95</subfield></datafield><datafield tag="082" ind1="0" ind2=" "><subfield code="a">003/.56</subfield><subfield code="2">20</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">SK 800</subfield><subfield code="0">(DE-625)143256:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">SK 820</subfield><subfield code="0">(DE-625)143258:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">MAT 607f</subfield><subfield code="2">stub</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Gheorghe, Adrian V.</subfield><subfield code="d">1945-</subfield><subfield code="e">Verfasser</subfield><subfield code="0">(DE-588)170056694</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Decision processes in dynamic probabilistic systems</subfield><subfield code="c">by Adrian V. Gheorghe</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">Dordrecht [u.a.]</subfield><subfield code="b">Kluwer Acad. Publ.</subfield><subfield code="c">1990</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">XVII, 354 S.</subfield><subfield code="b">Illustrationen</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="490" ind1="1" ind2=" "><subfield code="a">Mathematics and its applications / East European series</subfield><subfield code="v">42</subfield></datafield><datafield tag="650" ind1=" " ind2="7"><subfield code="a">Markov, Processus de</subfield><subfield code="2">ram</subfield></datafield><datafield tag="650" ind1=" " ind2="7"><subfield code="a">Prise de décision</subfield><subfield code="2">ram</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Decision making</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Markov processes</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Markov-Entscheidungsprozess</subfield><subfield code="0">(DE-588)4168927-6</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Markov-Entscheidungsprozess</subfield><subfield code="0">(DE-588)4168927-6</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="810" ind1="2" ind2=" "><subfield code="a">East European series</subfield><subfield code="t">Mathematics and its applications</subfield><subfield code="v">42</subfield><subfield code="w">(DE-604)BV000006709</subfield><subfield code="9">42</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">HBZ Datenaustausch</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=002637542&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="940" ind1="1" ind2=" "><subfield code="n">oe</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-002637542</subfield></datafield></record></collection> |
| id | DE-604.BV004239699 |
| illustrated | Illustrated |
| indexdate | 2024-07-09T16:10:15Z |
| institution | BVB |
| isbn | 0792305442 |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-002637542 |
| oclc_num | 21599383 |
| open_access_boolean | |
| owner | DE-12 DE-91 DE-BY-TUM DE-739 DE-824 DE-706 DE-11 DE-188 |
| owner_facet | DE-12 DE-91 DE-BY-TUM DE-739 DE-824 DE-706 DE-11 DE-188 |
| physical | XVII, 354 S. Illustrationen |
| publishDate | 1990 |
| publishDateSearch | 1990 |
| publishDateSort | 1990 |
| publisher | Kluwer Acad. Publ. |
| record_format | marc |
| series2 | Mathematics and its applications / East European series |
| spelling | Gheorghe, Adrian V. 1945- Verfasser (DE-588)170056694 aut Decision processes in dynamic probabilistic systems by Adrian V. Gheorghe Dordrecht [u.a.] Kluwer Acad. Publ. 1990 XVII, 354 S. Illustrationen txt rdacontent n rdamedia nc rdacarrier Mathematics and its applications / East European series 42 Markov, Processus de ram Prise de décision ram Decision making Markov processes Markov-Entscheidungsprozess (DE-588)4168927-6 gnd rswk-swf Markov-Entscheidungsprozess (DE-588)4168927-6 s DE-604 East European series Mathematics and its applications 42 (DE-604)BV000006709 42 HBZ Datenaustausch application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=002637542&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Gheorghe, Adrian V. 1945- Decision processes in dynamic probabilistic systems Markov, Processus de ram Prise de décision ram Decision making Markov processes Markov-Entscheidungsprozess (DE-588)4168927-6 gnd |
| subject_GND | (DE-588)4168927-6 |
| title | Decision processes in dynamic probabilistic systems |
| title_auth | Decision processes in dynamic probabilistic systems |
| title_exact_search | Decision processes in dynamic probabilistic systems |
| title_full | Decision processes in dynamic probabilistic systems by Adrian V. Gheorghe |
| title_fullStr | Decision processes in dynamic probabilistic systems by Adrian V. Gheorghe |
| title_full_unstemmed | Decision processes in dynamic probabilistic systems by Adrian V. Gheorghe |
| title_short | Decision processes in dynamic probabilistic systems |
| title_sort | decision processes in dynamic probabilistic systems |
| topic | Markov, Processus de ram Prise de décision ram Decision making Markov processes Markov-Entscheidungsprozess (DE-588)4168927-6 gnd |
| topic_facet | Markov, Processus de Prise de décision Decision making Markov processes Markov-Entscheidungsprozess |
| url | http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=002637542&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| volume_link | (DE-604)BV000006709 |
| work_keys_str_mv | AT gheorgheadrianv decisionprocessesindynamicprobabilisticsystems |