Information measures: information and its description in science and engineering
Saved in:
| Main Author: | |
|---|---|
| Format: | Book |
| Language: | English |
| Published: |
Berlin [u.a.]
Springer
2004
|
| Edition: | 1. ed., 2. print. |
| Subjects: | |
| Online Access: | Inhaltsverzeichnis |
| Physical Description: | XIX, 547 S. graph. Darst. |
| ISBN: | 354040855x |
Staff View
MARC
| LEADER | 00000nam a22000008c 4500 | ||
|---|---|---|---|
| 001 | BV019604431 | ||
| 003 | DE-604 | ||
| 005 | 20080110 | ||
| 007 | t | ||
| 008 | 041123s2004 gw d||| |||| 00||| eng d | ||
| 020 | |a 354040855x |9 3-540-40855-x | ||
| 035 | |a (OCoLC)76607863 | ||
| 035 | |a (DE-599)BVBBV019604431 | ||
| 040 | |a DE-604 |b ger |e rakddb | ||
| 041 | 0 | |a eng | |
| 044 | |a gw |c DE | ||
| 049 | |a DE-355 |a DE-91 |a DE-703 |a DE-83 |a DE-N2 | ||
| 084 | |a SK 830 |0 (DE-625)143259: |2 rvk | ||
| 084 | |a ELT 505f |2 stub | ||
| 084 | |a DAT 570f |2 stub | ||
| 100 | 1 | |a Arndt, Christoph |e Verfasser |4 aut | |
| 245 | 1 | 0 | |a Information measures |b information and its description in science and engineering |c C. Arndt |
| 250 | |a 1. ed., 2. print. | ||
| 264 | 1 | |a Berlin [u.a.] |b Springer |c 2004 | |
| 300 | |a XIX, 547 S. |b graph. Darst. | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b n |2 rdamedia | ||
| 338 | |b nc |2 rdacarrier | ||
| 650 | 0 | 7 | |a Informationsmaß |0 (DE-588)4330787-5 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Informationstheorie |0 (DE-588)4026927-9 |2 gnd |9 rswk-swf |
| 650 | 0 | 7 | |a Entropie |0 (DE-588)4014894-4 |2 gnd |9 rswk-swf |
| 689 | 0 | 0 | |a Entropie |0 (DE-588)4014894-4 |D s |
| 689 | 0 | 1 | |a Informationstheorie |0 (DE-588)4026927-9 |D s |
| 689 | 0 | |5 DE-604 | |
| 689 | 1 | 0 | |a Informationsmaß |0 (DE-588)4330787-5 |D s |
| 689 | 1 | 1 | |a Informationstheorie |0 (DE-588)4026927-9 |D s |
| 689 | 1 | |5 DE-604 | |
| 856 | 4 | 2 | |m HBZ Datenaustausch |q application/pdf |u http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=012934287&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |3 Inhaltsverzeichnis |
| 999 | |a oai:aleph.bib-bvb.de:BVB01-012934287 | ||
Record in the Search Index
| _version_ | 1804132961263550464 |
|---|---|
| adam_text | Table of Contents
Abstract l
Structure and Structuring 3
1 Introduction 7
Science and information 8
Man as control loop 13
Information, complexity and typical sequences 14
Concepts of information 15
Information, its technical dimension and the meaning of a message 16
Information as a central concept? 18
2 Basic considerations 23
2.1 Formal derivation of information 23
2.1.1 Unit and reference scale 28
2.1.2 Information and the unit element 30
2.2 Application of the information measure (Shannon s information) 31
2.2.1 Summary 39
2.3 The law of Weber and Fechner 42
2.4 Information of discrete random variables 44
3 Historic development of information theory 47
3.1 Development of information transmission 47
3.1.1 Samuel F. B. Morse 1837 47
3.1.2 Thomas Edison 1874 47
3.1.3 Nyquist 1924 48
3.1.4 Optimal number of characters of the alphabet used for the coding 49
3.2 Development of information functions 51
3.2.1 Hartley 1928 51
3.2.2 Dennis Gabor 1946 52
3.2.3 Shannon 1948 53
3.2.3.1 Validity of the postulates for Shannon s Information 57
3.2.3.2 Shannon s information (another possibility of a derivation) ...59
3.2.3.3 Properties of Shannon s information, entropy 61
XII Title pages
3.2.3.4 Shannon s entropy or Shannon s information? 66
3.2.3.5 The Kraft inequality 67
Kraft s inequality: 67
Proof of Kraft s inequality: 68
3.2.3.6 Limits of the optimal length of codewords 75
3.2.3.6.1 Shannon s coding theorem 75
3.2.3.6.2 A sequence of n symbols (elements) 76
3.2.3.6.3 Application of the previous results 79
3.2.3.7 Information and utility (coding, porfolio analysis) 82
4 The concept of entropy in physics 85
The laws of thermodynamics: 85
4.1 Macroscopic entropy 86
4.1.1 SadiCarnot 1824 86
4.1.2 Clausius s entropy 1850 86
4.1.3 Increase of entropy in a closed system 87
4.1.4 Prigogine s entropy 88
4.1.5 Entropy balance equation 89
4.1.6 Gibbs s free energy and the quality of the energy 90
4.1.7 Considerations on the macroscopic entropy 91
4.1.7.1 Irreversible transformations 92
4.1.7.2 Perpetuum mobile and transfer of heat 93
4.2 Statistical entropy 94
4.2.1 Boltzmann s entropy 94
4.2.2 Derivation of Boltzmann s entropy 95
4.2.2.1 Variation, permutation and the formula of Stirling 95
4.2.2.2 Special case: Two states 100
4.2.2.3 Example: Lottery 101
4.2.3 The Boltzmann factor 102
4.2.4 Maximum entropy in equilibrium 106
4.2.5 Statistical interpretation of entropy 112
4.2.6 Examples regarding statistical entropy 113
4.2.6.1 Energy and fluctuation 115
4.2.6.2 Quantized oscillator 116
4.2.7 Brillouin Schrodinger negentropy 120
4.2.7.1 Brillouin: Precise definition of information 121
4.2.7.2 Negentropy as a generalization of Carnot s principle 124
Maxwell s demon 125
4.2.8 Information measures of Hartley and Boltzmann 126
4.2.8.1 Examples 128
4.2.9 Shannon s entropy 128
4.3 Dynamic entropy 130
4.3.1 Eddington and the arrow of time 130
4.3.2 Kolmogorov s entropy 131
4.3.3 Renyi s entropy 132
Title pages XIII
5 Extension of Shannon s information 133
5.1 Renyi s Information 1960 133
5.1.1 Properties of Renyi s entropy 137
5.1.2 Limits in the interval 0 a °° 140
5.1.3 Nonnegativity for discrete events 143
5.1.4 Additivity and a connection to Minkowski s norm 145
5.1.5 The meaning ofS^A) for a 1 and a 1: 147
5.1.6 Graphical presentations of Renyi s information 155
5.2 Another generalized entropy (logical expansion) 156
5.3 Gain of information via conditional probabilities 162
5.4 Other entropy or information measures 173
5.4.1 Daroczy s entropy 173
5.4.2 Quadratic entropy 174
5.4.3 /? norm entropy 176
6 Generalized entropy measures 179
6.1 The corresponding measures of divergence 189
6.2 Weighted entropies and expectation values of entropies 193
7 Information functions and gaussian distributions 197
7.1 Renyi s information of a gaussian distributed random variable 197
7.1.1 Renyi s or information 198
7.1.2 Renyi s G divergence 200
7.2 Shannon s information 204
8 Shannon s information of discrete distributions 207
8.1 Continuous and discrete random variables 209
8.1.1 Summary 212
8.2 Shannon s information of a gaussian distribution 213
8.3 Shannon s information as the possible gain of information in an
observation 217
8.4 Limits of the information, limitations of the resolution 219
8.4.1 The resolution or the precision of the measurements 219
8.4.2 The uncertainty relation of the Fourier transformation 221
8.5 Maximization of the entropy of a continuous random variable 222
9 Information functions for gaussian distributions part II .227
9.1 Kullback s information 227
9.1.1 G for gaussian distribution densites 230
9.2 Kullback s divergence 237
9.2.1 Jensen s inequality for G, 238
9.3 Kolmogorov s information 239
XIV Title pages
9.4 Transformation of the coordinate system and the effects on the
information 246
9.4.1 Sa information 247
9.4.2 G divergence 249
9.4.3 5 information 250
9.4.3.1 Example 251
9.4.4 Discrimination information 252
9.4.5 Kolmogorov s information 253
9.4.6 Prerequisites for the transformations 255
9.5 Transformation, discrete and continuous measures of entropy 255
9.6 Summary of the information functions 257
10 Bounds of the variance 261
10.1 Cramer Rao bound 262
10.1.1 Fisher s information for gaussian distribution densities 265
10.1.2 Fisher s information and Kullback s information 268
10.1.3 Fisher s information and the metric tensor 273
10.1.4 Fisher s information and the stochastic observability 274
10.1.4.1 Fisher s information and the Matrix Riccati equation 276
10.1.5 Fisher s information and maximum likelihood estimation 279
10.1.6 Fisher s information and weighted least squares estimation 282
10.1.7 The availability of the Cramer Rao bound 284
10.1.8 Efficiency, asymptotic efficiency, consistency, bias 287
10.1.8.1 Unbiased estimator 287
10.1.8.2 Consistency 287
10.1.8.3 Efficiency 288
10.1.9 Summary 289
10.2 Chapman Robbins bound 289
10.2.1 Cramer Rao bound versus Chapman Robbins bound 293
10.3 Bhattacharrya bound 294
Remark: 298
Remark 302
10.3.1 Bhattacharrya bound and Cramer Rao bound 302
10.3.2 Bhattacharrya s bound for gaussian distribution densities 304
10.4 Barankin bound 307
10.5 Other bounds 312
Fraser Guttman bound 313
Kiefer bound 313
Extended Fraser Guttman bound 314
10.6 Summary 315
10.7 Biased estimator 316
10.7.1 Biased estimator versus unbiased estimator 321
Title pages XV
11 Ambiguity function 327
11.1 The ambiguity function and Kullback s information 332
11.2 Connection between ambiguity function and Fisher s information ..333
11.3 Maximum likelihood estimation and the ambiguity function 334
11.3.1 Maximum likelihood estimation = minimum Kullback estimation
= maximum ambiguity estimation = minimum variance estimation..334
11.3.2 Maximum likelihood estimation 336
11.3.2.1 Application: Discriminator (Demodulation) 336
11.4 The ML estimation is asymptotically efficient 340
11.5 Transition to the Akaike information criterion 344
12 Akaike s information criterion 347
12.1 Akaike s information criterion and regression 347
12.1.1 Least squares regression 347
12.1.2 Application of the results to the ambiguity function 351
12.2 B1C, SC or HQ 356
13 Channel information 363
13.1 Redundancy 370
13.1.1 Knowledge, redundancy, utility 374
13.2 Rate of transmission and equivocation 375
13.3 Hadamard s inequality and Gibbs s second theorem 377
13.4 Kolmogorov s information 379
13.5 Kullbacks divergence 382
13.6 An example of a transmission 387
13.7 Communication channel and information processing 389
13.7.1 Semantic, syntactic and pragmatic information 390
13.7.2 Information, first time occurrence, confirmation 391
13.8 Shannon s bound 391
13.9 Example of the channel capacity 396
14 Deterministic and stochastic information 399
14.1 Information in state space models 399
14.2 The observation equation 401
14.3 Transmission faster than light 417
14.4 Information about state space variables 419
15 Maximum entropy estimation 433
15.1 The difference between maximum entropy and minimum variance.435
15.2 The difference from bootstrap or resampling methods 436
15.3 A maximum entropy example 437
15.4 Maximum entropy: The method 439
XVI Title pages
15.4.1 Maximum Shannon entropy 439
15.4.2 Minimum Kullback Leibler distance 445
15.5 Maximum entropy and minimum discrimination information 450
15.6 Generation of generalized entropy measures 454
15.6.1 Example: Gaussian distribution and Shannon s information 457
16 Concluding remarks 463
16.1 Information, entropy and self organization 463
16.2 Complexity theory 466
16.3 Data reduction 467
16.4 Cryptology 468
16.5 Concluding considerations 469
16.5.1 Information, entropy and probability 471
16.6 Information 473
Appendix 475
A. 1 Inequality for Kullback s information 475
A.2 The log sum inequality 476
A.3 Generalized entropy, divergence and distance measures 481
A.3.1 Entropy measures 481
A.3.2 Generalized measures of distance 488
A.3.3 Generalized measures of the directed divergence 490
A.3.4 Generalized measures of divergence 493
A.3.4.1 Information radius and the J divergence 493
A.3.4.2 Generalization of the R divergence 494
A.3.4.3 Generalization of the J divergence 488
A.4 A short introduction to probability theory 500
A.4.1 Axiomatic definition of probability 501
A.4.1.1 Events, elementary events, sample space 501
A.4.1.2 Classes of subsets, fields 502
A.4.1.3 Axiomatic definition of probability according to
Kolmogorov 505
Probability space 506
A.4.1.4 Random variables 506
A.4.1.5 Probability distribution 507
A.4.1.6 Probability space, sample space, realization space 508
A.4.1.7 Probability distribution and distribution density function 508
A.4.1.8 Probability distribution density function (PDF) 510
A.5 The regularity conditions 513
A.6 State space description 516
Bibliography 519
|
| any_adam_object | 1 |
| author | Arndt, Christoph |
| author_facet | Arndt, Christoph |
| author_role | aut |
| author_sort | Arndt, Christoph |
| author_variant | c a ca |
| building | Verbundindex |
| bvnumber | BV019604431 |
| classification_rvk | SK 830 |
| classification_tum | ELT 505f DAT 570f |
| ctrlnum | (OCoLC)76607863 (DE-599)BVBBV019604431 |
| discipline | Informatik Elektrotechnik Mathematik |
| edition | 1. ed., 2. print. |
| format | Book |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01644nam a22004338c 4500</leader><controlfield tag="001">BV019604431</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="005">20080110 </controlfield><controlfield tag="007">t</controlfield><controlfield tag="008">041123s2004 gw d||| |||| 00||| eng d</controlfield><datafield tag="020" ind1=" " ind2=" "><subfield code="a">354040855x</subfield><subfield code="9">3-540-40855-x</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)76607863</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)BVBBV019604431</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">rakddb</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="044" ind1=" " ind2=" "><subfield code="a">gw</subfield><subfield code="c">DE</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-355</subfield><subfield code="a">DE-91</subfield><subfield code="a">DE-703</subfield><subfield code="a">DE-83</subfield><subfield code="a">DE-N2</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">SK 830</subfield><subfield code="0">(DE-625)143259:</subfield><subfield code="2">rvk</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">ELT 505f</subfield><subfield code="2">stub</subfield></datafield><datafield tag="084" ind1=" " ind2=" "><subfield code="a">DAT 570f</subfield><subfield code="2">stub</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Arndt, Christoph</subfield><subfield code="e">Verfasser</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Information measures</subfield><subfield code="b">information and its description in science and engineering</subfield><subfield code="c">C. Arndt</subfield></datafield><datafield tag="250" ind1=" " ind2=" "><subfield code="a">1. ed., 2. print.</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">Berlin [u.a.]</subfield><subfield code="b">Springer</subfield><subfield code="c">2004</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">XIX, 547 S.</subfield><subfield code="b">graph. Darst.</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Informationsmaß</subfield><subfield code="0">(DE-588)4330787-5</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Informationstheorie</subfield><subfield code="0">(DE-588)4026927-9</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="650" ind1="0" ind2="7"><subfield code="a">Entropie</subfield><subfield code="0">(DE-588)4014894-4</subfield><subfield code="2">gnd</subfield><subfield code="9">rswk-swf</subfield></datafield><datafield tag="689" ind1="0" ind2="0"><subfield code="a">Entropie</subfield><subfield code="0">(DE-588)4014894-4</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2="1"><subfield code="a">Informationstheorie</subfield><subfield code="0">(DE-588)4026927-9</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="0" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="689" ind1="1" ind2="0"><subfield code="a">Informationsmaß</subfield><subfield code="0">(DE-588)4330787-5</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="1" ind2="1"><subfield code="a">Informationstheorie</subfield><subfield code="0">(DE-588)4026927-9</subfield><subfield code="D">s</subfield></datafield><datafield tag="689" ind1="1" ind2=" "><subfield code="5">DE-604</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="m">HBZ Datenaustausch</subfield><subfield code="q">application/pdf</subfield><subfield code="u">http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=012934287&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA</subfield><subfield code="3">Inhaltsverzeichnis</subfield></datafield><datafield tag="999" ind1=" " ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-012934287</subfield></datafield></record></collection> |
| id | DE-604.BV019604431 |
| illustrated | Illustrated |
| indexdate | 2024-07-09T20:01:08Z |
| institution | BVB |
| isbn | 354040855x |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-012934287 |
| oclc_num | 76607863 |
| open_access_boolean | |
| owner | DE-355 DE-BY-UBR DE-91 DE-BY-TUM DE-703 DE-83 DE-N2 |
| owner_facet | DE-355 DE-BY-UBR DE-91 DE-BY-TUM DE-703 DE-83 DE-N2 |
| physical | XIX, 547 S. graph. Darst. |
| publishDate | 2004 |
| publishDateSearch | 2004 |
| publishDateSort | 2004 |
| publisher | Springer |
| record_format | marc |
| spelling | Arndt, Christoph Verfasser aut Information measures information and its description in science and engineering C. Arndt 1. ed., 2. print. Berlin [u.a.] Springer 2004 XIX, 547 S. graph. Darst. txt rdacontent n rdamedia nc rdacarrier Informationsmaß (DE-588)4330787-5 gnd rswk-swf Informationstheorie (DE-588)4026927-9 gnd rswk-swf Entropie (DE-588)4014894-4 gnd rswk-swf Entropie (DE-588)4014894-4 s Informationstheorie (DE-588)4026927-9 s DE-604 Informationsmaß (DE-588)4330787-5 s HBZ Datenaustausch application/pdf http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=012934287&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA Inhaltsverzeichnis |
| spellingShingle | Arndt, Christoph Information measures information and its description in science and engineering Informationsmaß (DE-588)4330787-5 gnd Informationstheorie (DE-588)4026927-9 gnd Entropie (DE-588)4014894-4 gnd |
| subject_GND | (DE-588)4330787-5 (DE-588)4026927-9 (DE-588)4014894-4 |
| title | Information measures information and its description in science and engineering |
| title_auth | Information measures information and its description in science and engineering |
| title_exact_search | Information measures information and its description in science and engineering |
| title_full | Information measures information and its description in science and engineering C. Arndt |
| title_fullStr | Information measures information and its description in science and engineering C. Arndt |
| title_full_unstemmed | Information measures information and its description in science and engineering C. Arndt |
| title_short | Information measures |
| title_sort | information measures information and its description in science and engineering |
| title_sub | information and its description in science and engineering |
| topic | Informationsmaß (DE-588)4330787-5 gnd Informationstheorie (DE-588)4026927-9 gnd Entropie (DE-588)4014894-4 gnd |
| topic_facet | Informationsmaß Informationstheorie Entropie |
| url | http://bvbr.bib-bvb.de:8991/F?func=service&doc_library=BVB01&local_base=BVB01&doc_number=012934287&sequence=000002&line_number=0001&func_code=DB_RECORDS&service_type=MEDIA |
| work_keys_str_mv | AT arndtchristoph informationmeasuresinformationanditsdescriptioninscienceandengineering |