Information Measures: Information and its Description in Science and EngineeringThis book is intended to be an introduction to the mathematical description of information in science. The necessary mathematical theory of this introduction will be treated in a more vivid way than in the usual theorem-proof structure. This, however, enables us to develop an idea of the connections between different information measures and to understand the trains of thought in their derivation, which is a crucial point for correct applications. It is therefore our intention in the mathematical descriptions to evolve the important ideas of the derivations, so that we obtain the resulting functions as well as the main thoughts and the conditions for the validity of the result. This simplifies the handling of the information measures, which are sometimes hard to classify without any additional background information. Though the mathematical descriptions are the exact formulations of the measures examined, we do not restrict ourselves to rigorous mathematical considerations, but we will also integrate the different measures into the structure and context of possible information measures. Nevertheless the mathematical approach is unavoidable when we are looking for an objective description and for possible applications in optimization. |
Contents
III | 7 |
IV | 8 |
V | 13 |
VI | 14 |
VII | 15 |
VIII | 16 |
IX | 18 |
X | 23 |
CIII | 250 |
CIV | 251 |
CV | 252 |
CVI | 253 |
CVII | 255 |
CIX | 257 |
CX | 261 |
CXI | 262 |
XI | 28 |
XII | 30 |
XIII | 31 |
XIV | 39 |
XV | 42 |
XVI | 44 |
XVII | 47 |
XIX | 48 |
XX | 49 |
XXI | 51 |
XXII | 52 |
XXIII | 53 |
XXIV | 57 |
XXV | 59 |
XXVI | 61 |
XXVII | 66 |
XXVIII | 67 |
XXX | 68 |
XXXI | 75 |
XXXII | 76 |
XXXIII | 79 |
XXXIV | 82 |
XXXV | 85 |
XXXVI | 86 |
XXXVII | 87 |
XXXVIII | 88 |
XXXIX | 89 |
XL | 90 |
XLI | 91 |
XLII | 92 |
XLIII | 93 |
XLIV | 94 |
XLV | 95 |
XLVI | 100 |
XLVII | 101 |
XLVIII | 102 |
XLIX | 106 |
L | 112 |
LI | 113 |
LII | 115 |
LIII | 116 |
LIV | 120 |
LV | 121 |
LVI | 124 |
LVII | 125 |
LVIII | 126 |
LIX | 128 |
LXI | 130 |
LXIII | 131 |
LXIV | 132 |
LXV | 133 |
LXVI | 137 |
LXVII | 140 |
LXVIII | 143 |
LXIX | 145 |
LXX | 147 |
LXXI | 155 |
LXXII | 156 |
LXXIII | 162 |
LXXIV | 173 |
LXXVI | 174 |
LXXVII | 176 |
LXXVIII | 179 |
LXXIX | 189 |
LXXX | 193 |
LXXXI | 197 |
LXXXIII | 198 |
LXXXIV | 200 |
LXXXV | 204 |
LXXXVI | 207 |
LXXXVII | 209 |
LXXXVIII | 212 |
LXXXIX | 213 |
XC | 217 |
XCI | 219 |
XCIII | 221 |
XCIV | 222 |
XCV | 227 |
XCVI | 230 |
XCVII | 237 |
XCVIII | 238 |
XCIX | 239 |
C | 246 |
CI | 247 |
CII | 249 |
CXII | 265 |
CXIII | 268 |
CXIV | 273 |
CXV | 274 |
CXVI | 276 |
CXVII | 279 |
CXVIII | 282 |
CXIX | 284 |
CXX | 287 |
CXXIII | 288 |
CXXIV | 289 |
CXXVI | 293 |
CXXVII | 294 |
CXXVIII | 298 |
CXXIX | 302 |
CXXXI | 304 |
CXXXII | 307 |
CXXXIII | 312 |
CXXXIV | 313 |
CXXXVI | 314 |
CXXXVII | 315 |
CXXXVIII | 316 |
CXXXIX | 321 |
CXL | 327 |
CXLI | 332 |
CXLII | 333 |
CXLIII | 334 |
CXLV | 336 |
CXLVII | 340 |
CXLVIII | 344 |
CXLIX | 347 |
CLII | 351 |
CLIII | 356 |
CLIV | 363 |
CLV | 370 |
CLVI | 374 |
CLVII | 375 |
CLVIII | 377 |
CLIX | 379 |
CLX | 382 |
CLXI | 387 |
CLXII | 389 |
CLXIII | 390 |
CLXIV | 391 |
CLXVI | 396 |
CLXVII | 399 |
CLXVIII | 401 |
CLXIX | 412 |
CLXX | 413 |
CLXXI | 433 |
CLXXII | 435 |
CLXXIII | 436 |
CLXXIV | 437 |
CLXXV | 438 |
CLXXVII | 442 |
CLXXVIII | 446 |
CLXXIX | 449 |
CLXXX | 452 |
CLXXXI | 463 |
CLXXXII | 466 |
CLXXXIII | 467 |
CLXXXIV | 468 |
CLXXXVI | 470 |
CLXXXVII | 472 |
CLXXXVIII | 475 |
CLXXXIX | 476 |
CXC | 481 |
CXCII | 488 |
CXCIII | 490 |
CXCIV | 493 |
CXCVI | 494 |
CXCVII | 498 |
CXCVIII | 500 |
CXCIX | 501 |
CCI | 502 |
CCII | 505 |
CCIII | 506 |
CCIV | 507 |
CCV | 508 |
CCVI | 510 |
CCVII | 513 |
CCVIII | 516 |
CCIX | 519 |
| 545 | |
Other editions - View all
Common terms and phrases
able achieved ambiguity function boundary conditions calculate channel codewords compute concave function concept of information conditional distribution conditional distribution density convex function coordinate system covariance matrix Cramér-Rao bound derivative describe determine deterministic discrete distribution density function energy equation examine expectation value formulation ƒ ƒ ƒyx G₁ gain of information gaussian distribution density increase inequality information function information measures integral joint entropy Kolmogorov's information Kullback-Leibler distance Kullback's information leads likelihood function limit logarithm maximum entropy maximum likelihood estimation mutual information N₁ negentropy noise number of possible observation obtain optimal P₁ parameter Pmax priori information quotient random variables relative frequency Rényi's information result sequence Shannon's entropy Shannon's information sigma-algebra signal symbols term theory transformation transmission uncertainty variance vector αξ θα στ дх
Popular passages
Page 539 - Estimating the Dimension of a Model", The Annals of Statistics, Vol. 6, No. 2, pp.
Page 542 - Information theoretical analysis of multivariate correlation. IBM Journal of Research and Development, 4(l):66-82, Jan.
Page 534 - Divergence measures based on the Shannon entropy," IEEE Transactions on Information Theory. 37, 145151, 1991. 16. Nelder, JA and Mead, R., "A simplex method for function minimization," The Computer Journal, 1, 308-313, 1965.
Page 536 - Ohya M.: On compound state and mutual information in quantum information theory, IEEE Trans. Information Theory, 29 (1983) 770-777.
Page 534 - ... estimator for non-linear models. Int. J. Syst. Sci. 3(1972)4, p. 395-406. 28. LUENBERGER, DG, Observers for multivariable systems. IEEE Trans. Autom. Control AC-1 1(1966)2, p. 190-197. 29. LUENBERGER, DG, Optimization by vector space methods. New York, John Wiley & Sons, Inc., 1969. 30. MAHALANABIS, AK and FAROOQ, M., A second-order method for state estimation of non-linear dynamical systems.
Page 520 - Markovian Representation of Stochastic Processes by Canonical Variables', SIAM Journal of Control, Vol. 13, no.
Page 529 - On the relationship between the information measures and the Bayes probability of error...
Page 537 - Incorporation of spatial information in bayesian image reconstruction: The maximum residual likelihood criterion, Publications of the Astronomical Society of the Pacific, Vol.