a utility-based approach to some information measures一些信息的基于用途的方法措施.pdfVIP

Entropy 2007, 9, 1-26 Entropy ISSN 1099-4300 /entropy/ Full paper A Utility-Based Approach to Some Information Measures Craig Friedman ∗ , Jinggang Huang ◦ and Sven Sandow † Standard Poor’s, 55 Water Street, 46th Floor, New York, NY 10041, USA E-Mail: ∗ craig friedman@, ◦ james huang@, † ssandow@ Received: 23 May 2006 / Accepted: 5 January 2007 / Published: 20 January 2007 Abstract: We review a decision theoretic, i.e., utility-based, motivation for entropy and Kullback- Leibler relative entropy, the natural generalizations that follow, and various properties of these generalized quantities. We then consider these generalized quantities in an easily interpreted spe- cial case. We show that the resulting quantities, share many of the properties of entropy and relative entropy, such as the data processing inequality and the second law of thermodynamics. We formulate an important statistical learning problem – probability estimation – in terms of a generalized relative entropy. The solution of this problem reflects general risk preferences via the utility function; moreover, the solution is optimal in a sense of robust absolute performance. Keywords: Generalized Entropy, Generalized Kullback-Leibler Relative Entropy, Decision The- ory, Expected Utility, Horse Race, Tsallis Entropy, Statistical Learning, Probability Estimation, Risk Neutral Pricing Measure Entropy 2007, 9, 1-26 2 1 Introduction It is well known that there are a number of ways to motivate the fundamental quantities of information theory, for example: • Entro