任意二进信源相对熵密度的若干性质

SOME PROPERTIES OF RELATIVE ENTROPY DENSITIES OF ARBITRARY BINARY INFOMATION SOURCES

  • 摘要: 设(Xnn>1是取值于0,1的二值随机变量序列,本文讨论它的相对熵密度(fnn≥1的极限性质,并用此极限性质给出了(xnn>1服从强大数定律的一个充分条件。

     

    Abstract: In this paper the limiting properties of relative entropy densities ((fnn≥1 are discussed. Whence, at sufficient condition that (Xnn>1 follows the strong law of large numbers is derived. Definition. A binary information source is a sequence of random variables Xn,n≥1, such that 1. Each Xn takes values in 0, 1; 2. For all ,n≥1 px1, …, xn)=PX1,…, Xn=xn>0, xi∈0, 1, 1≤in In the following theorems we assume that Xn,n≥1 is an arbitrary binary source Snω) =X1ω)+…+Xnω),Hx, 1-x) is the entropy of Bernoulli distribution, that is, Hx, 1-x)= -x log x-(1-x) log (1-x) where the logarithm base is 2, and \varphi_n(\omega)=\frac1n \sum_i=1^n\leftX_i \log p_i+\left(1-X_i\right) \log \left(1-p_k\right)\right-\frac1n \log p\left(X_1, \cdots, X_n\right) where pn∈(0, 1), n=1, 2, … is a given sequence of real numbers. Theorem 1. \limsup _n \rightarrow \infty \varphi_n(\omega)<0 \quad a.e. Corollary 1. \undersetn \rightarrow \infty\lim \sup \left-\frac1n \log p\left(X_1, \cdots, X_n\right)\right<1 a.e. Corollary 2. If \undersetn \rightarrow \infty\lim \sup \fracS_n(\omega)n \geqslant p \quad \text a.e., \quad \frac12 \leqslant p \leqslant 1 or \limsup _n \rightarrow \infty \fracS_n(\omega)n \leqslant p \quad \text a.e., 0 \leqslant p \leqslant \frac12 then \limsup _n \rightarrow \infty\left-\frac1n \log p\left(X_1, \cdots, X_n\right)\right \leqslant H(p, 1-p) \text a.e. Theorem 2. If \liminf _n \rightarrow \infty\left-\frac1n \log p\left(X_1, \cdots, X_n\right)\right \geqslant C \quad \text a.e. then \liminf _n \rightarrow \infty H\left(\fracS_n(\omega)n, \quad 1-\fracS_n(\omega)n\right) \geqslant C \quad \text a.e. Theorem 3. Letting \begingathered D=\left\\omega: \undersetn \rightarrow \infty\liminf \varphi_n(\omega) \geqslant 0\right\ ; \\ S_n(\omega)=X_1(\omega)+\cdots+X_n(\omega) \endgathered we have \lim _n \rightarrow \infty \fracS_n(\omega)-\left(p_1+\cdots+p_n\right)n=0 \quad \text a.e. in D .

     

/

返回文章
返回