Information Theory of Discrete Random Fields

In this article, we mainly review the progress of information theory for discrete random fields indexed on graphs including lattices such as \mathbbZ^d and trees. The first part gives the extension of information measures for random processes to random fields on an infinite tree and establish two AEPs with convergence in probability, and then for Markovian chain fields on trees with convergence almost surely. The second part concerns random fields on \mathbbZ^d, and focuses on rate-distortion function and critical distortion. The third part introduces briefly the related aspects of information theory for random fields on general graph, including the entropy aspects, the Ising channel, the I-measures and general framework of information theory for random fields on graphs. We also pursue the possible extension of sequence matching to configuration matching for random fields. This article concerns discrete random processes and random fields which means that all random variables take value in a finite alphabet.