Home/ Questions/Q 8289107
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-08T12:25:57+00:00

Mutual Information is defined by the formula I(X;Y) = H(X) – H(X|Y) = H(Y)

  • 0

Mutual Information is defined by the formula

I(X;Y) = H(X) - H(X|Y) = H(Y) - H(Y|X)

where X,Y are some column vectors.In my case,X is a continuous signal and Y is the discretized signal. size(X)=100 and number of discretizations for Y is say 10 and word length of Y is 5(say).Now,I know that first we have to find the joint probability,then the conditional probability and then I(X,Y).In this light,I have the following implementation issues

  1. While calculating the joint probabilities,would they be calculated till the size(x) or till word length of the discrretized signal Y?
  2. How to obtain a single numeric value of I and a plot of I
  3. How to find the channel capacity
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Look at the last part of your equation: what is H(Y|X)? It is the entropy of Y assuming we know X… but Y is just a discretization of X! More specifically, if we know X, then we precisely know Y, and so the entropy of Y given X is zero. This leaves you with 

    I(X;Y) = H(Y)

    Do any of your questions still need to be answered with this in mind?

    • 0