Understanding the Moment Generating Function (MGF) of a Random Variable

Introduction to the Moment Generating Function (MGF)

The Moment Generating Function (MGF) is a powerful tool in probability and statistics, particularly for understanding the moments of a random variable. It is defined as:

E(e^{tX})

where X is a random variable and t is a parameter. This function encapsulates all the moments of X. When the MGF exists in some interval around t 0, it can be expanded as:

E(e^{tX}) 1 mu_1 frac{x}{1!} mu_2 frac{x^2}{2!} mu_3 frac{x^3}{3!} dots

with mu_k representing the kth moment of X about zero.

Proof of the MGF

To prove this relationship, we start by expanding e^{tX} using the Taylor series expansion:

e^{tX} 1 X t frac{(X t)^2}{2!} frac{(X t)^3}{3!} dots

Integrating this series term by term gives us:

E(e^{tX}) 1 E(X t) frac{E(X^2 t^2)}{2!} frac{E(X^3 t^3)}{3!} dots

Recognizing the expected values of powers of X gives us the moments, leading to:

E(e^{tX}) 1 mu_1 t frac{mu_2 t^2}{2!} frac{mu_3 t^3}{3!} dots

MGF for a Centered Random Variable

When considering the MGF for a random variable centered around a different point, say a:

E(e^{t(X-a)}) E(e^{tX} e^{-ta}) e^{-ta} E(e^{tX})

If a is the mean of X (i.e., a E(X) mu), then:

E(e^{t(X-mu)}) e^{-tmu} E(e^{tX})

Thus, we can express the MGF about the mean by shifting the MGF about zero:

E[e^{tX-mu}] e^{-tmu} E(e^{tX})

Moments about a Point Other Than Zero

Additionally, moments about any point a can be derived using the binomial expansion. For example, if we want the kth moment about a:

mu'_k E[(X-a)^k]

This can be written as:

(X-a)^k sum_{i0}^{k} binom{k}{i} X^i (-a)^{k-i}

Integrating this expression gives:

mu'_k sum_{i0}^{k} binom{k}{i} mu_i (-a)^{k-i}

Here, mu_i are the moments about zero.

Conclusion

The MGF is a versatile tool for understanding and manipulating the moments of a random variable. Whether we are working with the MGF about zero, about the mean, or about any other point, the relationships and theorems discussed here provide a robust framework.