Tuesday, May 20, 2014

Kernel density estimate takes values larger than 1

During the research, I used a kernel density estimation and draw the graph.


When I first saw this, I was totally confused. Can density be taking value greater than 1? Isn't density probability? Probability shouldn't take value greater than 1!

Well, my thoughts turned out to be wrong. It's time for reviewing basic statistics courses.

PDF (Probability Density Function) can take value greater than 1. CDF (Cumulative Distribution Function) cannot exceed 1 but PDF can. For example, consider normal density function below.


If standard deviation is very small, then for x=mu, PDF takes very large value.
Even if the height under some value is greater than 1, it does not mean that the probability of having that value is greater than 1. <- This is the source of confusion.

Another thing to note is that PMD (Probability Mass function) for discrete random variable cannot take value greater than 1.