joint probability distribution
A probability distribution that quantifies the simultaneous uncertain behavior of two or more random variables. Specifically, a joint probability distribution expresses the likelihood of obtaining specified outcomes for some number of random variables. If there are two random variables, the joint distribution is called a bivariate distribution. If there are more than two variables it is called a multivariate distribution. A probability distribution over a single random variable is called a univariate distribution.
Like single variable probability distributions, the variables in a joint distribution can be discrete or continuous. If there are two discrete random events A and B, the joint probability for A and B is the likelihood that both events occur. The joint distribution for discrete random variables may be called a joint probability mass function If there are two continuous random variables X and Y, the joint probability distribution gives the probability that X and Y each fall into a specified range of values. In this case, the joint probability distribution may be called a joint density function. Like the case of a single-variable probability distribution, a joint probability distribution can be specified as a joint cumulative distribution, which gives the probability that the value of each random variable is less than or equal to any specified value.
Joint probability distributions are useful for quantifying the dependencies among uncertainties. Two events A and B are independent if the probability of both A and B occurring equals the probability that A occurs times the probability that B occurs. Likewise, if X and Y are two continuous random variables, and the distribution of X is not influenced by the value taken by Y, and vice versa, then the two random variables are independent.
When dealing with multiple random variables there are additional ways of expressing uncertainty that are useful for exploring interdependencies. Once we know the joint probability distribution for random variables, we can calculate their individual distributions. The conditional probability assumes that one event has taken place or will take place, and then asks for the probability of the other (e.g., the likelihood of A occurring, given that B has already occurred). Conditional probability distributions arise from joint probability distributions when we need to know the probability of one event given that the other event has happened and the random variables behind these events are joint. The marginal probability is the likelihood of an event occurring (e.g., event A), irrespective of whether or not B occurs. It may be thought of as an unconditional probability. The marginal distribution of one of the variables is the probability distribution of that variable considered by itself. Note that if the events are independent, the conditional probability that A occurs given that B has occurred will exactly equal the marginal probability of A occurring, since whether or not B occurs has no bearing on event A.
To provide an example, the bivariate normal distribution is an often-used joint probability distribution for two random variables, X and Y, each of which is normally distributed. The distribution is specified by 5 parameters, the means of the distribution (μx, μy) the standard deviations of the distributions (σx, σy), and the correlation coefficient ρ between X and Y. In the example, the conditional density function for Y is narrower than the marginal density function because the variables are positively correlated. Thus, if the value of the random variable X is known, that reduces the uncertainty over Y.
Joint, marginal, and conditional distributions for a bivariate normal probability distribution