We use the concept of probability to deal with uncertainty. Intuitively, the probability of an event is a number that measures the chance, or likelihood, that the event will occur. For instance, the probability that your favorite football team will win its next game measures the likelihood of a victory. The probability of an event is always a number between 0 and 1. The closer an event’s probability is to 1, the higher is the likelihood that the event will occur; the closer the event’s probability is to 0, the smaller is the likelihood that the event will occur. For example, if you believe that the probability that your favorite football team will win its next game is .95, then you are almost sure that your team will win. However, if you believe that the probability of victory is only .10, then you have very little confidence that your team will win.
When performing statistical studies, we sometimes collect data by performing a controlled experiment. For instance, we might purposely vary the operating conditions of a manufacturing process in order to study the effects of these changes on the process output. Alternatively, we sometimes obtain data by observing uncontrolled events. For example, we might observe the closing price of a share of General Motors’ stock every day for 30 trading days. In order to simplify our terminology, we will use the word experiment to refer to either method of data collection.
An experiment is any process of observation that has an uncertain outcome. The process must be defined so that on any single repetition of the experiment, one and only one of the possible outcomes will occur. The possible outcomes for an experiment are called experimental outcomes.
For example, if the experiment consists of tossing a coin, the experimental outcomes are “head” and “tail.” If the experiment consists of rolling a die, the experimental outcomes are 1, 2, 3, 4, 5, and 6. If the experiment consists of subjecting an automobile to a tailpipe emissions test, the experimental outcomes are pass and fail.
We often wish to assign probabilities to experimental outcomes. This can be done by several methods. Regardless of the method used, probabilities must be assigned to the experimental outcomes so that two conditions are met:
Sometimes, when all of the experimental outcomes are equally likely, we can use logic to assign probabilities. This method is called the classical method. As a simple example, consider the experiment of tossing a fair coin. Here, there are two equally likely experimental outcomes—head (H) and tail (T). Therefore, logic suggests that the probability of observing a head, denoted P(H), is 1/2 = .5, and that the probability of observing a tail, denoted P(T), is also 1/2 = .5. Notice that each probability is between 0 and 1. Furthermore, because H and T are all of the experimental outcomes, P(H) + P(T) = 1.
Probability is often interpreted to be a long-run relative frequency. As an example, consider repeatedly tossing a coin. If we get 6 heads in the first 10 tosses, then the relative frequency, or fraction, of heads is 6/10 = .6. If we get 47 heads in the first 100 tosses, the relative frequency of heads is 47/100 = .47. If we get 5,067 heads in the first 10,000 tosses, the relative frequency of heads is 5,067/10,000 = .5067.1 Since the relative frequency of heads is approaching (that is, getting closer to) .5, we might estimate that the probability of obtaining a head when tossing the coin is .5. When we say this, we mean that, if we tossed the coin an indefinitely large number of times (that is, a number of times approaching infinity), the relative frequency of heads obtained would approach .5. Of course, in actuality it is impossible to toss a coin (or perform any experiment) an indefinitely large number of times. Therefore, a relative frequency interpretation of probability is a mathematical idealization. To summarize, suppose that E is an experimental outcome that might occur when a particular experiment is performed. Then the probability that E will occur, P(E), can be interpreted to be the number that would be approached by the relative frequency of E if we performed the experiment an indefinitely large number of times. It follows that we often think of a probability in terms of the percentage of the time the experimental outcome would occur in many repetitions of the experiment. For instance, when we say that the probability of obtaining a head when we toss a coin is .5, we are saying that, when we repeatedly toss the coin an indefinitely large number of times, we will obtain a head on 50 percent of the repetitions.
Sometimes it is either difficult or impossible to use the classical method to assign probabilities. Since we can often make a relative frequency interpretation of probability, we can estimate a probability by performing the experiment in which an outcome might occur many times. Then, we estimate the probability of the experimental outcome to be the proportion of the time that the outcome occurs during the many repetitions of the experiment. For example, to estimate the probability that a randomly selected consumer prefers Coca-Cola to all other soft drinks, we perform an experiment in which we ask a randomly selected consumer for his or her preference. There are two possible experimental outcomes: “prefers Coca-Cola” and “does not prefer Coca-Cola.” However, we have no reason to believe that these experimental outcomes are equally likely, so we cannot use the classical method. We might perform the experiment, say, 1,000 times by surveying 1,000 randomly selected consumers. Then, if 140 of those surveyed said that they prefer Coca-Cola, we would estimate the probability that a randomly selected consumer prefers Coca-Cola to all other soft drinks to be 140/1,000 = .14. This is called the relative frequency method for assigning probability.
If we cannot perform the experiment many times, we might estimate the probability by using our previous experience with similar situations, intuition, or special expertise that we may possess. For example, a company president might estimate the probability of success for a onetime business venture to be .7. Here, on the basis of knowledge of the success of previous similar ventures, the opinions of company personnel, and other pertinent information, the president believes that there is a 70 percent chance the venture will be successful.
When we use experience, intuitive judgement, or expertise to assess a probability, we call this a subjective probability. Such a probability may or may not have a relative frequency interpretation. For instance, when the company president estimates that the probability of a successful business venture is .7, this may mean that, if business conditions similar to those that are about to be encountered could be repeated many times, then the business venture would be successful in 70 percent of the repetitions. Or, the president may not be thinking in relative frequency terms but rather may consider the venture a “one-shot” proposition. We will discuss some other subjective probabilities later. However, the interpretations of statistical inferences we will explain later, are based on the relative frequency interpretation of probability. For this reason, we will concentrate on this interpretation.