PROBABILITY

Chapter 13

Probability

Big idea: probability measures how likely something is. When you learn new information, the chances change. This chapter shows exactly how, and how to turn a "what is the chance of the clue?" into a "what is the chance of the cause?".

You already know the basic idea of probability. An experiment has several possible outcomes. The set of all outcomes is the sample space $S$. An event $E$ is a collection of outcomes, that is, a subset of $S$. When all outcomes are equally likely,

$$P(E) = \frac{n(E)}{n(S)}$$

where $n(E)$ is the number of outcomes in $E$ and $n(S)$ the number in $S$.

Roll two dice, one after the other. There are $6 \times 6 = 36$ equally likely outcomes, from $(1, 1)$ to $(6, 6)$. Let $E$ be the event "the sum is $8$". The outcomes are $(2, 6), (3, 5), (4, 4), (5, 3), (6, 2)$, so $n(E) = 5$ and

$$P(E) = \frac{5}{36} \approx 0.1389.$$

• $0 \le P(E) \le 1$, with $P(S) = 1$ and $P(\varnothing) = 0$.
• Complement: $P(E') = 1 - P(E)$, where $E'$ means "$E$ does not happen". Here $P(E') = 1 - \frac{5}{36} = \frac{31}{36}$.
• Addition rule: $P(E \cup G) = P(E) + P(G) - P(E \cap G)$, where $E \cup G$ means "$E$ or $G$ (or both)" and $E \cap G$ means "$E$ and $G$".

Check the addition rule. Let $G$ be "the first die shows $3$". Then $P(G) = \frac{1}{6}$, and "both" happens only for $(3, 5)$, so $P(E \cap G) = \frac{1}{36}$. So $P(E \cup G) = \frac{5}{36} + \frac{1}{6} - \frac{1}{36} = \frac{5}{18}$. Counting the outcomes directly gives the same answer, $\frac{5}{18}$.

Suppose someone tells you a clue about the outcome: "at least one die shows a $6$". The clue rules out many outcomes. Do the chances of other events change? Yes, and in this chapter you will learn how to calculate the new chances. You will meet:

• conditional probability: the chance of an event when you already know another event has happened;
• the multiplication rule: finding the chance that two or more events all happen;
• independent events: events that do not affect each other;
• the theorem of total probability and Bayes' theorem: combining several possible causes, and working backwards from an effect to its cause.

A clue changes what is possible. Roll two dice, and suppose you are told: "at least one die shows a $6$". Call this event $F$. Now only the outcomes with a $6$ are still possible. There are 11 of them. The sum is $8$ ($E$) in only 2 of these: $(2, 6)$ and $(6, 2)$. So, given $F$, the chance of $E$ is \frac{2}{11}$. Before the clue it was \frac{5}{36}$.

We write this as $P(E \mid F)$, read "the probability of $E$ given $F$". We can calculate it in two equivalent ways:

• Shrink the sample space. Count only the outcomes in $F$: $P(E \mid F) = \dfrac{n(E \cap F)}{n(F)} = \dfrac{2}{11}$.
• Use probabilities. Divide the chance of "both" by the chance of the clue:

$$P(E \mid F) = \frac{P(E \cap F)}{P(F)}, \qquad P(F) \ne 0.$$

Check: $P(E \cap F) = \frac{1}{18}$ and $P(F) = \frac{11}{36}$, so $P(E \mid F) = \frac{1}{18} \div \frac{11}{36} = \frac{2}{11}$, the same as before. The condition $P(F) \ne 0$ is needed because you cannot be given a clue that can never happen.

A bag holds $5$ red and $3$ blue balls. Two balls are drawn one after the other, without putting the first back. Given that the first ball is red, what is the chance that the second is red too? After a red ball is removed, $7$ balls are left, and $4$ of them are red. So the answer is $\frac{4}{7}$. The computer listed all $8 \times 7 = 56$ ordered draws and agrees: $\frac{4}{7}$.

• A card is drawn from a standard deck of $52$. Given that it is a face card (jack, queen or king; $12$ cards), the chance that it is a king is $\frac{4}{12} = \frac{1}{3}$.
• Given that the card is red, the chance that it is a heart is $\frac{1}{2}$.
• Given that the card is a heart, the chance that it is red is $1$. A heart is always red.

1. Toss a fair coin $3$ times. Given that the first toss is heads, what is the chance of at least $2$ heads in total?
2. Roll a die. Given that the number is even, what is the chance it is $6$?
3. Draw a card from a deck. Given that it is a spade, what is the chance it is an ace?
4. A bag has $4$ red and $6$ blue balls. Two are drawn without replacement. Given that the first is blue, what is the chance the second is blue?

Answers

1. $\frac{3}{4}$. The four outcomes that start with heads are HHH, HHT, HTH, HTT, and three of them have at least two heads.
2. $\frac{1}{3}$. The even numbers are $2, 4, 6$, and one of the three is a $6$.
3. $\frac{1}{13}$. Each suit has $13$ cards and exactly one ace.
4. $\frac{5}{9}$. Five blue balls remain among $9$.

Once the clue $F$ is known, $F$ acts like a new sample space. So conditional probabilities follow the same rules as ordinary probabilities. The examples below use one card drawn from a deck, with the clue $F$: "the card is red" ($26$ cards). Let $E$ be "the card is a king" and $G$ "the card is a heart". Every value was found by counting.

• Property 1. $0 \le P(E \mid F) \le 1$, and $P(F \mid F) = P(S \mid F) = 1$. Example: $P(E \mid F) = \frac{1}{13}$, and $P(F \mid F) = 1$.
• Property 2 (addition rule). $P\big((E \cup G) \mid F\big) = P(E \mid F) + P(G \mid F) - P\big((E \cap G) \mid F\big)$. Example: $P(E \mid F) = \frac{1}{13}$, $P(G \mid F) = \frac{1}{2}$, and $P\big((E \cap G) \mid F\big) = \frac{1}{26}$ (only the king of hearts). So the right side is $\frac{1}{13} + \frac{1}{2} - \frac{1}{26} = \frac{7}{13}$. Counting the red cards that are kings or hearts ($13 + 1 = 14$ out of $26$) gives $\frac{7}{13}$.
• Property 3 (complement). $P(E' \mid F) = 1 - P(E \mid F)$. Example: the chance that a red card is not a king is $1 - \frac{1}{13} = \frac{12}{13}$, and counting ($24$ out of $26$) gives $\frac{12}{13}$.
• Property 4 (disjoint events). If $E$ and $A$ cannot happen together, then $P\big((E \cup A) \mid F\big) = P(E \mid F) + P(A \mid F)$. Example: let $A$ be "the card is an ace". Kings and aces are disjoint, so $P\big((E \cup A) \mid F\big) = \frac{1}{13} + \frac{1}{13} = \frac{2}{13}$.

1. For a card drawn from a deck, find the chance that it is a jack, queen or king given that it is red, and the chance that it is not one of these, given that it is red. What do the two add up to?

Answers

1. $\frac{3}{13}$ and $\frac{10}{13}$. They add up to $1$, as Property 3 says.

1. Two dice are rolled. Find the probability that the sum is $10$, given that the first die shows $5$.
2. Two dice are rolled. Find the probability that the sum is $7$, given that the two dice show different numbers.
3. A card is drawn from a deck. Given that it is black (a spade or a club), find (a) the probability that it is a spade, (b) the probability that it is a king.
4. A fair coin is tossed $3$ times. Find the probability of exactly $2$ heads, given that at least one toss is heads.
5. A bag has $6$ red and $4$ blue balls. Two are drawn one after the other without replacement. Find the probability that both are red, given that at least one is red.
6. Two dice are rolled. Find the probability that the sum is $9$ or more, given that the first die shows $4$.
7. Two dice are rolled. Let $E$ be "the first die shows $6$" and $F$ "the sum is $9$ or more". Calculate $P(E \mid F)$ and $P(F \mid E)$ and show that they are different.
8. A box holds $20$ items, of which $5$ are faulty. Two items are checked one after the other without replacement. Given that the first is faulty, what is the probability that the second is also faulty?

Answers

1. Given the first die is $5$, there are $6$ equally likely outcomes. Only $(5, 5)$ has sum $10$. So the answer is $\frac{1}{6}$.
2. There are $30$ outcomes with different numbers. Six of them have sum $7$: $(1,6), (2,5), (3,4), (4,3), (5,2), (6,1)$. The answer is $\frac{1}{5}$.
3. (a) $\frac{1}{2}$: $13$ of the $26$ black cards are spades. (b) $\frac{1}{13}$: $2$ of the $26$ black cards are kings.
4. At least one head: $7$ outcomes (all except TTT). Exactly two heads: HHT, HTH, THH. The answer is $\frac{3}{7}$.
5. $P(\text{both red}) = \frac{1}{3}$ and $P(\text{at least one red}) = \frac{13}{15}$. So $P = \frac{1}{3} \div \frac{13}{15} = \frac{5}{13}$.
6. With a $4$ on the first die, the sum is $9$ or more only when the second die is $5$ or $6$. So $\frac{1}{3}$.
7. $F$ has $10$ outcomes. Four of them have a $6$ first: $(6,3), (6,4), (6,5), (6,6)$. So $P(E \mid F) = \frac{2}{5}$. $E$ has $6$ outcomes, and the same $4$ lie in $F$, so $P(F \mid E) = \frac{2}{3}$. They are different.
8. After one faulty item is removed, $19$ items remain and $4$ of them are faulty. The answer is $\frac{4}{19}$. (The computer listed all $20 \times 19 = 380$ ordered pairs and agrees.)

Sometimes you want the chance that two things both happen: first $E$, then $F$. Rearranging the definition $P(F \mid E) = \dfrac{P(E \cap F)}{P(E)}$ gives the multiplication theorem:

$$P(E \cap F) = P(E)\,P(F \mid E), \qquad P(E) \ne 0.$$

In words: the chance of "$E$ and then $F$" is the chance of $E$, times the chance of $F$ once $E$ has happened. Reversing the roles gives $P(E \cap F) = P(F)\,P(E \mid F)$ as well.

A bag has $5$ red and $3$ blue balls. Two balls are drawn without replacement. Find the chance that both are red. The first is red with chance $\frac{5}{8}$. Given that, the second is red with chance $\frac{4}{7}$. So

$$P(\text{both red}) = \frac{5}{8} \times \frac{4}{7} = \frac{5}{14}.$$

Listing all $56$ ordered draws by computer gives the same $\frac{5}{14}$.

The rule extends to more events: $P(E \cap F \cap G) = P(E)\,P(F \mid E)\,P(G \mid E \cap F)$. A bag has $4$ red and $6$ blue balls. Three balls are drawn without replacement. Find the chance of "red, then blue, then red". Multiply: $\frac{4}{10} \times \frac{6}{9} \times \frac{3}{8} = \frac{1}{10}$. Every time a ball is removed, the numbers in the next fraction change. Listing all $720$ ordered draws agrees: $\frac{1}{10}$.

Two cards are drawn from a deck without replacement. The chance that both are aces is $\frac{4}{52} \times \frac{3}{51} = \frac{1}{221}$. The computer listed all $52 \times 51 = 2652$ ordered pairs and agrees: $\frac{1}{221}$.

If the first ball is put back before the second draw, the bag is the same each time. For the bag in Example 1, $P(\text{both red}) = \frac{5}{8} \times \frac{5}{8} = \frac{25}{64} \approx 0.3906$. This is larger than $\frac{5}{14} \approx 0.3571$ from Example 1, because a red ball is still available to be drawn again.

1. Three cards are drawn from a deck without replacement. Find the chance that all three are hearts.
2. A bag has $6$ red and $4$ blue balls. Two balls are drawn without replacement. Find the chance of red first and blue second.
3. A die is rolled twice. Find the chance that the first roll is even and the second roll is more than $4$.
4. A fair coin is tossed three times. Find the chance of three heads.

Answers

1. $\frac{13}{52} \times \frac{12}{51} \times \frac{11}{50} = \frac{11}{850}$.
2. $\frac{6}{10} \times \frac{4}{9} = \frac{4}{15}$.
3. $\frac{1}{2} \times \frac{1}{3} = \frac{1}{6}$. (The second roll does not depend on the first.)
4. $\frac{1}{2} \times \frac{1}{2} \times \frac{1}{2} = \frac{1}{8}$.

Sometimes a clue does not change the chances. Toss a coin and roll a die. Knowing that the coin shows heads tells you nothing about the die. We say these events are independent.

Events $E$ and $F$ are independent when knowing one has happened does not change the chance of the other:

$$P(E \mid F) = P(E).$$

Putting this into the multiplication theorem gives the working test for independence:

$$P(E \cap F) = P(E)\,P(F).$$

(If $P(E) \neq 0$ and $P(F) \neq 0$, the two statements are equivalent.) Events that are not independent are called dependent.

Roll two dice. Let $E$ be "the first die is even" and $F$ "the sum is $7$". Then $P(E) = \frac{1}{2}$, $P(F) = \frac{1}{6}$, and $E \cap F$ has three outcomes, $(2, 5), (4, 3), (6, 1)$, so $P(E \cap F) = \frac{1}{12}$. And $\frac{1}{2} \times \frac{1}{6} = \frac{1}{12}$. The two are equal, so $E$ and $F$ are independent, even though both are about the same dice. Whatever the first die shows, there is always exactly one second-die number that makes the sum $7$.

Keep $E$ = "the first die is even" but let $F'$ now be "the sum is $8$". Then $P(E) = \frac{1}{2}$, $P(F') = \frac{5}{36}$, and $P(E \cap F') = \frac{1}{12}$ (the outcomes $(2, 6), (4, 4), (6, 2)$). But $\frac{1}{2} \times \frac{5}{36} = \frac{5}{72}$, which is not $\frac{1}{12}$. So these events are dependent: an even first die makes a sum of $8$ more likely.

Toss a coin and roll a die ($12$ outcomes). Let $A$ be "heads" and $B$ "the die is even". Then $P(A) = \frac{1}{2}$, $P(B) = \frac{1}{2}$, $P(A \cap B) = \frac{1}{4}$, and $\frac{1}{2} \times \frac{1}{2} = \frac{1}{4}$. Independent, as expected for two separate objects.

In a deck of cards, "the card is a king" and "the card is a heart" are also independent: $P(\text{king}) = \frac{1}{13}$, $P(\text{heart}) = \frac{1}{4}$, and $P(\text{king of hearts}) = \frac{1}{52} = \frac{1}{13} \times \frac{1}{4}$.

Two events are mutually exclusive if they cannot happen together, so $P(E \cap F) = 0$. If both have non-zero probability, then $P(E)\,P(F) \ne 0$, so they cannot be independent. Example: "the card is an ace" and "the card is a face card" are exclusive. The chance of both is 0, but the product $\frac{4}{52} \times \frac{12}{52}$ is not zero. Knowing the card is an ace tells you for certain it is not a face card, so the events are very dependent.

If $E$ and $F$ are independent, so are $E$ and $F'$ (and $E'$ and $F$, and $E'$ and $F'$). In Example 1, the event "the sum is not $7$" is also independent of "the first die is even": the computer confirms it. This matches common sense: if a clue about $F$ does not help, a clue about "not $F$" does not help either.

Three events $A, B, C$ are mutually independent only if every pair is independent and $P(A \cap B \cap C) = P(A)\,P(B)\,P(C)$. Pairwise independence is not enough. Toss two coins. Let $A$ = "first is heads", $B$ = "second is heads", $C$ = "the two coins match". Each pair is independent (each pair has probability $\frac{1}{4} = \frac{1}{2} \times \frac{1}{2}$). But $P(A \cap B \cap C) = \frac{1}{4}$, while $P(A)\,P(B)\,P(C) = \frac{1}{8}$. So the three events are not mutually independent.

1. Two dice are rolled. Are "the first die shows $3$ or $4$" and "the sum is $7$" independent?
2. A card is drawn. Are "it is a heart" and "it is a face card" independent?
3. A coin is tossed $3$ times. Are "at least two heads" and "the first toss is heads" independent?

Answers

1. Yes. $P = \frac{1}{3}$ and $\frac{1}{6}$ for the two events, and $P(\text{both}) = \frac{1}{18} = \frac{1}{3} \times \frac{1}{6}$.
2. Yes. $P(\text{heart}) = \frac{1}{4}$, $P(\text{face}) = \frac{3}{13}$, $P(\text{both}) = \frac{3}{52}$, and the product of the first two is $\frac{3}{52}$.
3. No. $P(E) = \frac{1}{2}$, $P(F) = \frac{1}{2}$, but $P(E \cap F) = \frac{3}{8}$, which is not $\frac{1}{4}$.

1. Two dice are rolled. Are the events "the first die is odd" and "the sum is $8$" independent?
2. Two dice are rolled. Are "the first die shows $3$ or $4$" and "the sum is $7$" independent?
3. A card is drawn from a deck. Are "the card is a heart" and "the card is a face card" independent?
4. A fair coin is tossed $3$ times. Are "at least two heads" and "the first toss is heads" independent?
5. A coin is tossed and a die is rolled. Find the probability of heads and a number greater than $4$.
6. $E$ and $F$ are independent events with $P(E) = \frac{1}{3}$ and $P(F) = \frac{1}{4}$. Find $P(E \cap F)$, $P(E \cup F)$, and the probability that neither happens.
7. A machine has $3$ independent parts, and each works with probability $\frac{9}{10}$. Find the probability that (a) all three work, (b) at least one works.
8. A die is rolled once. Let $E$ be "the number is $1$" and $F$ "the number is $2$". Are $E$ and $F$ mutually exclusive? Are they independent?

Answers

1. No. $P(E) = \frac{1}{2}$, $P(F) = \frac{5}{36}$, and $P(E \cap F) = \frac{1}{18}$ (the outcomes $(3, 5)$ and $(5, 3)$). But $P(E)\,P(F) = \frac{5}{72}$.
2. Yes. $P(E) = \frac{1}{3}$, $P(F) = \frac{1}{6}$, $P(E \cap F) = \frac{1}{18}$ (the outcomes $(3, 4)$ and $(4, 3)$), and $\frac{1}{3} \times \frac{1}{6} = \frac{1}{18}$.
3. Yes. $P(\text{heart}) = \frac{1}{4}$, $P(\text{face card}) = \frac{3}{13}$, and $P(\text{both}) = \frac{3}{52}$, which equals $\frac{3}{52}$.
4. No. $P(E) = \frac{1}{2}$ and $P(F) = \frac{1}{2}$, so the product is $\frac{1}{4}$. But $P(E \cap F) = \frac{3}{8}$.
5. The coin and die are independent: $\frac{1}{2} \times \frac{2}{6} = \frac{1}{6}$. (Listing all $12$ outcomes agrees.)
6. $P(E \cap F) = \frac{1}{3} \times \frac{1}{4} = \frac{1}{12}$. $P(E \cup F) = \frac{1}{3} + \frac{1}{4} - \frac{1}{12} = \frac{1}{2}$. Neither: $1 - \frac{1}{2} = \frac{1}{2}$. Check: $P(E')\,P(F') = \frac{2}{3} \times \frac{3}{4} = \frac{1}{2}$.
7. (a) $\left(\frac{9}{10}\right)^3 = \frac{729}{1000}$. (b) The only way none works is $\left(\frac{1}{10}\right)^3$, so the answer is $1 - \frac{1}{1000} = \frac{999}{1000}$.
8. They are mutually exclusive ($P(E \cap F) = 0$). They are not independent: $P(E)\,P(F) = \frac{1}{36}$ is not $0$. Exclusive events with non-zero probabilities are always dependent.

So far you calculated the chance of an effect when the cause was known. For example: "machine $A$ makes a part, so what is the chance the part is defective?" Real life often goes the other way. You find a defective part and ask: which machine made it?

This is a reverse question. You know $P(\text{defective} \mid \text{machine } A)$, and you want $P(\text{machine } A \mid \text{defective})$. These are not the same number, and you cannot just swap them. Bayes' theorem turns one into the other.

To state it we first need two ideas: a partition of the sample space (a way to split all the possible causes into separate cases), and the theorem of total probability (a way to add up the chance of the effect over all the causes).

A set of events $E_1, E_2, \dots, E_n$ is a partition of the sample space $S$ if:

• the events are complete: together they cover all of $S$, that is $E_1 \cup E_2 \cup \dots \cup E_n = S$;
• the events are separate: no two of them can happen together, that is $E_i \cap E_j = \varnothing$ for $i \ne j$;
• each event is possible: $P(E_i) > 0$ for every $i$.

In other words, every outcome belongs to exactly one of the events. Think of cutting a cake into slices: the slices do not overlap and together they make the whole cake.

• Roll two dice. "the sum is less than $6$", "the sum is $6$", and "the sum is more than $6$" form a partition. Their probabilities are $\frac{5}{18}$, $\frac{5}{36}$ and $\frac{7}{12}$, and they add up to 1.
• "The first die is even" and "the first die is odd" form a partition.
• In a deck of cards, the four suits form a partition. So do "aces" ($\frac{1}{13}$ of the deck), "face cards" ($\frac{3}{13}$) and "number cards from $2$ to $10$" ($\frac{9}{13}$).
• Any event $E$ with $0 < P(E) < 1$, together with its complement $E'$, forms a partition. (Checked for $E$ = "the sum is $8$".)

The probabilities of the events in a partition always add to $1$, because every outcome is counted exactly once.

• "The first die is less than $4$" and "the second die is less than $4$". These overlap: $(1, 2)$ is in both. They also do not cover everything: $(5, 5)$ is in neither.
• "The sum is $7$" and "the sum is $8$". They are separate, but they do not cover all outcomes (for example the sum $5$ is missing).
• "The first die is less than $7$" and "the first die is more than $9$". The second event is impossible, so it is not allowed in a partition.

Roll two dice. Which of these pairs or triples of events are partitions?

1. "the first die is $1$ or $2$", "the first die is $3$ or $4$", "the first die is $5$ or $6$";
2. "the sum is at most $7$", "the sum is at least $7$";
3. "the sum is even", "the sum is odd".

Answers

1. Yes. Each first-die number is in exactly one of the three events.
2. No. They overlap: the sum $7$ is in both.
3. Yes. Every sum is either even or odd, never both.

Let $E_1, E_2, \dots, E_n$ be a partition of $S$, with $P(E_i) > 0$ for each $i$, and let $A$ be any event. Because the $E_i$ are complete and separate, the event $A$ splits into separate pieces $A \cap E_1, A \cap E_2, \dots$. Adding their probabilities and using the multiplication theorem on each piece gives the theorem of total probability:

$$P(A) = P(E_1)\,P(A \mid E_1) + P(E_2)\,P(A \mid E_2) + \dots + P(E_n)\,P(A \mid E_n).$$

In words: the chance of $A$ is a weighted average of its chance in each case, where each weight is the chance of that case.

There are three bags, each holding $5$ balls. Bag $A$ has $3$ red and $2$ blue. Bag $B$ has $1$ red and $4$ blue. Bag $C$ has $4$ red and $1$ blue. One bag is chosen at random and one ball is drawn from it. What is the chance that the ball is red?

The three bags are a partition, each with chance $\frac{1}{3}$. The chance of red inside each bag is $\frac{3}{5}$, $\frac{1}{5}$, $\frac{4}{5}$. So

$$P(\text{red}) = \frac{1}{3} \cdot \frac{3}{5} + \frac{1}{3} \cdot \frac{1}{5} + \frac{1}{3} \cdot \frac{4}{5} = \frac{1}{5} + \frac{1}{15} + \frac{4}{15} = \frac{8}{15}.$$

Listing all $15$ balls (each equally likely to be the one drawn) gives $\frac{8}{15}$ as well.

The tree diagram shows the same calculation. To find the chance of a path, multiply along its branches. To find the chance of an effect, add the paths that lead to it.

Now we can answer the reverse question. If we know the ball is red, how likely is it that it came from bag $C$? Start from the definition of conditional probability, and use the multiplication theorem on the top and total probability on the bottom:

$$P(E_i \mid A) = \frac{P(E_i \cap A)}{P(A)} = \frac{P(E_i)\,P(A \mid E_i)}{\sum_{j=1}^{n} P(E_j)\,P(A \mid E_j)}.$$

This is Bayes' theorem. The numerator is the chance of the path through $E_i$ to $A$. The denominator is the chance of all paths to $A$. So Bayes' theorem says: the new chance of a cause is its share of all the paths that lead to the effect.

For the bags: $P(C \mid \text{red}) = \dfrac{\frac{4}{15}}{\frac{8}{15}} = \frac{1}{2}$. In the same way $P(A \mid \text{red}) = \frac{3}{8}$ and $P(B \mid \text{red}) = \frac{1}{8}$. These three add up to $1$, as they must. Direct counting among the $8$ red balls out of $15$ confirms the value for $C$: $\frac{1}{2}$. Notice that bag $C$ started with chance $\frac{1}{3}$ and, after seeing a red ball, its chance rose to $\frac{1}{2}$.

A factory has three machines. Machine $1$ makes half of all items ($\frac{1}{2}$), machine $2$ makes $\frac{3}{10}$ and machine $3$ makes $\frac{1}{5}$. The fractions of defective items are $\frac{1}{50}$, $\frac{1}{20}$ and $\frac{1}{10}$. An item is chosen at random and is found to be defective. Which machine most likely made it?

The products add up to $P(D) = \frac{9}{200}$, that is 4.5% of all items are defective. Dividing each product by this total gives the last column: $\frac{2}{9}$, $\frac{1}{3}$ and $\frac{4}{9}$. Machine $3$ makes the fewest items, but it makes the most defective ones, so it is the most likely source.

Suppose $1\%$ of the components in a batch are faulty. A test flags $95\%$ of faulty components, but it also flags $4\%$ of good ones by mistake. A component is flagged. What is the chance that it is really faulty? Most people guess about $95\%$. Count in a batch of $10\,000$:

• Faulty components: $100$. The test flags $95$ of them.
• Good components: $9\,900$. The test flags $396$ of them by mistake.
• Total flagged: $95 + 396 = 491$. Only $95$ of these are faulty.

So $P(\text{faulty} \mid \text{flagged}) = \dfrac{95}{491} = \frac{95}{491} \approx 0.1935$, about 19.3%. Bayes' theorem gives the same: $\dfrac{\frac{1}{100} \cdot \frac{19}{20}}{\frac{1}{100} \cdot \frac{19}{20} + \frac{99}{100} \cdot \frac{1}{25}}$. The surprise comes from the fact that good components are far more common, so even a small mistake rate flags many of them.

1. Box $X$ has $4$ red and $6$ blue balls. Box $Y$ has $7$ red and $3$ blue balls. A fair coin decides the box (heads: $X$, tails: $Y$), and a ball is drawn from that box. (a) Find the chance of red. (b) Given that the ball is red, find the chance it came from box $Y$.
2. Production line $P$ makes $70\%$ of the items of a factory and line $Q$ makes $30\%$. Of the items from $P$, $2\%$ are defective, and of the items from $Q$, $5\%$ are defective. (a) Find the chance that a random item is defective. (b) A defective item is found. What is the chance it came from $Q$?
3. A die is rolled. If it shows $1$ or $2$, a ball is drawn from bag $A$ (which has $2$ red and $2$ blue balls). Otherwise a ball is drawn from bag $B$ ($3$ red and $1$ blue). (a) Find the chance of red. (b) Given red, find the chance that bag $A$ was used.
4. $2\%$ of the components in a batch are faulty. A test flags $90\%$ of the faulty components and wrongly flags $5\%$ of the good ones. A component is flagged. What is the chance it is faulty?
5. A die is rolled. If it shows $1$, $2$ or $3$ use bag $X$; if $4$ or $5$ use bag $Y$; if $6$ use bag $Z$. The chance of drawing a red ball is $\frac{3}{4}$ from $X$, $\frac{1}{2}$ from $Y$ and $\frac{1}{4}$ from $Z$. A red ball is drawn. Find the chance it came from $X$.
6. For events $A$ and $B$: $P(A) = \frac{3}{5}$, $P(B \mid A) = \frac{1}{4}$ and $P(B \mid A') = \frac{1}{2}$. Find $P(B)$ and $P(A \mid B)$.
7. In a multiple-choice test, a student knows the answer to a question with chance $\frac{2}{3}$. Otherwise the student guesses among $4$ choices. Find the chance that an answer is correct, and the chance that the student really knew the answer given that it is correct.
8. One coin is fair and the other has heads on both sides. A coin is chosen at random and tossed. It shows heads. What is the chance that it is the two-headed coin?

Answers

1. (a) $\frac{1}{2} \cdot \frac{2}{5} + \frac{1}{2} \cdot \frac{7}{10} = \frac{11}{20}$. (b) $\dfrac{\frac{1}{2} \cdot \frac{7}{10}}{\frac{11}{20}} = \frac{7}{11}$. (Listing all $20$ balls agrees with both.)
2. (a) $\frac{7}{10} \cdot \frac{1}{50} + \frac{3}{10} \cdot \frac{1}{20} = \frac{29}{1000}$, that is 2.9%. (b) $\dfrac{\frac{3}{200}}{\frac{29}{1000}} = \frac{15}{29} \approx 0.517$.
3. (a) $\frac{1}{3} \cdot \frac{1}{2} + \frac{2}{3} \cdot \frac{3}{4} = \frac{2}{3}$. (b) $\dfrac{\frac{1}{6}}{\frac{2}{3}} = \frac{1}{4}$.
4. $\dfrac{\frac{1}{50} \cdot \frac{9}{10}}{\frac{1}{50} \cdot \frac{9}{10} + \frac{49}{50} \cdot \frac{1}{20}} = \dfrac{18}{67}$. The total flagged is 6.7% and the answer is $\frac{18}{67} \approx 0.2687$.
5. $P(\text{red}) = \frac{1}{2} \cdot \frac{3}{4} + \frac{1}{3} \cdot \frac{1}{2} + \frac{1}{6} \cdot \frac{1}{4} = \frac{7}{12}$. Then $P(X \mid \text{red}) = \dfrac{3/8}{\frac{7}{12}} = \frac{9}{14}$.
6. $A$ and $A'$ form a partition, so $P(B) = \frac{3}{5} \cdot \frac{1}{4} + \frac{2}{5} \cdot \frac{1}{2} = \frac{7}{20}$. Then $P(A \mid B) = \dfrac{3/20}{\frac{7}{20}} = \frac{3}{7}$.
7. $P(\text{correct}) = \frac{2}{3} \cdot 1 + \frac{1}{3} \cdot \frac{1}{4} = \frac{3}{4}$. $P(\text{knew} \mid \text{correct}) = \dfrac{2/3}{\frac{3}{4}} = \frac{8}{9}$.
8. $P(\text{heads}) = \frac{1}{2} \cdot \frac{1}{2} + \frac{1}{2} \cdot 1 = \frac{3}{4}$. So the answer is $\dfrac{1/2}{\frac{3}{4}} = \frac{2}{3}$. (Listing the $4$ faces, $3$ of which are heads and $2$ of those on the two-headed coin, agrees.)

Two dice are rolled. Given that the sum is at least $10$, find the chance that at least one die shows $6$. The sum is at least $10$ in $6$ outcomes: $(4, 6), (5, 5), (5, 6), (6, 4), (6, 5), (6, 6)$. Five of them contain a $6$. So the answer is $\frac{5}{6}$.

Two cards are drawn from a deck without replacement. What is the chance that the second card is an ace? Split by the first card. If the first is an ace (chance $\frac{1}{13}$), the second is an ace with chance $\frac{3}{51}$. If not (chance $\frac{12}{13}$), it is $\frac{4}{51}$. Total probability gives $\frac{1}{13} \cdot \frac{3}{51} + \frac{12}{13} \cdot \frac{4}{51} = \frac{1}{13}$. This is the same as the chance that the first card is an ace. (Listing all $2652$ ordered pairs agrees.) Now reverse the question: given that the second card is an ace, the chance that the first was an ace is $\dfrac{\frac{1}{13} \cdot \frac{3}{51}}{\frac{1}{13}} = \frac{1}{17}$.

Two independent alarms detect a fault with chances $\frac{4}{5}$ and $\frac{7}{10}$. Both detect: $\frac{4}{5} \cdot \frac{7}{10} = \frac{14}{25}$. Neither detects: $\frac{1}{5} \cdot \frac{3}{10} = \frac{3}{50}$. At least one detects: $1 - \frac{3}{50} = \frac{47}{50}$. Exactly one detects: $\frac{4}{5} \cdot \frac{3}{10} + \frac{1}{5} \cdot \frac{7}{10} = \frac{19}{50}$. The three cases (both, exactly one, neither) form a partition: $\frac{14}{25} + \frac{19}{50} + \frac{3}{50} = 1$.

Bag $I$ has $3$ red and $2$ blue balls. Bag $II$ has $2$ red and $3$ blue balls. One ball is moved at random from bag $I$ to bag $II$, and then a ball is drawn from bag $II$. Find the chance that it is red. If it is red, what is the chance that the moved ball was red?

The moved ball is red with chance $\frac{3}{5}$. Then bag $II$ has $3$ red out of $6$, so the chance of red is $\frac{1}{2}$. The moved ball is blue with chance $\frac{2}{5}$. Then bag $II$ has $2$ red out of $6$, so the chance is $\frac{1}{3}$. So $P(\text{red}) = \frac{3}{5} \cdot \frac{1}{2} + \frac{2}{5} \cdot \frac{1}{3} = \frac{13}{30}$. By Bayes' theorem, $P(\text{moved red} \mid \text{drawn red}) = \dfrac{3/10}{\frac{13}{30}} = \frac{9}{13}$.

Suppose $P(F) = \frac{1}{2}$, $P(E \mid F) = \frac{1}{5}$ and $P(E) = \frac{1}{4}$. Then $P(E \cap F) = P(F)\,P(E \mid F) = \frac{1}{10}$. Next, $P(E \cup F) = \frac{1}{4} + \frac{1}{2} - \frac{1}{10} = \frac{13}{20}$. And $P(F \mid E) = \dfrac{P(E \cap F)}{P(E)} = \dfrac{\frac{1}{10}}{1/4} = \frac{2}{5}$. Notice that $P(E \mid F) \ne P(F \mid E)$.

1. A fair coin is tossed $3$ times. Given that the three results are not all the same, find the chance of at least two heads.
2. Two cards are drawn from a deck without replacement. Find the chance that both have the same suit.
3. Two dice are rolled. Given that neither die shows a $6$, find the chance that the sum is even.
4. $A$ and $B$ are independent events with $P(A) = \frac{1}{2}$ and $P(A \cup B) = \frac{3}{5}$. Find $P(B)$.
5. A box holds new, used and old batteries in the proportions $\frac{1}{4}$, $\frac{1}{2}$ and $\frac{1}{4}$. They last a full year with chances $\frac{9}{10}$, $\frac{3}{5}$ and $\frac{1}{5}$. A battery picked at random lasts a full year. What is the chance that it was a new one?
6. Two dice are rolled. $E$ is "the first die is even" and $F$ is "the sum is $7$". Show that $E$ and $F'$ ("the sum is not $7$") are independent.
7. A fair coin is tossed $3$ times. Find the chance of three heads, given that the first toss is heads.
8. A bag has $5$ red and $5$ blue balls. Three balls are drawn without replacement. Find the chance that all three have the same colour.

Answers

1. The outcomes that are not all the same: $6$ of the $8$ (all except HHH and TTT). Three of those have exactly two heads (HHT, HTH, THH); none has more. So the answer is $\frac{1}{2}$.
2. Whatever the first card is, $12$ of the remaining $51$ cards have its suit. The answer is $\frac{12}{51} = \frac{4}{17}$. (Listing all $2652$ ordered pairs agrees.)
3. With no $6$, each die shows $1$ to $5$: $25$ outcomes. The sum is even when both are odd ($3 \times 3 = 9$) or both are even ($2 \times 2 = 4$). So $\frac{13}{25}$, that is $\frac{13}{25}$.
4. For independent events, $P(A \cup B) = P(A) + P(B) - P(A)\,P(B) = \frac{1}{2} + \frac{1}{2}P(B)$. So $\frac{1}{2}P(B) = \frac{3}{5} - \frac{1}{2} = \frac{1}{10}$ and $P(B) = \frac{1}{5}$. Check: $\frac{1}{2} + \frac{1}{5} - \frac{1}{10} = \frac{3}{5}$.
5. $P(\text{lasts}) = \frac{9}{40} + \frac{12}{40} + \frac{2}{40} = \frac{23}{40}$. $P(\text{new} \mid \text{lasts}) = \dfrac{9/40}{\frac{23}{40}} = \frac{9}{23}$.
6. $P(E) = \frac{1}{2}$ and $P(F') = \frac{5}{6}$, so the product is $\frac{5}{12}$. $P(E \cap F') = \frac{5}{12}$ by counting ($18$ even-first outcomes minus the $3$ with sum $7$, giving $15$ of $36$). They are equal, so $E$ and $F'$ are independent.
7. Given the first toss is heads, the other two tosses are free: the chance both are heads is $\frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4}$.
8. All red: $\frac{5}{10} \cdot \frac{4}{9} \cdot \frac{3}{8} = \frac{1}{12}$. All blue: the same. Total $\frac{1}{12} + \frac{1}{12} = \frac{1}{6}$. (Listing all $720$ ordered draws agrees.)