Introduction to Probability

When an experiment is repeated under similar and controlled conditions, we typically come across two types of situations:

The outcome is unique or certain.
The outcome is not unique and may vary.

Deterministic Events

These are events where the outcome is predictable with certainty if the initial conditions are known. The same conditions always produce the same result.

Examples:

Calculating the area of a rectangle when length and breadth are known.
Water boiling at 100°C under normal atmospheric pressure.
A ball dropped from a height in a vacuum will fall due to gravity.

Probabilistic Events

These are events where the outcome is uncertain and may vary even under the same conditions. We can only estimate the chances of each possible result.

Examples:

Tossing a fair coin (Outcome: Heads or Tails, each with 0.5 probability).
Rolling a die (Possible outcomes: 1 to 6).
Predicting tomorrow's weather (e.g., 70% chance of rain).

Definitions of Various Terms in Probability

In the study of probability, several key terms help us describe and analyze uncertain events. Below are some fundamental definitions:

1. Trial and Event

When an experiment is repeated under identical or similar conditions but does not produce a unique outcome, it is called a trial.

Each possible outcome of that trial is called an event.

Examples:

(i) Throwing a die is a trial. Getting 1, 2, 3, 4, 5, or 6 is an event.
(ii) Tossing a coin is a trial. Getting a Head (H) or a Tail (T) is an event.
(iii) Drawing two cards from a well-shuffled pack is a trial. Getting a king and a queen is an event.

2. Exhaustive Events (or Exhaustive Cases)

The total number of all possible outcomes of a trial is known as the set of exhaustive events or exhaustive cases.

Examples:

(i) In tossing a coin, there are 2 exhaustive cases: Head and Tail.
(We ignore the rare case where the coin stands on its edge.)
(ii) In throwing a die, there are 6 exhaustive cases: 1, 2, 3, 4, 5, and 6.
(iii) In drawing 2 cards from a deck of 52, the exhaustive number of cases is
$\binom{52}{2} = 1326$ since 2 cards can be chosen in 1326 different ways.
(iv) In throwing two dice, the exhaustive number of cases is
$6 \times 6 = 36$ , as each of the 6 outcomes of the first die can be paired with any of the 6 outcomes of the second die.

In general in throwing of n dice the exhaustive number of cases is 6^n.

Favourable Events (or Cases)

The number of cases favourable to an event is the count of all outcomes that result in the happening of that event.

Examples:

(i) In drawing one card from a deck of 52:
- Number of favourable cases for drawing an ace = 4 (one from each suit).
- Number of favourable cases for drawing a spade = 13.
- Number of favourable cases for drawing a red card = 26 (13 hearts + 13 diamonds).
(ii) In throwing two dice, the favourable cases for getting a sum of 5 are:
(1, 4), (4, 1), (2, 3), (3, 2) → Total = 4 favourable cases.

Mutually Exclusive Events

Two or more events are said to be mutually exclusive (or incompatible) if the occurrence of one event rules out the occurrence of the others in the same trial.

In other words, they cannot happen at the same time.

Examples:

(i) In throwing a die, the outcomes 1, 2, 3, 4, 5, 6 are mutually exclusive. If you get a 3, you cannot get any other number in the same throw.
(ii) In tossing a coin, the outcomes Head (H) and Tail (T) are mutually exclusive. Both cannot occur in a single toss.

Equally Likely Events

Events are equally likely if, given all relevant evidence, there is no reason to expect one outcome more than another.

Examples:

(i) In tossing a fair coin, the outcomes Head and Tail are equally likely (each with probability ½).
(ii) In throwing a fair die, all six faces (1 to 6) are equally likely (each with probability 1⁄6).

Independent Events

Two or more events are independent if the occurrence or non-occurrence of one event does not affect the probability of the other.

Examples:

(i) In tossing a coin multiple times:
- Getting a head in the first toss is independent of the results in the second, third, etc.
(ii) In drawing a card from a well-shuffled deck with replacement:
- The outcome of the second draw is independent of the first.

But, if the first card is not replaced, then the second draw depends on the first — the events are dependent

🎯 Classical (Mathematical / A Priori) Definition of Probability

This definition is used when:

All outcomes of an experiment (trial) are known in advance,
They are mutually exclusive (no two happen at the same time),
And equally likely (all outcomes have the same chance).

Definition:

If an experiment has:

$n$ = Total number of exhaustive, equally likely, and mutually exclusive outcomes,
$m$ = Number of outcomes favourable to an event $E$ ,

Then the probability of event $E$ happening is:

P (E) = \frac{m}{n}

✅ Example 1: Tossing a fair coin

Outcomes: Head (H), Tail (T) → $n = 2$
Favourable outcomes for event “getting Head” → $m = 1$

P(\text{Head}) = \frac{1}{2}

✅ Example 2: Throwing a fair die

Outcomes: 1 to 6 → $n = 6$
Favourable outcomes for event “getting a 4” → $m = 1$

P (4) = \frac{1}{6}

✅ Example 3: Drawing a red card from a well-shuffled deck

Total cards: 52 → $n = 52$
Red cards: 26 → $m = 26$

P(\text{Red}) = \frac{26}{52} = \frac{1}{2}

Random Experiment and Sample Space

🎯 What Is a Random Experiment?

A random experiment is a process or activity that:

Can be repeated under identical conditions, and
Has uncertain outcomes that cannot be predicted exactly in advance.

Examples:

Tossing a coin
Rolling a die
Drawing a card from a shuffled deck
Conducting a scientific/agricultural test (e.g., testing fertilizer effects)

👉 Each repetition of the experiment is called a trial.

🎲 Outcome and Sample Space

The result of a trial is called an outcome or elementary event (also called a sample point).
The set of all possible outcomes is called the sample space.

💡 Examples of Sample Spaces:

Tossing a coin:
Sample space, $S = \{ \text{Head}, \text{Tail} \}$
Rolling a die:
Sample space, $S = \{ 1, 2, 3, 4, 5, 6 \}$
Drawing a card from a deck:
Sample space, $S$ includes all 52 cards

🔍 Formal Definition

Let a random experiment be denoted by $E$ , and let the possible outcomes be:

e_1, e_2, \dots, e_n

These outcomes satisfy the following:

Mutually exclusive – no two outcomes occur at the same time.
Exhaustive – one and only one outcome occurs in each trial.

Then the sample space is the set:

S = \{ e_1, e_2, \dots, e_n \}

This set $S$ is the foundation on which we define probabilities.

🧠 Why It Matters?

We need the idea of a sample space and random experiment because:

They help us model real-world uncertainty.
They allow us to use mathematical tools to calculate probabilities.
They support additive and frequency-based interpretations of probability.

For example:

The probability of getting a Head when tossing a fair coin is $\frac{1}{2}$ .
The probability of getting either a 5 or a 6 when rolling a fair die is:
$\frac{1}{6} + \frac{1}{6} = \frac{1}{3}$

📌 Summary

A random experiment is any process with uncertain results.
An outcome is a single result of a trial.
The sample space is the set of all possible outcomes.
The sample space helps us assign probabilities in a logical and systematic way.

🔹 1. Universal Set

The sample space $S$ acts like a universal set for all outcomes related to a random experiment.
Any event or group of outcomes is a subset of this sample space.

🔹 2. Finite vs Infinite Sample Space

A sample space is called finite if it contains a limited number of outcomes.
- Example: Tossing a coin once
  $S = \{ H, T \}$ → 2 outcomes → finite
A sample space is infinite if it contains endless outcomes.
- Example: Tossing a coin until the first Head appears
  $S = \{ H, TH, TTH, TTTH, \dots \}$ → continues forever → infinite
Each outcome here represents:
- $H$ : Head on first toss
- $TH$ : Tail, then Head
- $TTH$ : Two Tails, then Head
  and so on...

🔹 3. Discrete vs Continuous Sample Space

A discrete sample space contains either:
- A finite number of outcomes, or
- An infinite but countable number of outcomes (like natural numbers)
A continuous sample space has uncountably infinite outcomes, like all real numbers between 0 and 1.
- Example: Measuring exact time, distance, etc.

🧠 Note: In this book, we are dealing only with discrete sample spaces.

📘 4 Event

🔹 What is an Event?

An event is:

Any non-empty subset of the sample space $S$ .
It may consist of one outcome, multiple outcomes, or even no outcome.

✍️ Formal Definition:

“Of all the possible outcomes in the sample space, some may satisfy a specific condition. The set of those outcomes is called an event.”

🔹 Types of Events

✅ Elementary Event:
- Contains only one outcome
- Example: In rolling a die, getting a 4
  $A = \{4\}$
❌ Impossible Event:
- Contains no outcome
- Represented by the empty set: $\emptyset$
- Example: Rolling a die and getting an 8
🎯 Certain Event:
- The event that includes all possible outcomes
- Represented by the sample space itself: $S$
- Always happens when the experiment is performed
🎲 Compound Event:
- Contains more than one outcome
- Example: Getting an even number when rolling a die
  $A = \{2, 4, 6\}$

✅ Summary Table

Term	Meaning	Example
Sample Space (S)	All possible outcomes for a die	$\{1,2,3,4,5,6\}$
Elementary Event	One single outcome	$\{4\}$
Compound Event	More than one outcome	$\{2, 4, 6\}$
Impossible Event	No outcome	$\emptyset$
Certain Event	All outcomes	$S$

🔹 Example 1: Single Toss of a Coin

When you toss a coin once, the possible outcomes are:
- H = Head
- T = Tail
So the Sample Space is:
$S = \{H, T\}$
The number of sample points is:
$n(S) = 2$
Each of these outcomes (H and T) is an elementary event.

🔹 Example 2: Two Tosses of a Coin

Now we toss a coin two times.
The possible outcomes are:
- First toss: H or T
- Second toss: H or T
- So the combinations are:
  - HH: Head in both tosses
  - HT: Head then Tail
  - TH: Tail then Head
  - TT: Tail in both tosses
Thus, the Sample Space is:
$S = \{HH, HT, TH, TT\}$
Total number of outcomes:
$n(S) = 4$

🔹 Example Event: Getting at Least One Head

To form an event, we select a subset of the sample space.
Let’s define the event:

A = Getting at least one Head
The outcomes that satisfy this condition are:
- HH
- HT
- TH
So, the event A is:
$A = \{HH, HT, TH\}$
Note:
- TT is not included in A because it has no Head.
- This event is a compound event because it has multiple outcomes.

🧠 Key Takeaway:

A sample space lists all possible outcomes of an experiment.
An event is any subset of that sample space.

✅ Algebra of Events (Set Operations on Events)

Let A, B, C be events from a sample space S. The following rules apply:

🔹 (i) Union of Events

A \cup B = \{\omega \in S : \omega \in A \text{ or } \omega \in B\}

🔸 Meaning: The event that either A or B or both occur.

🔹 (ii) Intersection of Events

A \cap B = \{\omega \in S : \omega \in A \text{ and } \omega \in B\}

🔸 Meaning: The event that both A and B occur together.

🔹 (iii) Complement of an Event A

A^c = \{\omega \in S : \omega \notin A\}

🔸 Meaning: The event that A does not occur.

🔹 (iv) Difference of Events (A minus B)

A - B = \{\omega \in S : \omega \in A \text{ but } \omega \notin B\}

🔸 Meaning: Outcomes that are in A only, not in B.

🔹 (v) Generalizations

For a collection of events $A_1, A_2, A_3, \dots, A_n$ :
- Union:
  $\bigcup_{i=1}^{n} A_i = A_1 \cup A_2 \cup \dots \cup A_n$
  Meaning: At least one of the events occurs.
- Intersection:
  $\bigcap_{i=1}^{n} A_i = A_1 \cap A_2 \cap \dots \cap A_n$
  Meaning: All of the events occur together.

🔹 (vi) Subset Relation

A \subseteq B \iff \text{For every } \omega \in A, \omega \in B

🔸 Meaning: Every outcome in A is also in B.

🔹 (vii) Superset Relation

B \supseteq A \iff A \subseteq B

🔹 (viii) Equality of Sets

A = B \iff A \subseteq B \text{ and } B \subseteq A

🔸 Meaning: A and B contain exactly the same outcomes.

🔹 (ix) Disjoint Events (Mutually Exclusive)

A \cap B = \emptyset

🔸 Meaning: A and B cannot occur together.

🔹 (x) Alternate Notation for Disjoint Union

If A and B are disjoint, then:

A \cup B = A + B

🔹 (xi) Symmetric Difference (Exactly One of A or B)

A \triangle B = (A \cup B) - (A \cap B)

or equivalently:

A \triangle B = (A - B) \cup (B - A)

🔸 Meaning: Outcomes that are in A or B but not both.

Remark. Since the events are subsets of S, all the laws of set theory viz., commutative laws. associative laws. distributive laws, DeMorgan's law. etc., hold for aIgebra of events.

✅ Given: Three arbitrary events A, B, and C

We are to express certain compound events in terms of set operations (intersection ∩, union ∪, complement Aᶜ, etc.).

🔹 (i) Only A occurs

This means A occurs, but B and C do not.

$\boxed{A \cap B^c \cap C^c}$

🔹 (ii) Both A and B occur, but not C

$\boxed{A \cap B \cap C^c}$

🔹 (iii) All three events occur

$\boxed{A \cap B \cap C}$

🔹 (iv) At least one occurs

At least one of A, B, or C occurs means the union of all three.

$\boxed{A \cup B \cup C}$

🔹 (v) At least two occur

This means any two or all three occur. So we combine the three pairwise intersections excluding cases where only one occurs:

$\boxed{ (A \cap B \cap C^c) \cup (A \cap B^c \cap C) \cup (A^c \cap B \cap C) \cup (A \cap B \cap C) }$

🔹 (vi) Only one occurs

This means exactly one of A, B, or C occurs, and the others do not:

$\boxed{ (A \cap B^c \cap C^c) \cup (A^c \cap B \cap C^c) \cup (A^c \cap B^c \cap C) }$

🔹 (vii) Exactly two occur (and no more)

Here, any two occur and the third does not:

$\boxed{ (A \cap B \cap C^c) \cup (A \cap B^c \cap C) \cup (A^c \cap B \cap C) }$

🔹 (viii) None occurs

This means A, B, and C all do not occur. So:

$\boxed{A^c \cap B^c \cap C^c}$

Or alternatively, the complement of “at least one occurs”:

$\boxed{(A \cup B \cup C)^c}$

📘 Mathematical Notion of Probability

When performing a random experiment, the sample space $S$ consists of all possible outcomes.

Let:

$N$ = total number of trials (or sample points),
$N_A$ = number of times event A occurs.

🔹 Frequency Interpretation of Probability

In the long run (as $N \to \infty$ ), the probability of event A, denoted $P(A)$ , is approximated by the relative frequency:

$\boxed{P(A) = \frac{N_A}{N}}$

This gives an empirical (frequentist) view of probability.

📘 Mathematical (Axiomatic) Definition of Probability

Since the frequency definition depends on observations and can't define probabilities purely mathematically, we need a formal definition.

We define a function $P$ called the probability function on a sample space $S$ , satisfying certain axioms.

Let:

$\mathcal{D}$ = a σ-field (sigma-field or sigma-algebra) of subsets of $S$ , i.e., the collection of all events,
$A \in \mathcal{D}$ = any event (subset of S).

📌 Axioms of Probability (Kolmogorov's Axioms)

The probability function $P$ must satisfy:

Non-Negativity (Positiveness)
$\boxed{P(A) \geq 0} \quad \text{for all } A \in \mathcal{D}$
Normalization (Certainty)
$\boxed{P(S) = 1}$
That is, the probability that some outcome occurs is 1.
Countable Additivity (Union Axiom)
If $A_1, A_2, A_3, \dots \in \mathcal{D}$ are mutually disjoint (i.e., $A_i \cap A_j = \emptyset$ for $i \neq j$ ), then:
$\boxed{P\left(\bigcup_{i=1}^{\infty} A_i\right) = \sum_{i=1}^{\infty} P(A_i)}$

✅ Summary of Axioms

Axiom	Name	Meaning
1. $P(A) \geq 0$	Positiveness	Probability is never negative.
2. $P(S) = 1$	Certainty	The entire sample space always has probability 1.
3. Additivity	Countable Additivity	Probabilities of disjoint events add up correctly.

From Frequency to Axioms of Probability

Let’s assume a random experiment repeated $N$ times, with:

$N_A$ : number of times event $A$ occurs
Then, the probability of event A is:

\boxed{P(A) = \frac{N_A}{N}}

Now, let’s see how this definition satisfies the three axioms of probability.

✅ Axiom 1: Non-negativity

Since $N_A \geq 0$ , we get:

\boxed{P(A) = \frac{N_A}{N} \geq 0}

✅ So the probability is never negative.

✅ Axiom 2: Normalization

The sample space $S$ includes all possible outcomes. So the number of outcomes in $S$ is:

N_S = N \Rightarrow P(S) = \frac{N}{N} = \boxed{1}

✅ The total probability of the sample space is 1.

✅ Axiom 3: Additivity (for disjoint events)

Let $A$ and $B$ be mutually exclusive (disjoint) events:

A \cap B = \emptyset

Let:

$N_A$ = number of times $A$ occurs
$N_B$ = number of times $B$ occurs
$N_{A \cup B} = N_A + N_B$

Then:

P(A \cup B) = \frac{N_A + N_B}{N} = \frac{N_A}{N} + \frac{N_B}{N} = \boxed{P(A) + P(B)}

✅ So, probabilities of disjoint events add.

📘 Extended Axiom of Addition (Countable Additivity)

If an event $A$ can happen by the occurrence of any one of countably many mutually disjoint events $A_1, A_2, A_3, \dots$ i.e.,

A = \bigcup_{i=1}^{\infty} A_i \quad \text{with } A_i \cap A_j = \emptyset \text{ for } i \neq j

Then,

\boxed{P(A) = \sum_{i=1}^{\infty} P(A_i)}

This is the generalization of Axiom 3 to infinitely many disjoint events, and is a foundation of modern probability theory.

THEOREMS ON PROBABILITIES OF EVENTS

Theorem:Probability of the Impossible Event is Zero

🔷 Statement:

$\boxed{P(\emptyset) = 0}$

Theorem: Probability of the Complementary Event

🔷 Statement:

If $A$ is an event in the sample space $S$ , and $A^{'}$ (or $\overline{A}$ ) denotes the complement of $A$ , then:

\boxed{P(A') = 1 - P(A)}

Or equivalently:

\boxed{P(A) = 1 - P(A')}

A \cup B