Essential Probability Theory
1

Probability theory : Terminology
Sample space
𝛀
:
𝛀
is the set of all possible outcomes
𝑒
of an experiment.
•
Example 1. In tossing of a die we have
𝛀 = {1, 2, 3, 4, 5, 6}.
•
Example 2. The life-time of a bulb
𝛀 = {? ∈ 𝑅 | ? > 0}.
Event:
An event is a subset of the sample space
𝛀
. An event is usually denoted by a capital letter
?, ?, ⋯ .
If the outcome of an experiment is a member of event
?
, we say that
?
has occurred.
An impossible event is an empty subset
∅
of
𝛀
.
•
Example 1. The outcome of tossing a die is an even number:
? = {2, 4, 6} ⊂ 𝛀.
•
Example 2. The life-time of a bulb is at least 3000 hours:
? = {? ∈ 𝑅 | ? > 3000} ⊂ 𝛀.
Probability theory is a subset of measure theory. Probability makes extensive use of set operations, so
let us introduce the more relevant notations.
2

Algebra of events (Boolean Algebra)
Union:
“
?
or
?
”.
? ∪ ? = {𝑒 ∈ 𝛀 | 𝑒 ∈ ? or 𝑒 ∈ ?}
Intersection:
(joint event) “A and B”.
? ∩ ? = {𝑒 ∈ 𝛀 | 𝑒 ∈ ? and 𝑒 ∈ ?}
Events
?
and
?
are mutually exclusive
, if
? ∩ ? = ∅
.
Complement:
“not
?
”.
? = ?
𝑐
= {𝑒 ∈ 𝛀 | 𝑒 ∉ ?}
Partition of the sample space
A set of events
?
1
, ?
2
, ⋯
is a partition of the sample space
𝛀
if
1.
The events are mutually exclusive,
?
?
∩ ?
?
= ∅
, when
? ≠ ?
.
2.
Together they cover the whole sample space,
ڂ
?
?
?
= 𝛀
.
3

Algebra of events (continued)
In many ways the algebra of events differs from the algebra of numbers, as some of the identities below
indicate.
? ∪ ? = ?,
? ∪ ?
𝑐
= 𝛀,
? ∩ ? = ?,
? ∩ ?
𝑐
= ∅
? ∩ 𝛀 = A,
A ∪ ∅ = ?,
? ∪ 𝛀 = 𝛀,
? ∩ ∅ = ∅
A method of verifying relations among events involves algebraic manipulation, using the identities. Four
examples are given below; in working out the identities, it may be helpful to write
? ∪ ?
as
? + ?
and
? ∩ ?
as
??.
1.
? ∪
? ∩ ?
= ?
Proof
.
? + ?? = ?𝛀 + ?? = ?
𝛀 + ?
= ?𝛀 = ?
2.
? ∪ ?
∩
? ∪ ?
= ? ∪
? ∩ ?
Proof
.
? + ?
? + ?
=
? + ? ? +
? + ? ? = ?? + ?? + ?? + ??
note ?? = ??
= ? + ?? + ?? + ?? = ?
𝛀 + ? + ?
+ ?? = ?𝛀 + ?? = ? + ??
3.
? ∪
? ∩ ?
𝑐
= 𝛀
Proof
.
? + (??)
𝑐
= ? + ?
𝑐
+ ?
𝑐
=
? + ?
𝑐
+ ?
𝑐
= 𝛀 + ?
𝑐
= 𝛀
4.
? ∩ ?
𝑐
∪
? ∩ ?
∪
?
𝑐
∩ ?
= ? ∪ ?
Proof
.
??
𝑐
+ ?? + ?
𝑐
? = ??
𝑐
+ ?? + ?? + ?
𝑐
? = ?
?
𝑐
+ ?
+
? + ?
𝑐
? = ?𝛀 + 𝛀? = ? + ?
4

Algebra of events (continued)
Example:
Let
𝛀
be the set of nonnegative real numbers.
Let
?
𝑛
=
0,1 −
1
𝑛
=
? ∈ 𝛀 ∶ 0 ≤ ? ≤ 1 −
1
𝑛
𝑛 = 1,2, …
Then,
ራ
𝑛=1
∞
?
𝑛
=
0,1
=
?: 0 ≤ ? < 1 ,
ሩ
𝑛=1
∞
?
𝑛
= {0}
As an illustration of the DeMorgan laws,
ራ
𝑛=1
∞
?
𝑛
𝑐
= [0,1)
𝑐
=
1, ∞
=
?: ? ≥ 1 ,
ሩ
𝑛=1
∞
?
𝑛
𝑐
= ሩ
𝑛=1
∞
1 −
1
𝑛
, ∞
=
1, ∞
(Notice that
? > 1 − 1/𝑛
for all
𝑛 = 1,2, … ⇔ ? ≥ 1).
Also
ሩ
𝑛=1
∞
?
𝑛
𝑐
= {0}
𝑐
=
0, ∞
=
?: ? > 0 ,
ራ
𝑛=1
∞
?
𝑛
𝑐
= ራ
𝑛=1
∞
1 −
1
𝑛
, ∞
= (0, ∞)
5

Probability model
Probability model
: A probabilistic model is a mathematical description of an uncertain situation.
A probability model has three components:
•
A
sample space
𝛀
, which is the set of all possible outcomes of the experiment that is modelled
by the probability model;
•
A family of sets
𝓕
representing the
class of allowable events
, where each set is a subset of the
sample space
𝛀
; and
•
A
probability function
𝑃: 𝓕 ⟼ 𝓡[0,1]
which, also called as a
probability measure
,
satisfies some probability axioms given in the next slide.
Definition
: A probability space is a triplet (
𝛀, 𝓕, 𝑃
).
We can also consider more general measures
𝜇
, not just probability measures, in which case (
𝛀, 𝓕, 𝜇
)
is called a
measure space
. You probably have some familiarity with
𝛀
and
𝑃
, but not necessarily with
𝓕
because you may not have been aware of the need to restrict the class (set) of events you could
consider. In a given problem, there will be a particular class of subsets of
𝛀
called as
class of events
.