ECE600 F13 probability spaces mhossain - Rhea

Random Variables and Signals

Topic 2: Probability Spaces

Definitions and Notation

Definition $\qquad$ An outcome is a possible result of a random experiment. Different outcomes are mutually exclusive; only one outcome can occur on each trial. The collection of all possible outcomes forms the sample space. e.g. When you roll a six-sided fair die and record which surface faces up, the sample space is S = {1,2,3,4,5,6} e.g. when you flip two coins, the sample space is S = {HH,HT,TH,TT}

Definition $\qquad$ An event is a set of outcomes of an experiment to which a probability is assigned. A single outcome can be an element in several different outcomes. E.g. in the die rolling experiment, the set {1,2} describes the event that a number less than three is rolled. The set {2,4,6} describes the event that an even number is rolled. The number 2 is a possible outcome in both events Typically, when S is finite, we can use to the power set of S, denoted P(S) as the set of all events. Then, an event is a subset of S. If S contains n outcomes, then the power set contains $$ 2^n $$ events including the empty set and S itself. This is not true however for all S. We will examine this in more detail in our discussion of event spaces. An event may contain only one outcome. This event is then called an elementary event. It is important to be able to differentiate between an event and an outcome. The event {1} is not the same thing as the outcome 1.

The probability space is a mathematical construct built to model a random experiment. It consists of three parts:

The set of all possible outcomes, the sample space, S.
The set of events F or F(S), which is the collection of events to which we will assign probabilities
$$ P $$ the function mapping probabilities to the events.

The Sample Space S

The sample space is a non empty set of elements called outcomes. We will call the sample space S. Note that S is not required to have any structure. This allows us to model any experiment as long as it has at least one outcome. The sample space may be discrete. In that case, S may be finite or countable. For example Experiment: Toss a coin ⇒ S={H,T} Experiment: count cars passing a certain point ⇒ S={0,1,2,...} In either case, we can list the elements of S as $\{\omega_1,\omega_2,...\}$

The sample space may be continuous such that S is uncountable. For example, in the experiment measuring the temperature in a room, S = [ $T_{min},T_{max}$ ], where S is a closed interval on the real number line.

Note that our examples cover each type of set, finite, countable and uncountable respectively.

The Event Space

The event space F or script F is the collection of subsets of S to which we will assign probabilities. Not all collections of subsets of S are valid event spaces. We require the event space to be a nonempty collection of subsets of S satisfying the following properties:
1. If $$ A $$ ∈F, then $$ A^c $$ F(S). 2. If for finite n, $$ A_i $$ ∈F, for all i = 1,2,...n, then

\bigcup_{i=1}^n A_i \in \mathcal{F}

3. If $$ A_i $$ ∈F, for all i = 1,2,..., then

\bigcup_{i=1}^{\infty} A_i \in \mathcal{F}

Property 1 ensures that if we assign a probability to the occurrence of event $$ A $$ , then there will also be a probability of event $$ A $$ not occurring. Properties 2 and 3 ensure that we can find the probability that at least on of a set of events occurs if each event has a probability associated with it.

In general, a collection of sets satisfying the properties above is referred to as a $\sigma$ -field. A nonempty collection of sets satisfying properties 1 and 2 only is called a field. Another term used for $\sigma$ -field in math is $\sigma$ -algebra.

Note: it can be shown that property 3 implies property 2, so it is sufficient to show only properties 1 and 3 (in addition to the nonempty requirement) to show that a collection is a $\sigma$ -field.

Some properties of a $\sigma$ -field (and hence our event space):
1. for any $\sigma$ -field F, Ø ∈ F and S ∈ F
Proof: let $$ A $$ ∈ F (we know such an $$ A $$ exists because F is nonempty). Then $$ A^c $$ ∈ F by property 1 of $\sigma$ -fields and $$ A $$ ∪ $$ A^c $$ ∈ F. So S∈F, and by property 1, $$ S^c $$ = Ø ∈ F.

2. If $$ A,B $$ ∈ F, then $$ A $$ ∩ $$ B $$ ∈F.
Proof: by deMorgan's law,

\begin{align} A\cap B &= ((A\cap B)^c)^c \\ &= (A^c \cup B^c)^c \end{align}

but if $$ A,B $$ ∈ F, then $$ A^c, B^c $$ ∈ F so ( $$ A^c $$ ∪ $$ B^c $$ ) ∈ F and $$ (A^c $$ ∪ $$ B^c)^c $$ ∈ F. Thus $$ A $$ ∩ $$ B $$ ∈ F.
One can also show that

A_i\in\mathcal{F}\;\forall i=1,2,...\;\; \Rightarrow \bigcap_{i=1}^{\infty}A_i \in \mathcal{F}

Examples of event spaces:

F = {Ø,S} is a valid (though not very interesting) event space. It is the smallest possible space since F must be non empty.
For any space S, the collection of all subsets is S is a valid $\sigma$ -field. This set is called the power set denoted

\mathcal{P(S)} \; or \;2^{\mathcal{S}}

For a finite or countable S, the power set is a valid event space, although it might not always be the best choice. However, for an uncountable S, the power set can be problematic.

Event Space for an Uncountable Sample Space

There are some rules we will impose on our probability mappings (to be discussed shortly in the section on probability mapping). It can be shown that if S is uncountable, we cannot satisfy those rules if the event space is the power set of S. We will construct an event space S in the reals, to address this problems. We will not address the general case pf uncountable sample spaces, but the construction here will be easily adaptable is S is an interval on the real number line or if S=R $$ ^n $$ for finite n.

Definition $\qquad$ given S, consider a family of subsets of S, G, where

G = \{A_i, \; A_i \subset \mathcal{S}\;\forall i \in I\}

Note that G is not necessarily a $\sigma$ -field. The $\sigma$ -field generated by G, denoted $\sigma(G)$ , is the smallest $\sigma$ -field containing every element of G. In other words, for any $\sigma$ -field $\mathcal{G}$ containing G,

\sigma(G) \subset \mathcal{G}

We can also write this as

\sigma(G) = \bigcap_{i\in I_G} \mathcal{F}_i

where { $F_i,<\math> <math>i$ ∈ $$ I_G $$ } is the set of all $\sigma$ -fields containing G.
When S is the set of reals,, we will let G = {open intervals in the set of reals}; ie

G = \{(a,b): \; a\in\mathbb{R},\; b\in\mathbb{R},\; a<b\}

then let

,math>\mathcal{F} = \sigma(g)</math>

The $\sigma$ -field $\sigma(G)$ , where G is the collection is all open intervals in the set of reals, is called the Borel field. We will always use the Borel field as our event space if S is the set of all real and denote it as

B(\mathbb{R})

So,

\mathcal{S} = \mathbb{R} \; \Rightarrow \;\mathcal{F}=B(\mathbb{R})

Note: We will also refer to

B(\mathbb{R}^n),\; B((a,b)), \; B([a,b])

where [a,b] is a real intervals, to be respective event spaces when the sample space is

\mathbb{R}^n,\; (a,b), \;and\; [a,b]

Note that the collection of all open intervals in the set of reals, which we are calling G, is not a $\sigma$ -field. So what does $\sigma(G)$ contain besides open intervals? Some others include: $\bullet\;\;\mathcal{S} = \bigcup_{n=1}^{\infty}(-n,n)\;\in\;B(\mathbb{R})$
$\bullet\;\;\varnothing = \mathcal{S}^c\in B(\mathbb{R})$
$\bullet\;\;\forall \; a,b\in\mathbb{R}, \; a<b$

\begin{align} &\bullet\;\;(-\infty,b)=\bigcup_{n=1}^{\infty}(-n,b)\in B(\mathbb{R}) \\ &\bullet\;\;(a,\infty) = \bigcup_{n=1}^{\infty}(a,n) \in B(\mathbb{R}) \\ &\bullet\;\;\{a\} = \bigcap_{n=1}^{\infty}(a-\frac{1}{n},a+\frac{1}{n}) \in B(\mathbb{R}) \\ &\bullet\;\;(-\infty,b] = (-\infty,b)\cup \{b\}\in B(\mathbb{R}) \\ &\bullet\;\;[a,b] = (a,b)\cup \{a\}\cup\{b\}\in B(\mathbb{R}) \\ &\bullet\;\;\mathbb{Q} = \bigcup_{n=1}^{\infty} \{q\};\;q_n\;is\;the\;n^{th}\;rational\;\Rightarrow \mathbb{Q}\in B(\mathbb{R}) \\ \end{align}

Note that

\begin{align} b\leq -n \; &\Rightarrow \;(-n,b) = \varnothing \\ a\geq n \; &\Rightarrow \;(a,n) = \varnothing \\ \end{align}

Note that all of these sets are in the Borel field because the Borel field is a $\sigma$ -field.

Probability Mapping