Random Variables and Signals
Topic 2: Probability Spaces
Definitions and Notation
Definition $ \qquad $ An outcome is a possible result of a random experiment. Different outcomes are mutually exclusive; only one outcome can occur on each trial. The collection of all possible outcomes forms the sample space. e.g. When you roll a six-sided fair die and record which surface faces up, the sample space is S = {1,2,3,4,5,6} e.g. when you flip two coins, the sample space is S = {HH,HT,TH,TT}
Definition $ \qquad $ An event is a set of outcomes of an experiment to which a probability is assigned. A single outcome can be an element in several different outcomes. E.g. in the die rolling experiment, the set {1,2} describes the event that a number less than three is rolled. The set {2,4,6} describes the event that an even number is rolled. The number 2 is a possible outcome in both events Typically, when S is finite, we can use to the power set of S, denoted P(S) as the set of all events. Then, an event is a subset of S. If S contains n outcomes, then the power set contains $ 2^n $ events including the empty set and S itself. This is not true however for all S. We will examine this in more detail in our discussion of event spaces. An event may contain only one outcome. This event is then called an elementary event. It is important to be able to differentiate between an event and an outcome. The event {1} is not the same thing as the outcome 1.
The probability space is a mathematical construct built to model a random experiment. It consists of three parts:
- The set of all possible outcomes, the sample space, S.
- The set of events F or F(S), which is the collection of events to which we will assign probabilities
- $ P $ the function mapping probabilities to the events.
The Sample Space S
The sample space is a non empty set of elements called outcomes. We will call the sample space S. Note that S is not required to have any structure. This allows us to model any experiment as long as it has at least one outcome. The sample space may be discrete. In that case, S may be finite or countable. For example Experiment: Toss a coin ⇒ S={H,T} Experiment: count cars passing a certain point ⇒ S={0,1,2,...} In either case, we can list the elements of S as $ \{\omega_1,\omega_2,...\} $
The sample space may be continuous such that S is uncountable. For example, in the experiment measuring the temperature in a room, S = [$ T_{min},T_{max} $], where S is a closed interval on the real number line.
Note that our examples cover each type of set, finite, countable and uncountable respectively.
The Event Space
The event space F or script F is the collection of subsets of S to which we will assign probabilities. Not all collections of subsets of S are valid event spaces. We require the event space to be a nonempty collection of subsets of S satisfying the following properties:
1. If $ A $∈F, then $ A^c $ F(S).
2. If for finite n, $ A_i $∈F, for all i = 1,2,...n, then
3. If $ A_i $∈F, for all i = 1,2,..., then
Property 1 ensures that if we assign a probability to the occurrence of event $ A $, then there will also be a probability of event $ A $ not occurring. Properties 2 and 3 ensure that we can find the probability that at least on of a set of events occurs if each event has a probability associated with it.
In general, a collection of sets satisfying the properties above is referred to as a $ \sigma $-field. A nonempty collection of sets satisfying properties 1 and 2 only is called a field. Another term used for $ \sigma $-field in math is $ \sigma $-algebra.
Note: it can be shown that property 3 implies property 2, so it is sufficient to show only properties 1 and 3 (in addition to the nonempty requirement) to show that a collection is a $ \sigma $-field.
Some properties of a $ \sigma $-field (and hence our event space):
1. for any $ \sigma $-field F, Ø ∈ F and S ∈ F
