Scribd
Upload
11 views

STA 412 stochastic process

This document introduces stochastic processes, defining them as families of random variables indexed by time. It explains the concepts of sample spaces, random variables, and the classification of stochastic processes based on their time and state spaces. Examples illustrate discrete and continuous processes in various contexts, such as consumer behavior and population growth.

Uploaded by

Immortal David Onyedikachi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
11 views

STA 412 stochastic process

This document introduces stochastic processes, defining them as families of random variables indexed by time. It explains the concepts of sample spaces, random variables, and the classification of stochastic processes based on their time and state spaces. Examples illustrate discrete and continuous processes in various contexts, such as consumer behavior and population growth.

Uploaded by

Immortal David Onyedikachi
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Lecture 1:

Introduction to Stochastic Processes


Introduction
Stochastic modelling is an interesting and challenging area of probability and statistics. Our aims
in this introductory section of the notes are to explain what a stochastic process is and what is
meant by the Markov property, give examples and discuss some of the objectives that we might
have in studying stochastic processes.

We begin with a:
• quick revision of sample spaces and random variables;
• formal definition of stochastic processes.
Revision: Sample spaces and random variables
Definition: A random experiment is a physical situation whose outcome cannot be predicted
until it is observed.

Definition: A sample space, Ω, is a set of possible outcomes of a random experiment.

Example: Random experiment: Toss a coin once.


Sample space: Ω = {head, tail}

Definition: A random variable, X, is definedt as a function from the sample space to the real
numbers: X : Ω → R. That is, a random variable assigns a real number to every possible
outcome of a random experiment.
➢ The random variable assigns a numerical value for each of the outcomes in the
sample space of a probability experiment.

An example of a random variable: X : Ω → R maps “head” → 1, “tail” → 0.


For example, if we toss three coins and the random variable x represents the number
of tails that occur, then the only possible values it can assume are x = 0, 1, 2, 3.

Sample space: Ω = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

Stochastic Processes
➢ Definition: A stochastic process is a family of random variables, {X(t) : t ∈ T}, where t
denotes time. That is, at every time t in the set T, a random number X(t) is observed.
Definition:
➢ {X(t) : t ∈ T} is a discrete-time process if the set T is finite or countable.
In practice, this generally means T = {0, 1, 2, 3, . . .}
Thus a discrete-time process is {X(0), X(1), X(2), X(3), . . .}: a random number associated with
every time 0, 1, 2, 3, . . .
Definition:
➢ {X(t) : t ∈ T} is a continuous-time process if T is not finite or countable.
In practice, this generally means T = [0, ∞).
Thus a continuous-time process {X(t) : t ∈ T} has a random number X(t) associated with every
instant in time.
(Note that X(t) need not change at every instant in time, but it is allowed to change at any time;
i.e. not just at t = 0, 1, 2, . . . , like a discrete-time process.)
Definition:
➢ The state space, S, is the set of real values that X(t) can take. That is, the state space S is
the set of states that the stochastic process can be in. It is the set of possible values which
the random variables, X(t) : t ∈ T may assume.
Definition:
➢ The state space S is discrete if it is finite or countable. Otherwise, it is continuous.
➢ Example 1: The state of the process at time t is Xt = the number of animals with allele A
at generation t.
Each Xt could be 0, 1, 2, . . . , N. The state space is {0, 1, 2, . . ., N}.
Example 2: if Xt = 6, we say the process is in state 6 at time t.
Example 3:
Maryland and Lekki are the two chains of cafes in Lagos. John travels to Lagos and decides to try
out Lekki. John then falls into a habit: if he has been to Lekki there is a 5/6 chance he chooses to
go to Maryland on his next trip to Lagos, and a 1/6 chance he will go back to Lekki. Alternatively
if John has been to Maryland, then there is a 1/3 chance that he decides to go Lekki on his next
day out in Lagos, and a 2/3 she returns to Maryland.
John’s behaviour can be modeled by a stochastic process.
➢ S ={Lekki, Maryland} is the state space, and
➢ Xt indicate whether John goes to Lekki or Maryland on his tth subsequent trip to Lagos.
Stochastic processes are classified by their
➢ state space, and
➢ time index,
The state space and time can be discrete or continuous. We shall see examples of all four
combinations (discrete/continuous time in conjunction with discrete/continuous state space)

Classification of Stochastic Processes


1. Stochastic Processes with Discrete time and State Spaces
Example 1: A Brand-Switching Model for Consumer Behavior
Suppose that consumer preferences on three brands on sales, say A, B, C. are observed on a
monthly basis. Then we have a discrete-time, discrete-state stochastic process.
Example 2: A Queueing Problem
Consider the number of students waiting at the time of arrival of a bus when time is counted by
the number of times the bus arrives. Then we again have a discrete time, discrete-state
stochastic process.
➢ If both T and S are discrete, the stochastic process is called Discrete random process
2. Stochastic Processes with Continuous time and Discrete State Space
Example 3: consider the number of students waiting for a bus at any time of day – in this case
the parameter space is continuous and we speak about a continuous-time, discrete state
stochastic process. If both T and S are discrete, the stochastic process is called Discrete random
process
➢ If T continuous and S is discrete, the stochastic process is called Discrete random
process
Example 4: A Population Growth
Consider the size of a population at a given time – we have a continuous-time, discrete state
stochastic process (the population is finite).

3. Stochastic Processes with Discrete time and Continuous State Space

Example 5. Stock Market


Consider the values of the Dow-Jones Index at the end of the nth week. Then we have a
discrete-time stochastic process with the continuous state space (0,∞).

Example 6: consider waiting time of the n-th student arriving at a bus stop – in this case we also
have a continuous-state stochastic process.
➢ If T discrete and S is continuous, the stochastic process is called continuous random
process

4. Stochastic Processes with Continuous time and State Spaces


Example: If Xt represents the maximum temperature of a place in the interval (0,t),
{X(t)} is a random process.

➢ If both Tand S are continuous, the stochastic process is called Continuous random
process
➢ Discrete or continuous is used to refer to the nature of S and the sequence or process is
used to refer to the nature of T

FURTHER READING Stochastic Processes


More on stochastic processes including other examples can be found in following Internet
addresses ➥ http://math.uc.edu/~brycw/probab/books/smplbook/smplbook.html ➥
http://asrl.ecn.uiowa.edu/dbricker/Stacks_pdf1/Stochastic_Processes_Intro% 20 pdf
http://asrl.ecn.uiowa.edu/dbricker/ss_notes.html
http://cpk.auc.dk/dicom/E02/StochasticProcesses.htm
Exercises

Give five real-life examples and state of the process on each of the four combinations
(discrete/continuous time in conjunction with discrete/continuous state space)

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy