Scribd
Upload
9 views

Definitions

Definitions

Uploaded by

lllt
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
9 views

Definitions

Definitions

Uploaded by

lllt
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

Stochastic Process

Introduction

Noha Youssef

The American University in Cairo


nayoussef@aucegypt.edu

1 / 13
Table of Contents

Introduction
Definition of a Stochastic Process
Examples

Markov Property

Deterministic and Stochastic Models

2 / 13
Introduction

Stochastic processes are systems which evolve in time (usually)


whilst undergoing random fluctuations. We need to understand how
the world works. This can happen through understanding chance:
by learning what can be said about unpredictable events and by
learning how to manipulate the occurrence of unpredictable events.

3 / 13
Examples
Below is a brief list of some important areas in which stochastic
processes arise:
Economics daily stock market quotations or monthly
unemployment figures.
Social sciences population birth rates and school enrollments series
have been followed for many centuries in several
countries.
Epidemiology numbers of influenza cases are often monitored over
long periods of time.
Medicine blood pressure measurements are traced over time to
evaluate the impact of pharmaceutical drugs used in
treating hypertension.
Markov Chain Monte Carlo (MCMC) is a modern statistical tool
which brings together results from SPs and
simulation enabling potentially computationally
intractable (Bayesian) statistics to be used in
practice.
4 / 13
Definition of a Stochastic Process

Definition 2.1
A stochastic process is a family of random variables {Xθ },
indexed by a parameter θ, where θ belongs to some index set θ.

If θ is a set of integers, representing specific time points, we have a


stochastic process in discrete time and we shall replace the general
subscript θ by t.

In a spatial process, θ would be a vector, representing location in


space rather than time.

Guess the definition of spatio-temporal processes!

5 / 13
Definitions
Definition 2.2
The index t is often interpreted as time and, thus, X(t) is referred
to as the state of the process at time t.

Definition 2.3
The set T is called index set (or parameter set).

Definition 2.4
If T is a countable set (e.g., N0 or Z) then we are dealing with a
discrete-time process.

Definition 2.5
If T is a continuum (e.g., R) then {X(t) : t ∈ T } is said to be a
continuous-time process.

6 / 13
More Definitions

Definition 2.6
For each t ∈ T , X(t) is a r.v. Any realization of the stochastic
process {X(t) : t ∈ T } is called a sample path.

Definition 2.7
The set of all possible values that the r.v. X(t) can take at all t,
say S, is said to be the state space of the stochastic process
{X(t) : t ∈ T }.

Definition 2.8
If S is a countable set (e.g., N0 or Z), {X(t) : t ∈ T } is a discrete
value process.

7 / 13
Example: Gambler’s ruin

Player A has $k, Player B has $(a − k), a > k > 0 play a series of
games in which A has prob. p of winning and q = 1 − p of losing.
Define r.v. Xn : A’s money after n games. Then (x1 , x2 , · · · , xn )
is a realisation of a discrete time random process.

8 / 13
Cont’D

Definition 2.9
If S is a continuum (e.g.,R) then {X(t) : t ∈ T } is a continuous
value process.

Definition 2.10
Discrete Time random process is observed only at specific times.

9 / 13
More Examples

Continuous Time random process


e.g. Number of customers in queue for ATM

Discrete State Space


e.g. Gambler’s ruin.
e.g. ATM customers

Continuous State Space


e.g. The closing price for a given stock per day is a discrete-time
stochastic process with a continuous state space.

10 / 13
More Definitions

I A counting process is a process X(t) in discrete or


continuous time for which the possible values of X(t) are the
natural numbers (0, 1, 2, · · · ) with the property that X(t) is a
non-decreasing function of t. Often, X(t) can be thought of
as counting the number of ’events’ of some type that have
occurred by time t. The basic example of a counting process
is the Poisson process.
I A sample path of a stochastic process is a particular
realization of the process, i.e. a particular set of values X(t)
for all t (which may be discrete of continuous), generated
according to the (stochastic) ‘rules’ of the process.
I The increments of a process are the changes X(t) − X(s)
between time points s and t, (s < t).

11 / 13
Markov Property

We are often interested in conditional distributions of the form

P r(Xtk |Xtk−1 , Xtk−2 , · · · , Xt1 )

for some times tk > tk−1 > tk−2 , · · · > t1 .

In general, this conditional distribution will depend upon values of


Xtk−1 , Xtk−2 , · · · , Xt1 . However, we shall focus particularly in this
module on processes that satisfies the Markov property, which says
that

P r(Xtk |Xtk−1 , Xtk−2 , · · · , Xt1 ) = P r(Xtk |Xtk−1 ).

Given the present (Xk−1 ), the future (Xk ) is independent of the


past (Xk−2 , Xk−3 , · · · , X1 ).
The Markov property is sometimes referred to as the lack of memory
property.
12 / 13
Deterministic and Stochastic Models

A deterministic model is specified by a set of equations that de-


scribe exactly how the system will evolve over time.

In a stochastic model, the evolution is at least partially random and


if the process is run several times, it will not give identical results.
Different runs of a stochastic process are often called realizations
of the process.

13 / 13

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy