Chapter 5 Random Processes

random process : a collection of time functions

and an associated probability description
marginal or joint pdf
ensemble
{x(t)}
X(t) : a sample function
X(t
1
) random variable
=
X
1

How to extended the concepts of random variables

to those of random processes? simple

more difficult is
to relate the mathematical representations for random variables
to the physical properties of the process
classification of random processes
continuous / discrete
deterministic / nondeterministic
stationary / nonstationary
ergodic / nonergodic

5.2 Continuous & Discrete Random Processes

Dependent on the possible values of the random variables

continuous random process :

random variables X(t
1
), X(t
2
), can assume any value
within a specified range of possible values
PDF is continuous (pdf has NO function)
X(t)
X1
t
1
t
sample function
pdf
f(x)
x

discrete random process :

random variables can assume only certain isolated values
pdf contains only functions

mixed process : have both continuous and discrete component

X(t)
t
sample function pdf
f(x)
x
100
0 0 100


5.3 Deterministic & Non deterministic Random Processes

nondeterministic random process :

future values of each sample ftn cannot be
exactly predicted from the observed past values
(almost all random processes are nondeterministic)

deterministic random process :

~ can be exactly predicted ~

random ftn of time

(Example)
X(t) = A cos(t + )
A, known constant
constant for all t
but different for each sample function
random variation over the ensemble, not wrt time
still possible to define r.v. X(t
1
), X(t
2
),
and to determine pdf for r.v.
Remark : convenient way to obtain a probability model
for signals that are known except for one or two
parameters
Tx Rx


5.4 Stationary & Nonstationary Random Processes

dependency of pdf on the value of time

stationary random process:

If all marginal and joint density function of the process do not depend
on the choice of time origin, the process is said to be stationary
( mean moment )
ensemble
sample
ftn
time
t
1
X(t
1
)
r.v.
pdf

nonstationary random process:

If any of the pdf does change with the choice of time origin,
the process is nonstationary.
All maginal & joint density ftn should be independent of
the time origin!
too stringent
relaxed condition
mean value of any random variable X(t
1
) is independent of t
1
& the correlation of two r.v. depends only on the time difference 1 2
t t

stationary in the wide sense
mean, mean-square, variance, correlation coefficient of
any pair of r.v. are constant
random
input
response
system analysis strictly stationary stationary
in the wide sense !

5.5 Ergodic & Nonergodic Random Process

If Almost every member of the ensemble shows the

same statistical behavior as the whole ensemble,
then it is possible to determine the statistical
behavior by examining only one typical sample
function.
Ergodic process

For ergodic process, the mean values and

moments can be determined by time averages as
well as by ensemble averages, that is,
(Note) This condition cannot exist
unless the process is stationary.
ergodic means stationary (not vice verse)

T
T
n
T
n
n
dt t X
T
dx x f x X ) (
2
1
lim ) (

5.6 Measurement of Random Processes

Statistical parameters of a random process

= the sets of statistical parameters associated with
the r.v. X(t) at various times t

these parameters are the same for all r.v.
consider only one set of parameters
if stationary

How to estimate the process parameters from

the observations of a single sample function?
We cannot make an ensemble average for
obtaining the parameters!
if erogodic
but, we cannot have a sample function
over infinite time interval
make a time average
make a time average over a finite
time interval approximation to the true value

Will determine

How good this approximation is?

Upon what aspects of measurement the goodness of the

approximation depends?

Estimation of the mean value of an ergodic random process {X(t)}

random variable true mean value

T
dt t X
T
X
0
) (
1

X
X X

X
mean variance !

[ ]
X dt X
T
dt t X
T
E X E
T
T

1
]
1

0
0
1
) (
1

( )
T
X
1

var
(see Ch.6)
T good estimate !
(Remark) X(t)

discrete measurement

If we measure X(t) at equally spaced time interval , that is,
then the estimate of can be expressed as
mean
mean-square
... ), 2 ( ), (
2 1
t X X t X X
X

N
i
X
N
X
1
1

[ ] [ ]


1
]
1

X X
N
X E
N
X
N
E X E
i i
1 1 1

( ) [ ]


1
]
1

1
]
1

j i j i
X X E
N
X X
N
E X E
2 2
2
1 1

: statistically independent, that is,
mean of estimate = true mean
[ ]
( ) j i X
j i X X X E
j i


2
2
( ) ( )( ) [ ]
( )
( )
2
2
2
2
2
2 2
2
2
1
1
1
1
1

X
N
X
N
X
N
X N N X N
N
X E
X
+

,
_

+
+
1
]
1

( ) ( ) [ ] { }
2
2 2
1

var
X
N
X E X E X

1
]
1

( )
N
X
1

var

See the example in pp.201~202
zero-mean
Gaussian
Random process
( )
2
, 0
Y
N
( )
2
) (t Y ) ( ) (
2
t Y t X

5.7 Smoothing Data with a Moving Window Average
i
Sample
i
X
Noise
i
N
Observed Data
i
Y
X X X
X
X X
X
i-n
L
i-1 i+1 i+n
R


+
+ +

R
L
n
n k
k i
R L
i
Y
n n
X
1
1

A kind of
LPF