Chapter 5

Statistical Decision Theory

5.1 (a) The LRT is
<
>
=
0
1
0 |
1 |
) | (
) | (
) (
0
1
H
H
H y f
H y f
y
H Y
H Y


We observe that for ) 2 ln(
2 / 1
2 0
2
1
2
1

<
>

<
>


H
H
y
H
H
e
y
y
, while for 2 > y , we
always decide H
0
.

(b) For minimum probability of error criterion 0
11 00
= = C C and
1
10 01
= = C C
(i) = = = 693 . 0 ) 2 ln( 1
2
1
0
P choose H
0
for 693 . 0 0 y ;
otherwise choose H
1
. The minimum probability of error is
355 . 0
2
1
) (
693 . 0
0
0
2
693 . 0
1
= + =


dy P dy e P P
y

72
) | (
0
0
|
H y f
H Y
) | (
1
1
|
H y f
H Y
2
1

0.693 1 2
1
y
Statistical Decision Theory

73
(ii) Similarly, =
3
2
1
P choose H
1
for 2 39 . 1 y and 308 . 0 ) ( = P .
(iii)
3
1
1
= P ,
<
>
0
0
1
H
H
y always decide H
1
and 288 . 0 ) ( = P .

5.2 (a)
<
>
=
0
1
0 |
1 |
) | (
) | (
) (
0
1
H
H
H y f
H y f
y
H Y
H Y




(i)
2
1
< ,


> ) ( y always decide H
1




(ii)
2
1
> ,


< ) ( , 1 0 y y decide H
0

> ) ( , 2 1 y y decide H
1

2
1
1 2

) ( y
2
1
1 2
y
) ( y

2
1
y
) ( y

1 2
Signal detection and estimation
74

(iii) ,
2
1
=



decide H
1
or H
0
in at the range 1 0 y and decide H
1
for 2 1 < y .
(b) (i)
2
1
< , the probability of false alarm is
1 1 ) | (
1
0
0 |
1
0
= = =

dy dy H y f P
Z
H Y F
.
The probability of detection is
0 1 1
2
1
) | (
2
0
1 |
1
1
= = = = =
D M
Z
H Y D
P P dy dy H y f P
(ii)
2
1
> , 0 0 ) | (
2
1
0 |
1
0
= = =

dy dy H y f P
Z
H Y F
and
2
1
2
1
2
1
) | (
2
1
1 |
1
1
= = = =
M
Z
H Y D
P dy dy H y f P
5.3 Minimum probability of error criterion 1 and 0
10 01 11 00
= = = = C C C C .

(a) The conditional density functions are
( )
(
(

=
2
2
0 |
2
exp
2
1
) | (
0
A y
H y f
H Y

( )
(
(

=
2
2
1 |
2
exp
2
1
) | (
1
A y
H y f
H Y

2
1
=
y
) ( y
1 2
Statistical Decision Theory

75
( )
( ) ( )
1
0
2
0
1
1
0
0
1
2
2
2
2
1
0
0
1
2 2
2 2
|
|
ln
2
ln
2 2
ln
] 2 / ) ( exp[
] 2 / ) ( exp[
) (
0
1
P
P
A
H
H
y
P
P
H
H
A y A y
y
P
P
H
H
A y
A y
f
f
y
H Y
H Y

<
>

<
>

+
+

=
=
<
>
+

= =


(b)
A A
H
H
y
P
P
2 2
0
1
0
1
549 . 0
3 ln
2

3
=
<
>
=
0
0
1
0 1
H
H
y P P
<
>
=

A A
H
H
y
P
P
2 2
0
1
0
1
256 405 . 0
3
5
=
<
>
=

As P
1
increases
D
P increases and
F
P increases, but
F
P increases at a faster
rate.

5.4 The received signals under each hypothesis are
N A Y H
N Y H
N A Y H
+ =
=
+ =
:
:
:
2
0
1

-A
H1 H0
A 0
) | (
0
0
|
H y f
H Y
) | (
1
1
|
H y f
H Y
y
Signal detection and estimation
76

(a) By symmetry, we observe that the thresholds are and , and
) | (error ) | (error
2 1
H P H P =
( )

(
(

= dy
A y
H P
2
2
1
2
exp
2
1
) | (error
( )

(
(

= dy
A y
H P
2
2
2
2
exp
2
1
) | (error

|
|
.
|

\
|

=
|
|
.
|

\
|

+
|
|
.
|

\
|

=
dy
y
dy
y
dy
y
H P
2
2
2
2
2
2
0
2
exp
2
1
2
2
exp
2
1
2
exp
2
1
) | (error

But ) | (error ) | (error
2 1
H P H P = and hence,

(
(

|
|
.
|

\
|

+
(
(

dy
A y
dy
y
dy
A y
P
2
2
2
2
2
2
2
) (
exp
2
exp 2
2
) (
exp
2
1
3
1
(error)

-A

H0
A 0
-
H1 H2
y
) | (
0
0
|
H y f
H Y
) | (
2
2
|
H y f
H Y
) | (
1
1
|
H y f
H Y
Statistical Decision Theory

77

(
(

+
|
|
.
|

\
|

dy
A y
dy
y
2
2
2
2
2
) (
exp
2
exp
2
1
3
2

Now,
( )
2
0
2
exp
2
exp 0
2
2
2
2
A A P
e
= =
(
(

+
|
|
.
|

\
|

(b) Substituting for the value of
2
A
= and solving the integrals we obtain
|
|
.
|

\
|

=
|
|
.
|

\
|

=
2
3
4
2 2
erfc
3
2
(error)
A
Q
A
P
5.5 (a)

+

+
= =
otherwise , 0
3 1 ,
8
3
8
1
1 1 ,
4
1
1 3 ,
8
3
8
1
) ( ) ( ) | (
1 |
1
y y
y
y y
n f s f H y f
N S H Y

as shown below




The LRT is then

< <
+

+

=
3 2 ,
2 1 ,
2
3
2
1
1 1 , 1
1 2 ,
2
3
2
1
2 3 ,
) (
y
y y
y
y y
y
y
1/4
1 2 3 -1 -2
-3
y
) | (
1
1
|
H y f
H Y
Signal detection and estimation
78
(i)
4
1
= , we have



(y) > always decide H
1
.
(ii) =1



2 cases: decide H
1
when

< < <

< <
=
3 2 and 1 1 and 2 3 when
2 1 and 1 2 when
) (
1
0
y y y H
y y H
y
or, decide H
0
when

< <
<
=
3 2 and 2 3 when
2 2 when
) (
1
0
y y H
y H
y
(iii) 2 =




decide H
0
when 2 2 y since 2 ) ( = < y
decide H
1
when 2 3 y and 3 2 y since > ) ( y
1/2
1 2 3 -1 -2
-3
y
1

) ( y
4 / 1 =
1/2
1 2 3 -1 -2
-3
y
1

=1
1/2
1 2 3 -1 -2
-3
y
1

=2
) ( y
Statistical Decision Theory

79
(b)

=
1
0
) | (
0 |
Z
H Y F
dy H y f P and

=
1
1
) | (
1 |
Z
H Y D
dy H y f P
(i) 1
4
1
= = =
D F
P P
(ii) 625 . 0 and
2
1
1 = = =
D F
P P or, 125 . 0 and 0 = =
D F
P P
(iii) 125 . 0 and 0 2 = = =
D F
P P

(c) The ROC is shown below

5.6 (a) 0 all for ) , ( ) (
0 0
0 0
0
= =

= =

s e dn e dn e
N
dn n s f s f
s
N
s
N
s
SN S

0
0
0
0
0 ,
1
) ( N n
N
ds e
N
n f
s
N
=

(b) = ) ( ) ( ) , ( n f s f n s f
N S SN
S and N are statistically independent.
(c)

= =
0
) ( ) ( ) ( ) ( ) ( d y f f n f s f y f
S N N S Y

Solving the convolution as shown in Chapter 2 in detail, we obtain

PD
PF
1
1
1/2
1/2
0.625
0.162
2
1
=
1
2
1
< <
1 >
Signal detection and estimation
80
| | { }

<

=

y N y N y
N
N y e
N
y f
y
Y
0 0
0
0
0
, ) exp( ) ( exp
1
0 , ) 1 (
1
) (


5.7 (a) The LRT is
=
|
|
.
|

\
|

<
>
=
<
>

2
ln
2
1
) (
2
1
2
1
) (
0
1
2
0
1
2
2
H
H
y y y T
H
H
e
e
y
y
y

Solving, + = = = 2 1 1 0
2
1
) (
2
y y y y T as shown below


To determine the decisions, we observe that we have 3 cases:
(i)
2
1
, (ii) 0
2
1
< < and (iii) 0 >
T(y)
y
-1/2
0 1 2 -1 -2
fY (y)
y
N0
) 1 (
1
0
0
N
e
N


Statistical Decision Theory

81
(i)
e 2 2
1






> ) ( y T always decide H
1







(ii)
2 2
0
2
1
<

< <
e


+ = 2 1 1
1
y , + + = 2 1 1
2
y , + = 2 1 1
3
y and + + = 2 1 1
4
y

Decide H
1
when
1
y y ,
3 2
y y y and
4
y y .
Decide H
0
when
2 1
y y y < < and
4 3
y y y < < .

(iii)
2
0

> >







y
-1/2
1
2
-1
-2

y1 y2 y3 y4
H1
H1
H1
H0 H0
T(y)
y
-1/2
0 1 2 -1 -2

y
-1/2
0 1
2
-1
-2

y1 y2
H1 H1 H0
Signal detection and estimation
82
+ = 2 1 1
1
y and + + = 2 1 1
2
y .

Decide H
1
when
1
y y and
2
y y
Decide H
0
when
2 1
y y y < <

(b) 47 . 0 2
3
2
1
0
0
= = =
P
P
P
02 . 0 ) 393 . 2 ( 2
2
1
2
1
) | ( ) | (
2 1 1
2
2 1 1
2
0 | 0 1
2 2
1
0
= =

=
= =

+ +

Q dy e dy e
dy H y f H H P P
y y
Z
H Y F

09 . 0
2
1
2
1
) | ( ) | (
2 1 1
2 1 1
1 | 1 1
1
1
= + = = =


+ +

+

+
dy e dy e dy H y f H H P P
y y
Z
H Y D

(c) ) 2 1 1 ( 2 + + = Q P
F

) 2 1 1 exp( 1 1 )] 2 1 1 ( exp[ + = = + + =
D M D
P P P
The optimum threshold
opt
is obtained when
M F
P P = , or
) 2 1 1 ( 2 ) 2 1 1 exp( 1
opt opt
Q + + = +
(d) 2 . 0
2
1
2
1
1
2
1
2
2 2
=

dy e dy e P
y y
F

or, 0 22 . 0 2 . 0
2
erf 1 . 0 ) ( 2 . 0 ) ( 2
1
1
1 1
> =
|
|
.
|

\
|
= = Q Q ; that is the
decision regions as given in (a) part (iii).



Statistical Decision Theory

83
5.8
(
(

=
2
) 1 (
exp
2
1
) | (
2
1 |
1
y
H y f
H Y

|
|
.
|

\
|

=
2
exp
2
1
) | (
2
0 |
0
y
H y f
H Y

(a) The LRT is
2
1
1
2
exp
2
1
2
) 1 (
exp
2
1
) (
0
1
0
1
2
2
H
H
y
H
H
y
y
y
<
>

<
>
(
(

(
(

=
(b) 005 . 0
2
1
) | (
2
0 |
2
1
0
=

= =

dy e dy H y f P
y
Z
H Y F

581 . 2 005 . 0
2
erf 1
2
1
) ( =
(
(

|
|
.
|

\
|
= Q
(c)

=
=

=
(
(

=
1
2
581 . 2
2
013 . 0 ) 581 . 1 (
2
1
2
) 1 (
exp
2
1
2
Q dx e dy
y
P
x
D


5.9 The LRT is

<
>
|
|
.
|

\
|

(
(

= =

=
=
0
1
1
2
2
1
2
2
0 |
1 |
2
exp
2
1
2
) (
exp
2
1
) | (
) | (
) (
0
1
H
H
y
m y
H f
H f
y
K
k
k
K
k
k
H
H
y
y
Y
Y

4 4 3 4 4 2 1
3 2 1

=
+

<
>

2
ln
2
0
1
) (
1
Km
m
H
H
y
y T
K
k
k
as given in Example 5.2. Hence,
<
>
0
1
) (
H
H
y T .

Signal detection and estimation
84
5.10

=
|
|
.
|

\
|

=
K
k
k
H
y
H f
1
2
0
2
0
0 |
2
exp
2
1
) | (
0
y
Y

=
|
|
.
|

\
|

=
K
k
k
H
y
H f
1
2
1
2
1
1 |
2
exp
2
1
) | (
1
y
Y

=

<
>
=
K
k
k
H
H
y T
1
0
1
2
) ( y where,
|
|
.
|

\
|

=
1
0
2
0
2
1
2
1
2
0
ln ln
2
K from Example 5.9.

5.11 (a) The probability of false alarm is
|
|
.
|

\
|

=

= =

0
2
0
0 |
0
2
1
0
2
1
) | ( Q dy e d H f P
y
Z
H F
y y
Y

where, 1 and ln ln
2
1
0
2
0
2
1
2
1
2
0
=
|
|
.
|

\
|

= K .
|
|
.
|

\
|

=
|
|
.
|

\
|

=

= =

1 1
2
1
1
2
1
1
1
2
Q P Q dy e P P
M
y
M D

(b) The ROC is P
D
versus P
F
. For 2 2
2
0
2
1
= = , we have
(
(

=
2
) 2 ln( 4
Q P
D
and )] 2 ln( 4 [ = Q P
F
for various values of . Hence,



1/2
1/2
PD
PF
Statistical Decision Theory

85
(c) The minimax criterion when 0
11 00
= = C C and 1
10 01
= = C C yields
M F
P P = . Hence,
|
|
.
|

\
|

=
|
|
.
|

\
|

0 1
1
opt opt
Q Q .
5.12 (a)
|
|
.
|

\
|

=
2
2
0 |
2
exp
2
1
) | (
0
y
H y f
H Y


(
(

=
2
2
1 |
2
) (
exp
2
1
) | (
1
m y
H y f
H Y


(
(

=
2
2
2 |
2
) (
exp
2
1
) | (
2
m y
H y f
H Y


The receiver based on the minimum probability of error selects the hypothesis
having the largest a posteriori probability ) | ( y H P
j
, where
) (
) ( ) | (
) | (
|
y f
H P H y f
y H P
Y
j j H Y
j
j
=
3
1
) ( =
j
H P and ) ( y f
Y
is common to all a posteriori probabilities We choose
H
j
for which ) | (
| j H Y
H y f
j
is largest. This is equivalent to choosing H
j
for which
) (
j
m y is smallest. Hence, we have

(b) The minimum probability of error is

= =
= =
3
1
3
1
) | (
3
1
) | ( ) ( ) (
j
j
j
j j
H P H P H P P where,
H0 H2 H1
-m m
y
-m/2 m/2
0
Signal detection and estimation
86
|
.
|

\
|

=
(
(

=
|
|
.
|

\
|
> =

2
2
1
2
) (
exp
2
1
2
) | (
2 /
2
2 /
2
2
1
1 1
2
m
Q dx e
dy
m y
H
m
y P H P
m
x
m

By symmetry, ) | ( ) | (
3 1
H P H P = and
dy e dy e
H
m
Y
m
Y P H
m
Y P H P
m
y
m
y

+

=
|
|
.
|

\
|
> < =
|
|
.
|

\
|
> =
2 /
2
2 /
2
0 0 0
2
2
2
2
2
1
2
1
2
and
2 2
) | (

|
.
|

\
|

= |
.
|

\
|

2 3
4
) (
2
2
2
1
2
2 /
2
2
m
Q P
m
Q dx e
m
x

(c) The conditional density functions become
2
0 |
2
0
2
1
) | (
y
H Y
e H y f

=
(
(

=
2
) 1 (
exp
2
1
) | (
2
1 |
1
y
H y f
H Y

8
2 |
2
2
2 2
1
) | (
y
H Y
e H y f

=
The boundary between H
0
and H
1
is
2
1
= y , while the boundary between H
0
and
H
2
is obtained from
36 . 1 ) | ( ) | (
8 2
2 | 0 |
2 2
2 0
= =

y e e H y f H y f
y y
H Y H Y

For the boundary between H
1
and H
2
, we have
Statistical Decision Theory

87
18 . 0 and 85 . 2
2
1
) | ( ) | (
2 1
2 2
) 1 (
2 | 1 |
2 2
2 1
= =

y y e e H y f H y f
y y
H Y H Y



) | ( ) ( ) (
3
1
j
j
j
H P H P P =

=
where,
527 . 0 ) 0 ( ) 36 . 1 (
2
1
2
1
) | ( ) | (
0
2
36 . 1
2
0 | 0
2 2
2 1
0
= + =

+

= =

Q Q
dy e dy e dy H y f H P
y y
Z Z
H Y
U
29 . 0 ) 85 . 1 (
2
1
2
1
2
1
2
1
2
1
) | ( ) | (
85 . 1
2
2 / 1
2
85 . 2
2
) 1 (
2 / 1
2
) 1 (
1 | 1
2 2
2 2
2 0
1
= + |
.
|

\
|
=

= =

Q Q
dx e dx e
dy e dy e dy H y f H P
x x
y y
Z Z
H Y
U

H0 H2
H1
y
2.85 1.36 1 0.5
0 -0.18 -1.36
H2
-1 2
2.85 1.36 1
0.5
y
0 -0.18 -1.36
) | (
1 1 |
H y f
H Y
) | (
0
0
|
H y f
H Y

) | (
2
2
|
H y f
H Y

Signal detection and estimation
88
1 ) 7 . 5 ( ) 72 . 2 ( 1
) 7 . 5 ( ) 72 . 2 (
2
1
2
1
2
1
2 2
1
) | ( ) | (
7 . 5
2
72 . 2
2
7 . 5
72 . 2
2
85 . 2
36 . 1
8
2 | 2
2 2
2 2
1 0
2
=
=

= =

Q Q
Q Q dx e dx e
dx e dy e dy H y f H P
x x
x y
Z Z
H Y
U

6 . 0 )] | ( ) | ( ) | ( [
3
1
) (
2 1 0
+ + = H P H P H P P
5.13

=
|
|
.
|

\
|

=
K
k
k
H
y
H f
1
2
2
0 |
2
exp
2
1
) | (
0
y
Y

= (
(

+
=
K
k
m
k
m
H
y
H f
1
2 2
2
2 2
1 |
) ( 2
exp
1
) | (
1
y
Y

1
) ( 2
exp ) (
) | (
) | (
0
1
2 2 2
2
2 /
2 2
2
0 |
1 |
0
1
H
H
H f
H f
m
m T
K
m
H
H
<
>
(
(

+

|
|
.
|

\
|
+

= y y y
y
y
Y
Y

Taking the logarithm on both sides and rearranging terms, we obtain the decision
rule

|
|
.
|

\
|

+
<
>
2
2 2
2
2 2 2
0
1
ln
) (
m
m
m T
K
H
H
y y
or,

<
>

=
0
1
1
2
H
H
y
K
k
k


Statistical Decision Theory

89
5.14 The conditional density functions are
| |
(
(

+
= y y y
Y
T
m
K
m
H
H f
) 1 ( 2
1
exp
) 1 ( 2
1
) | (
2 2 /
2
1 |
1

where | |
T
K
y y y L
2 1
= y and
|
.
|

\
|

= y y y
Y
T
K
H
H f
2
1
exp
) 2 (
1
) | (
2 /
0 |
0

The LRT is then

<
>

(
(

|
|
.
|

\
|
+
=
0
1
2
2
2 /
2
) 1 ( 2
exp
1
1
) (
H
H
m
m T
K
m
y y y
Taking logarithm
<
>
|
|
.
|

\
|
+
+
+

ln
1
1
ln
2
) 1 ( 2
0
1
2 2
2
H
H
K
m m
m T
y y
1
2
2
2
0
1
) 1 ln(
2
) 1 ( 2
+ +

+
<
>

m
m
m T
K
H
H
y y
or,
1
0
1
1
2

<
>

=
H
H
y
K
k
k

We observe that the LRT does not require knowledge of
2
m
to make a decision.
Therefore, a UMP test exists.

Signal detection and estimation
90
5.15

=
|
|
.
|

\
|

=
K
k
k
H
y
H f
1
2
0 |
2
exp
2
1
) | (
0
y
Y

= (
(

=
K
k
k
H
m y
H f
1
2
1 |
2
) (
exp
2
1
) | (
1
y
Y

(a) |
.
|

\
|
=
<
>
=
=
K
k
k
H
H
m
K
y m
H
H
H f
H f
1
2
0
1
0 |
1 |
2
exp ) (
) | (
) | (
) (
0
1
y
y
y
y
Y
Y

or,
+
<
>

=
m
Km
H
H
y
K
k
k
2
ln 2
2
0
1
1
. Therefore, a test can be conducted without
knowledge of m A UMP test exists.
(b) = 05 . 0
F
P The test decides H
0
when > =

=
K
k
k
y y T
1
) ( , where T is
Gaussian with mean zero and variance K under H
0
. Hence,
05 . 0
2
erf 1
2
1
) | (
0 |
0
=
(
(

|
|
.
|

\
|
=
|
|
.
|

\
|
= =

K K
Q dt H t f P
H T F

Using 9 . 0 ) | (
1 |
1
> =

dt H t f P
H T D
where T is Gaussian with mean Km under H
1
,
we obtain from the table in the appendix 16 K .
5.16 Since the observations are independent, the LRT becomes

<
>
(
(

|
|
.
|

\
|

+ + +
|
|
.
|

\
|

= =
0
1
0 1
2 1
1
0
0 |
1 | 1 1
) ( exp
) | (
) | (
) (
0
1
H
H
y y y
H f
H f
K
K
H
H
L
y
y
y
Y
Y

Statistical Decision Theory

91

<
>
(
(

|
|
.
|

\
|


|
|
.
|

\
|

=

=
0
1
1 0
0 1
1 1
0
exp ) (
H
H
y
K
k
k
K
y
Taking the natural logarithm and simplifying the expression, we obtain
=
|
|
.
|

\
|

<
>
=

=
K
k
k
H
H
y T
1 0
1
0 1
1 0
0
1
ln ln ) ( y
For a UMP test of level , we need
05 . 0 ] | ) ( [
0
= > H T P Y or 95 . 0 ] | ) ( [
0
= H T P Y
We now determine the distribution of the test statistic ) (Y T using the
characteristic function such that
| | | | | | | | | |
) ( ) ( ) (
) (
2 1
2 1 2 1
) (
=
= = =
+ + +
K
K K
y y y
Y j Y j Y j Y Y Y j j
t
e E e E e E e E e E
L
L
L Y

since the Y
k
s, K k , , 2 , 1 L = are statistically independent. From (2.93),
K
j

= ) 1 ( ) (
t
. Hence, from (2.102), ) (Y T is a gamma distribution
) , ( P K G with density function

>
=

otherwise , 0
0 ,
) (
1
) (
/ 1
t e t
K t f
t K
K
T

Therefore, for 21 = K , (see Table 9 page 456, Dudewicz
1
)
62 . 290 062 . 29 95 . 0 ] | ) ( [
0
0
= =

= H T P Y
The test decides H
1
(rejects H
0
) if 62 . 290 ) ( > Y T

1 Dudewicz, E. J., Introduction to Statistics and Probability, Holt, Rinehart and Winston, New York,
1976.