Advanced Algorithms Design and

Analysis
By

Nazir Ahmad Zafar
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Lecture No 3


Recurrence Relations

Mathematical Models, Analysis Techniques
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
What is Recursion?
Recursive Mathematical Models
Solving Recurrence Relations
First Order Linear Homogenous Recurrence
Relations, with Constant Coefficients
Second Order Linear Homogenous Recurrence
Relations, Constant Coefficients
Characteristics of Second Order Recurrences
Solution to Second order with distinct roots
Solution to Second order with repeated roots
General Recurrence solution
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Today Covered
Some times problems are too difficult or too
complex to solve because these are too big.
A problem can be broken down into sub-problems
and find a way to solve these sub-problems
Then build up to a solution to the entire problem.
This is the idea behind recursion
Recursive algorithms break down problem in pieces
which you either already know the answer, or can
be solved applying same algorithm to each piece
And finally combine the results of these sub-
problems to find the final solution
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
What is Recursion?
More concisely, a recursive function or definition
is defined in terms of itself.
Recursion is a computer algorithm that calls itself
in a steps having a termination condition.
The successive repetitions are processed up to
the critical step where the condition is met.
In recursive algorithms, each repetition is
processed from the last one called to the first.
Recursion is a wonderful technique for dealing
with many problems where problems and sub-
problems have same mechanism to solve it.
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
What is Recursion?
Recursive solutions are much easier to conceive of
and code than their iterative counterparts.
Every problem is not solvable using this approach
What kinds of problems are solved with recursion?
Generally, problems which are defined in terms of
themselves are usually good candidates for
recursive techniques.

Example
Finding factorial of a number is an easiest example
one can think using recursion
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Merits and Demerits of Recursion
The indispensable last step when analyzing an
algorithm is often to solve a recurrence equation.
With little experience recurrence equation can be
solved by intelligent guesswork.
However, there exist a powerful techniques that
can be use to solve certain classes of recurrence
almost automatically.
This is a main topic of this lecture, techniques
solving recurrence equations of form:
First order linear homogenous recurrences
Second order linear homogenous recurrences
Higher order recurrences
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Solving Recurrence
First Order Linear Homogenous Recurrence
Relations with Constant Coefficients
(FOLHRRCC)
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis

. 2
2
=
k k
a a
1 k
5 b

=
k
b

2
1
c
1 k
=
k
c k
No
YES
No
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Definition and Examples of FOLHRRCC

1 1
=
k k k
a a a No
1
1
=

k a A a
Definition
k k
0 A and numbers real a is where A
Second Order Linear Homogenous Recurrence
Relations with Constant Coefficients
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis

2 3
2 1
+ =
k k k
a a a
b
3 2 1 k
+ + =
k k k
b b b

7
3
2
1
c
2 1 k
=
k k
c c
YES
NO
YES
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Definition and Examples of SOLHRRCC
2

2 1
+ =

k a B a A a
Definition
k k k
0 with numbers real are and where B B A
d
2 1
2
1 k
+ =
k k k
d d d
2 e
2 k
=
k
e
NO
YES
1 2 f
1 k
+ =
k
f
2 - k 1 k
g g g
k
+ =

2 1 k
) 1 ( ) 1 ( h

+ =
k k
h k h
NO
YES
NO
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Some More Examples
2 1
+ =
k k k
Ba Aa a
0 ,.... ,.... , , 1
2
t t t t
n
the satisfies if only and if no, real zero - non a is where t t
0 : equation
2
= B At t
Theorem: Let A and B are real numbers. A recurrence of the form
is satisfied by the sequence
Proof:
Let us suppose that the sequence
0 , ,..., ,..., , , 1
2
t where t t t
n
Satisfies the recurrence relation
2 1
+ =
k k k
Ba Aa a
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Theorem
2 1
+ =
k k k
Bt At t
0 0 since
2

k
t t
2 k
t
It mean each form of sequence is equal to A times
the previous form plus B times the form before it

Hence
0
2
= B At t
Dividing both sides by
Conversely suppose that 0
2
= B At t
2 1
+ =
k k k
Bt At t
... ..., , , , 1 Hence
3 2 n
t t t t
satisfies the above
recurrence relation
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Contd...
2
2 1
+ =

k a B a A a
k k k
0
2
= B At t
2 2
2 1
+ =

k a a a
k k k
Definition:
Given a second order linear homogeneous
recurrence relation with constant coefficients
The characteristic equation of the relation is
Example 1:
Using characteristic equation, find solution to a
recurrence relation
Find all sequences that satisfy relation and have the form
0 ,..., ,..., , , 1
2
t t t t
n
Solution: Above recurrence relation is satisfied by a sequence
0 ,..., ,..., , , 1
2
t t t t
n
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Characteristic Equation
0 2 if, only and if
2
= t t
( )( ) 1 , 2 0 1 2 = = + t t t
Hence
,... 2 ,..., 2 , 2 , 2 , 2
3 2 0 n
and
( ) ( ) ( ) ( ) ,... 1 , 1 , 1 , 1
3 2 1 0

are both particular solutions for this recurrence relation
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Contd...
0
2
= B At t
... ,..., 3 , 2 , 0, and ,..., , , 1
3 2 2 n n
nr r r r r r r
Theorem
Let A and B be real numbers, and suppose that
the characteristic equation
has a single root r, than the sequences
Repeated Roots
2 1
+ =
k k k
Ba Aa a
Both satisfy the recurrence relation
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
0 if
2
= B At t
( )
2
2 2 2
2
2
and 2
2

r B r A
r rt t B At t
r t B At t
= =
+ =
=
Repeated Roots
Proof
has a single repeated root r than
We know that r
n
is a solution. Now we prove that
n
n
nr S =
is also a solution, i.e.
2 1
+ =
k k k
Ba Aa a
n
S satisfies the recurrence
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
k
k k k
kr BS AS e i = + =
2 1
S ., .
( ) ( )
proved hence RHS, k.r
) 2 2 2 (
) 2 ( ) 1 ( 2
) 2 ( ) 1 ( 2
) 2 ( ) 1 (
side hand left he Consider t
k
2 2 1
2 1
2 1
= =
+ =
=
=
+ = +



k
k k
k k
k k
k k
r k k
r k r k
r k r r k r
r k B r k A BS AS
Repeated Roots
k
k k
r k S B S A . . .
that prove to have fact we In
2 1
= +

Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
( )
conditions initial e satisfy th not do but relation, recurrence
same e satisfy th which sequences are 2 . and 2
root repeated is , 2 0 2
n
2
n
n
t t = =
3 and 1 b : condition initial with
2 4 4
1 0
2 1
= =
=

b
k b b b
k k k
Repeated Roots
Example 1: Single root case
Suppose sequence, b0, b1, b2,. . . satisfies recurrence relation
then find the explicit formula for b0, b1, b2, . . .
Solution:
Characteristic equation
0 4 4
2
= + t t
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
1 1 2 0 , 0 or
3 , 1 Since
constants are D and C , recurrence original the satisfies which
2 . . 2 . b : solution general Suppose
0
1 0
n
= = + =
= =
+ =
C D C n F
b b
n D C
n n
Repeated Roots
roots. repeated for solution required the is 2
2
1
2
1 2 2
2
1
2 1 Hence,
2
1
2
2 3
3 2 1 2 3 2 2
2 1 2 , 1 For
1 1
1
n
n n n
n
n
n
n b
D D D C
D C b n

+ =

+ = + =
=

= = + = +
+ = =
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
b
n
= (1 + n/2).2
n

First term
b
0
= (1 + 0/2).2
0
= 1.1 = 1
Second term
b
1
= (1 + 1/2).2
1
= 3/2.2 = 3
Third term
b
2
= (1 + 2/2).2
2
= 2.4 = 8
Fourth term
b
3
= (1 + 3/2).2
3
= 5/2.8 = 20, and so on
Checking Explicit Formula
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
2 2
2 1
+ =

k a a a
k k k
8 and 1
1 0
= = a a
2 2
2 1
+ =

k a a a
k k k
Example 1:
Find a sequence that satisfies the recurrence relation
And that also satisfies the initial condition
Solution: The characteristics equation of the relation
is
( )( ) 0 1 2 0 2
2
= + = t t t t
2 , 1 = t
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Solution as Linear Combination
8 and 1
1 0
= = a a

n n n
Ds Cr a + =
o o o
Ds Cr a + = = get we 0 n For
Are both sequence which satisfy above relation but neither one is having
Now since also satisfies the same recurrence relation and
(1) 1 D C + =
2 - D 3 get, we (2) and (1) equation Solving = = and C
Now
n n n
Ds Cr a + =
( ) ( ) 1 2 2 3
n n
n
a =
,... 2 , 2 , 2
2 1 0
=
n
r ( ) ( ) ( ) ,... 1 , 1 , 1
2 1 0
=
n
s
and
1 1 1
get we 1 n For Ds Cr a + = =
(2) 2 8 D C =
conditions given the satisfies which sequence required the is
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis
Contd..
Why recurrence?
Type of recurrences
Techniques solving recurrences
Generalized form of recurrence
Linear combination of solutions itself solution
Reusability of models when initial conditions are
changed but pattern remains same
It empowers the recursive models to be used in
more general phenomena
Easy implementation
Conclusion
Dr Nazir A. Zafar Advanced Algorithms Design and Analysis