Divide-and-Conquer

• Divide the problem into a number of subproblems.

•Conquer the subproblems by solving them recursively. If

the subproblem sizes are small enough, solve the

subproblems in a straightforward manner.

 Divide-and-Conquer

• Combine the solutions to the subproblems into the

solution for the original problem.

 Merge Sort Algorithm

Divide: Divide the n-element sequence into two

subsequences of n/2 elements each.

Conquer: Sort the two subsequences recursively using

merge sort.

Combine: Merge the two sorted sequences.

 How to merge two sorted
sequences
•We have two subarrays A[p..q] and A[q+1..r] in sorted

order.

• Merge sort algorithm merges them to form a single

sorted subarray that replaces the current subarray A[p..r]

 Merging Algorithm

Merge( A, p, q, r)

1. n1 q – p + 1

2. n2  r – q

3. Create arrays L[1..n1+1] and R[1..n2+1]

 Merging Algorithm

4. For i  1 to n1

5. do L[i]  A[p + i -1]

6. For j  1 to n2

7. do R[j]  A[q + j]
 Merging Algorithm

8. L[n1 + 1]  ∞

9. R[n2 + 1]  ∞

10. i  1

11. j  1
 Merging Algorithm

12. For k  p to r

13. Do if L[i] <= R[j]

14. Then A[k]  L[i]

15. i=i+1

16. else A[k]  R[j]

17. j=j+1
 Merging Algorithm
8 9 10 11 12 13 14 15 16 17
A… 2 4 5 7 1 2 3 6 …
k

1 2 3 4 5 1 2 3 4 5
L 2 4 5 7 ∞ R 1 2 3 6 ∞
i j

(a)
 Merging Algorithm
8 9 10 11 12 13 14 15 16 17
A… 1 4 5 7 1 2 3 6 …
k

1 2 3 4 5 1 2 3 4 5
L 2 4 5 7 ∞ R 1 2 3 6 ∞
i j

(b)
 Merging Algorithm
8 9 10 11 12 13 14 15 16 17
A… 1 2 5 7 1 2 3 6 …
k

1 2 3 4 5 1 2 3 4 5
L 2 4 5 7 ∞ R 1 2 3 6 ∞
i j

(c)
 Merging Algorithm
8 9 10 11 12 13 14 15 16 17
A… 1 2 2 7 1 2 3 6 …
k

1 2 3 4 5 1 2 3 4 5
L 2 4 5 7 ∞ R 1 2 3 6 ∞
i j

(d)
 Merging Algorithm
8 9 10 11 12 13 14 15 16 17
A… 1 2 2 3 1 2 3 6 …
k

1 2 3 4 5 1 2 3 4 5
L 2 4 5 7 ∞ R 1 2 3 6 ∞
i j

(e)
 Merging Algorithm
8 9 10 11 12 13 14 15 16 17
A… 1 2 2 3 4 2 3 6 …
k

1 2 3 4 5 1 2 3 4 5
L 2 4 5 7 ∞ R 1 2 3 6 ∞
i j

(f)
 Merging Algorithm
8 9 10 11 12 13 14 15 16 17
A… 1 2 2 3 4 5 3 6 …
k

1 2 3 4 5 1 2 3 4 5
L 2 4 5 7 ∞ R 1 2 3 6 ∞
i j

(g)
 Merging Algorithm
8 9 10 11 12 13 14 15 16 17
A… 1 2 2 3 4 5 6 6 …
k

1 2 3 4 5 1 2 3 4 5
L 2 4 5 7 ∞ R 1 2 3 6 ∞
i j

(h)
 Merging Algorithm
8 9 10 11 12 13 14 15 16 17
A… 1 2 2 3 4 5 6 7 …
k

1 2 3 4 5 1 2 3 4 5
L 2 4 5 7 ∞ R 1 2 3 6 ∞
i j

(i)
 Merge Sort Algorithm
Merge-Sort( A, p, r)

1. If p<r

2. then q  (p+r)/2

3. Merge-Sort( A, p, q)

4. Merge-Sort( A, q+1, r)

5. Merge( A, p, q, r)
 Operation of Merge Sort
1 2 2 3 4 5 6 7

2 4 5 7 1 2 3 6

2 5 4 7 1 3 2 6

5 2 4 7 1 3 2 6
 Analyzing Divide-and-Conquer
Algorithm
When an algorithm contains a recursive call to

itself, its running time can be described by a

recurrence equation or recurrence which

describes the running time

 Recurrence

If the problem size is small enough, say

n<=c for some constant c, the

straightforward solution takes constant

time, can be written as θ(1).

 Recurrence

If we have

•a subproblems, each of which is 1/b the

size of the original.

•D(n) time to divide the problem

 Recurrence

•C(n) time to combine the solution

The recurrence
θ(1) if n <= c
T(n)= aT(n/b) + D(n) + C(n) otherwise
 Recurrence

Divide: The divide step computes the

middle of the subarray which takes

constant time, D(n)=θ(1)

 Recurrence

Conquer: We recursively solve two

subproblems, each of size n/2, which

contributes 2T(n/2) to the running time.

 Recurrence

Combine: Merge procedure takes θ(n) time

on an n-element subarray. C(n)=θ(n)

The recurrence
θ(1) if n=1
T(n)= 2T(n/2) + θ(n) if n>1
 Recurrence

Let us rewrite the recurrence

C if n=1
T(n)= 2T(n/2) + cn if n>1

C represents the time required to solve

problems of size 1
 A Recursion Tree for the
Recurrence
T(n)
Cn

T(n/2) T(n/2)
A Recursion Tree for the
Recurrence
Cn

Cn/2
Cn/2

T(n/4) T(n/4) T(n/4) T(n/4)

 A Recursion Tree for the
Recurrence
C(n)
cn

Cn/2 cn
Cn/2

lg n

Cn/4 Cn/4 Cn/4 cn

Cn/4

C C C C C C cn
C
 Total Running Time

The fully expanded tree has lg n +1 levels

and each level contributes a total cost of

cn. Therefore T(n)= cn lg n + cn = θ(nlg n)

 Growth of Functions

We look at input sizes large enough to

make only the order of growth of the

running time relevant.

 Asymptotic Notation

Used to describe running time of an

algorithm and defined in terms of functions

whose domains are the set of natural

numbers N={0,1,2--------------}
 θ-Notation

θ(g(n)) = { f(n) : there exist positive

constants C1, C2, and n0 such that

0 <= C1g(n)<=f(n) <=C2g(n) for all n>=n0}

 θ-Notation

C2g(n)

f(n)

C1g(n)

n
n0
f(n)=θ(g(n))
 θ-Notation

For all n>=n0, the function f(n) is equal to

g(n) to within a constant factor. So g(n) is

asymptotically tight bound for f(n).

 θ-Notation

Let us show that 1/2n2- 3n=θ(n2)

To do so, we must determine C1, C2, and

n0 such that C1n2<=1/2n2-3n<=C2n2 for all

n>=n0
 θ-Notation

Diving by n2

C1<=1/2 – 3/n <= C2

By choosing C1=1/14 , C2=1/2, and n0=7,

we can verify that 1/2n2- 3n=θ(n2)

 O-Notation

O(g(n)) = { f(n) : there exist positive

constants C and n0 such that

0 <= f(n) <=Cg(n) for all n>=n0}

 O-Notation

Cg(n)

f(n)

n0 n

f(n)=O(g(n))
 O-Notation

O(n2) bound on worst-case running time of

insertion sort also applies to its running

time on every input.

 O-Notation

θ(n2) bound on worst-case running time of

insertion sort, however, does not imply a

θ(n2) bound on the running time of

insertion sort on every input.

 Ω-Notation

Ω(g(n)) = { f(n) : there exist positive

constants C and n0 such that

0 <= Cg(n)<= f(n) for all n>=n0}

 Ω-Notation

f(n)

Cg(n)

n0 n

f(n)=Ω(g(n))
 Ω-Notation

Since Ω-notation describes a lower bound,

we use it to bound the best-case running

time of an algorithm, we also bound the

running time of algorithm on arbitrary inputs

 o-Notation

We use o-notation to denote a upper bound

that is not asymptotically tight. The bound

2n2=O(n2) is asymptotically tight, but the

bound 2n=O(n2) is not.

 ω-Notation

We use ω-notation to denote a lower bound

that is not asymptotically tight.