Está en la página 1de 28

# 2010 - 02 -17

## CS 515: Parallel Algorithms

Chandrima Sarkar Atanu Roy

Agenda
Architecture Parallel Programming Languages Precedence Graph Elementary Parallel Algorithms Sorting Matrix Multiplication Download :http://www.cs.montana.edu/~atanu.roy/Classes/ CS515.html

Architecture
Flynns Classification S = single , M = multiple , I = instruction (stream), D = data (stream)

SISD

SIMD

Architecture
Flynns Classification S = single , M = multiple , I = instruction (stream), D = data (stream)

MISD

MIMD

## Static Inter-connection Network

Linear Array Ring

Chordal ring

Tree

Tree Cont.

FAT TREE

STAR

Hypercube
100 000 110

010
101 111

5-D

## Parallel Programming Languages

Control Mechanism Communication Mechanism

## Shared Memory Control driven

Message-passing

Fortran 90/HPF , C++ , HEP CSP , Ada , OCCAM (Von PL/I , Ada , Concurrent Neumann Language Pascal Modula-2 , MultiLisp Extension ) (MIMD), Lisp Connection Machine (SIMD) VAL , ID LAU , SISAL ( dataflow languages )
Concurrent Prolog ( Shapiro ) Actors FP

Data driven
Pattern driven Demand driven ( reduction language )

## Dijkstras High Level language construct

Degree of Parallelism is static Algol-68,CSP Precendence Graph

A
parbegin C begin B parbegin D E parend G end parend H

## Elementary Parallel Algorithms

Finding sum using a 2D mesh architecture

Algorithm 1

Algorithm 2

(1) p = n

## Odd Even Transposition Sort (contd)

(2) p << n
S= {12, 7, 2, 4, 1, 11, 9, 5, 6, 3, 10, 8}, p = 4
P1 {12, 7, 2} {2, 7, 12} {1, 2, 4} {1, 2, 4} {1, 2, 3} {1, 2, 3} P2 {4, 1, 11} {1, 4, 11} {7, 11, 12} {3, 5, 6} {4, 5, 6} {4, 5, 6} P3 {9, 5, 6} {5, 6, 9} {3, 5, 6} {7, 11, 12} {7, 8, 9} {7, 8, 9} P4 {3, 10, 8} {3, 8, 10} {8, 9, 10} {8, 9, 10} {10, 11, 12} {10, 11, 12}

Pseudocode
Proc MERGE-SPLIT(S)
for i:= 1 to p do in parallel
QUICKSORT(Si)

end for for (i := 1 to ceil(p/2)) for odd-numbered processor do in parallel MERGE(Si , Si + 1) SPLIT end for for odd-numbered processor do in parallel MERGE(Si , Si + 1) SPLIT end for end for

## 2 D mesh with Snake Order

Thompson and Kung (1977)

Input : {23, 6, 1, 5, 11, 13, 55, 19, -3, 12, -5, -7, 9, 55, 28, -2}

## Bitonic Merge Sort

Bitonic Sequence :- 1, 3, 7, 8 Comparator Note :- Batchers Bitonic Merge Sort compares elements whose indices differ by a single bit. 6, 5, 4, 2

## Bitonic Merge Sort

Shuffle-Exchange Network
Bitonic Mergesort on Shuffle-Exchange Network A list of n = 2k unsorted elements can be sorted in time (lg2 n) with a network 2k-1[k (k-1) + 1] comparators using the shuffle-exchange network.

Sorting Network

## Systolic Matrix Multiplication

1. 2. 3. 4. Multiply ai,k by ak,j Add the result to ri,j Send ai,k to cell ci+1,j Send bk,j to cell ci,j+1

Home Work
Show how the following 16 values would be sorted by Batchers Bitonic sort. 16, 7, 4, 12, 2, 10, 13, 9, 1, 8, 11, 3, 15, 6, 5, 14