ME 361-System of Equations-26 Sept 2013

ME 361 NUMERICAL METHODS FOR ENGINEERS
Prof. Dr. Faruk Arn Fall 2013

SYSTEM OF EQUATIONS
1 1 2 3 11 1 12 2 13 3 1 1
2 1 2 3 21 1 22 2 23 3 2 2
3 1 2 3
f (x , x , x , ...) 0 => a x + a x + a x + .... + a x = c
f (x , x , x , ...) 0 => a x + a x + a x + .... + a x = c
f (x , x , x , ...) 0 =
n n
n n
=
=
=
31 1 32 2 33 3 3 3
1 2 3 1 1 2 2 3 3
> a x + a x + a x + .... + a x = c
....
f (x , x , x , ...) 0 => a x + a x + a x + .... + a x = c
n n
n n n n nn n n
=
Set of n equations, linear in the variables, x
n
s :
Set of n equations, nonlinear in the variables, x
,
y and z :
( ) ( )
( )
2
2
1 1 1 1 1 1
d z
2 2 2 2 2
3
f (x, y, z) 0 => a x + b ln y + c cos d z = e
1
f (x, y, z) 0 => a + b sin y + c e = e
x
f (x, y, z) 0 => . etc
=
=
=
MATRICES
A Matrix is a rectangular array of elements in m rows and n columns, in general.
{ }
1 1 1 2 1 n
2 1 2 2 2 n
i j
m 1 m 2 m n
a a . . a
a a . . a
A a
. . . . .
a a . . a
| |
|
|
= =
|
|
|
\ .
where i is the row and j is the column.
Square Matrix : m = n
Determinant of a matrix : |A| = |a
i j
|
Transpose of a matrix : A
T
= {a
j i
}
Matrix Operations : Addition A + B = {a
i j
+ b
i j
}
Product with a scalar k A = {k a
i j
}
Product of two matrices
A is m by n
B is n by p
A . B = C
C matrix is m by p
A must be conformable
with B
Column and Row Matrix Vector
Matrix Definitions and Properties
Unit(y) Matrix
I has 1's in the
principal diagonal
If A is 1 by n
B is n by 1
W is n by n
A W B = c where c is a scalar
If c = 0, A and B vectors are mutually orthogonal
with respect to the weight matrix, W.
If A is 1 by n
B is n by 1
B A is called a tensor
|
|
|
|
|
.
|
\
|
=
1 . . . 0 0
. . . . . .
0 . . . 1 0
0 . . . 0 1
I
( )
b a . . . b a
. . . . .
. . . . .
b a . . . b a
a . . . a
b
.
.
b
A B
n n n 1
1 n 1 1
n 1
n
1
|
|
|
|
|
.
|
\
|
=
|
|
|
|
|
.
|
\
|
=
Inverse Matrix : A
- 1
(If A is square)
A A
- 1
= A
- 1
A = I
Not all square matrices have inverses.
Singular Matrix |A| = 0
Given a square matrix A
If A X = C X = A
- 1
C when |A| = 0
{ }

A
A

A
A Adj
A A of Inverse
T
j i 1 -
= = =
Cofactor of a
i , j
= A
i j
= (-1)
i + j
Adjoint of A = Adj A = {A
i , j
}
T
SYSTEM OF LINEAR EQUATIONS
|
|
|
|
|
.
|
\
|
=
|
|
|
|
|
.
|
\
|
|
|
|
|
|
.
|
\
|
m
2
1
n
2
1
n m 2 m 1 m
n 2 2 2 1 2
n 1 2 1 1 1
c
.
c
c

x
.
x
x

a . . . a a
. . . . . .
a . . . a a
a . . . a a
The given equations are linear in the independent variables, x
n
s:
a
11
x
1
+ a
12
x
2
+ .. + a
1n
x
n
= c
1
a
21
x
1
+ a
22
x
2
+ .. + a
2n
x
n
= c
2
.
a
m1
x
1
+ a
m2
x
2
+ .. + a
mn
x
n
= c
m
In Matrix form:
Assume that r number of equations are linearly independent. Then, the number of
residual equations is (m - r).
Linearly independent equation: It cannot be formed by linear combination of the
others.
A number of possibilities exists when m > r
If the equations contradict one another, they are said to be inconsistent or
incompatible and no solution exists
If r = n, a unique solution exists
If r < n, a family of solutions exists. (n - r) is called the defect of the system.
Note that if r > n, the equations must be inconsistent since r number of
equations are linearly independent.
GAUSS(IAN) ELIMINATION
|
|
|
|
|
|
.
|
\
|
n n 2 n 1 n
n 2 2 2 1 2
n 1 2 1 1 1
a a a
. . . . . .
. . . . . .
a . . . a a
a . . . a a
|
|
|
|
|
|
|
.
|
\
|
*
n 2
*
n 2
*
2 2
n 1 2 1 1 1
a . . . 0 0
. . . . . .
. . . . 0 .
a . . . a 0
a . . . a a
It is a method of systematically re-
ordering and linearly combining a
given set of linear equations to
reduce a n by n (n x n) matrix of
coefficients
to an upper-triangular matrix
and then back substitute to determine the unknown variables.
1) If , re-order the equations
2) Multiply (1) by and (2) by , substract (1) from (2), replace (2)
Re-name coefficients
Given a set of m equations in n
unkowns:
0 0 0 0
1 1 1 1 2 2 1 n n 1
0 0 0 0
m 1 1 m 2 2 m n n m
a x a x ... a x c (1)
. . . .
. . . .
a x a x ... a x c (m)
+ + + =
+ + + =
Elimination Procedure:
0 a
0
1 1
=
a
0
1 2
0
1 1
a
( ) c a - c a ... x a a - a a 0
0
1 1 2
0
2
0
1 1 2
0
2 1
0
1 2
0
1 1
0
2 2
= + +
Repeat steps 1 and 2 for all the remaining rows and obtain:
. . . . . .
) (3' c x a ... x a 0
) (2' c x a ... x a 0
(1) c x a ... x a x a
1
3 n
1
n 3 2
1
2 3
1
2 n
1
n 2 2
1
2 2
0
1 n
0
n 1 2
0
2 1 1
0
1 1
= + + +
= + + +
= + + +
Steps 1 through 3 are repeated to the extent possible.
) (m' c x a x a
. . . . .
) (2' c x a ... x a
(1) c x a ... x a x a
*
m n
*
n m m
*
m m
1
2 n
1
n 2 2
1
2 2
0
1 n
0
n 1 2
0
2 1 1
0
1 1
= +
= + +
= + + +
i) If n > m, the equations may look like this:
The final result may assume a number of different forms:
ii) If n = m, the equations may look like this:
*
n n
*
n n
*
n n
*
n n 1 - n
*
1 - n 1 - n
1
2 n
1
n 2 2
1
2 2
0
1 n
0
n 1 2
0
2 1 1
0
1 1
c x a
1)' - (n c x a x a
. . . . .
) (2' c x a ... x a
(1) c x a ... x a x a
=
= +
= + +
= + + +
This means there is a unique solution.
This means that there is a family of solutions with m variables expressed as
linear combination of remaining (n - m) variables. If the original equations were not
all linearly dependent, some of the defect equations may be identical.
Note that none of the main diagonal elements must be zero. Otherwise, re-order
equations.
iii) If the process terminates, i.e.
) (m' c 0
. .
) 1' (k c 0
) (k' c x a ...
. . . . .
) (2' c x a ... x a
(1) c x a ... x a x a
*
m
*
1 k
*
k n
*
n k
1
2 n
1
n 2 2
1
2 2
0
1 n
0
n 1 2
0
2 1 1
0
1 1
=
+ =
= +
= + +
= + + +
+
there are a number of possibilities:
I. If c
k+1
= c
k+2
= = c
m
= 0 and n > k, there is a family of solutions
II. If c
k+1
= c
k+2
= = c
m
= 0 and n = k, there is a unique solution
III. If any one of c
k+1
, , c
m
0, the equations are inconsistent or incompatible.
EXAMPLE on GAUSS(IAN) ELIMINATION
Given set of linear equations in augmented matrix form:

3 - 3 - 2 3 1
5 1 2 1 1
4 1 - 1 1 2
1 - 2 - 1 - 2 1
|
|
|
|
|
.
|
\
| (1)
(2)
(3)
(4)

2 - 1 - 3 1 0
6 3 3 1 - 0
6 3 3 3 - 0
1 - 2 - 1 - 2 1
|
|
|
|
|
.
|
\
|
(1)
(2') = 1 x (2) - 2 x (1)
(3') = 1 x (3) 1 x (1)
(4') = 1 x (4) 1 x (1)
1 2 -1 -2 -1
0 -3 3 3 6
0 0 -6 -6 -12
0 0 0 -72 -144
| |
|
|
|
|
\ .
1 2 -1 -2 -1
0 -3 3 3 6
0 0 -6 -6 -12
0 0 -12 0 0
| |
|
|
|
|
\ .
(1)
(2')
(3'') = - 3 x (3') + 1 x (2')
(4'') = - 3 x (4') - 1 x (2')
(1)
(2')
(3'')
(4''') = -12 x (3'') + 6 x (4'')
Back substitution yields:
x
4
= 2 x
3
= 0 x
2
= 0 x
1
= 3
Row-echelon
form
Given set of linear equations:
2 x
1
+ 3 x
2
- x
3
= 4
x
1
- 2 x
2
+ x
3
= 6
- x
1
- 12 x
2
+ 5 x
3
= 10
Augmented Matrix:
2 3 -1 4
1 -2 1 6
-1 -12 5 10
| |
|
|
|
\ .
2 3 -1 4
0 7 -3 -8
0 0 0 0
| |
|
|
|
\ .
Linear Row Operations:
Family of Solution:
2 x
1
+ 3 x
2
x
3
= 4 and 7 x
2
3 x
3
= - 8
Set x
2
= C
3 1
7 8 20 1
x = C + and x = - C
3 3 6 3
Given set of linear equations:
x
1
+ x
2
+ 3 x
3
= 1
x
1
+ 2 x
2
+ 5 x
3
= 2
3 x
1
+ 1 x
2
+ 5 x
3
= - 1
Augmented Matrix:
1 1 3 1
1 2 5 2
3 1 5 -1
| |
|
|
|
\ .
1 1 3 1
0 1 2 1
0 0 0 -4
| |
|
|
|
\ .
Linear Row Operations:
Inconsistency or Incompatibility
No Solution
EXAMPLE on GAUSS-JORDAN REDUCTION
Given set of linear equations in augmented matrix form:
1 2 -1 -2 -1
2 1 1 -1 4
1 1 2 1 5
1 3 2 -3 -3
| |
|
|
|
|
\ .
(1)
(2)
(3)
(4)
1 2 -1 -2 -1
0 1 -1 -1 -2
0 -1 3 3 6
0 1 3 -1 -2
| |
|
|
|
|
\ .
(1)
(2') = [(2) - 2 x (1)] / - 3
(3') = (3) - (1)
(4') = (4) - (1)
1 0 0 0 3
0 1 0 0 0
0 0 1 0 0
0 0 0 1 2
| |
|
|
|
|
\ .
1 0 0 -1 1
0 1 0 0 0

0 0 1 1 2
0 0 0 1 2
| |
|
|
|
|
\ .
1 0 1 0 3
0 1 -1 -1 -2

0 0 1 1 2
0 0 1 0 0
| |
|
|
|
|
\ .
(1') = - 2 x (2') + (1)
(2')
(3'') = [(3') + (2')] / 2
(4'') = [(4') + -1 x (2')] / 4
(1'') = -1 x (3'') + (1')
(2'') = (3'') + (2')
(3'')
(4''') = [-1 x (3'') + (4'')] x -1
(1''') = (4''') + (1'')
(2'')
(3''') = -1 x (4''') + (3'')
(4''')
SCALED PARTIAL (ROW) PIVOTING
RULE:
In order to select the pivot row and hence the pivot element at every operation j
(equal to 1 at the beginning), form the ratios
i j
i j
a
for each row, i
max a
The row i which has the maximum ratio is the pivot row, and the corresponding
element is the pivot element.
EXAMPLE:
The augmented matrix and the first set of above ratios are given below. The first
question is which one of the rows is to be the first pivot row.
Augmented Matrix Elimination step
1 2 -1 -2 -1
2 1 1 -1 4
1 1 2 1 5
1 3 2 -3 -3
| |
|
|
|
|
\ .
1 / 2
2 / 2 Max
1 / 2
1 / 3
The second equation has the largest ratio. So, it is to be the first pivot row.
Interchange first and second rows so that the second equation is the first row (or
the first pivot):
Given Augmented Matrix Pivoting Ratios
2 1 1 -1 4
1 2 -1 -2 -1
1 1 2 1 5
1 3 2 -3 -3
| |
|
|
|
|
\ .
2 1 1 - 1 4
0 1.5 -1.5 -1.5 -3
0 0.5 1.5 1.5 3
0 2.5 1.5 -2.5 -5
| |
|
|
|
|
\ .
The next step is to determine the second pivot. Find the ratios again:
Augmented Matrix Pivoting Ratios
2 1 1 - 1 4
0 1.5 -1.5 -1.5 -3
0 0.5 1.5 1.5 3
0 2.5 1.5 -2.5 -5
| |
|
|
|
|
\ .
-
1.5 / 1.5
0.5 / 1.5
2.5 / 2.5
The second and the fourth rows have the same maximum ratio. Choose any one of
them as the pivot. For instance, if the fourth row is chosen, interchange second and
fourth rows:
2 1 1 - 1 4
0 2.5 1.5 -2.5 -5
0 0.5 1.5 1.5 3
0 1.5 -1.5 -1.5 -3
| |
|
|
|
|
\ .
2 1 1 - 1 4
0 2.5 1.5 -2.5 -5
0 0 etc
0 0
| |
|
|
|
|
\ .
Augmented Matrix
Elimination Step
GAUSS-SEIDEL METHOD
This is similar to fixed-point
iteration method of root finding.
Given a set of linear equations:
1 1 1 1 2 2 1 n n 1
2 1 1 2 2 2 2 n n 2
n 1 1 n 2 2 n n n n
a x a x ... a x c
a x a x ... a x c
. . . . . . .
a x a x ... a x c
+ + + =
+ + + =
+ + + =
Re-write the equations in the form:
m+1 m m
1 1 1 2 2 1 n n 1 2 3 n
1 1
m+1 m+1 m
2 2 2 1 1 2 n n 2 1 3 n
2 2
m+1 m+1
n n n 1 1
n n
1
x (c - a x - ... - a x ) g (x , x , ..., x )
a
1
x (c - a x - ... - a x ) g (x , x , ..., x )
a
. . . . .
1
x (c - a x - ... -
a
= =
= =
=
m+1
n n-1 n-1 n 1 2 n-1
a x ) g (x , x , ..., x ) =
Note that the coefficients of the diagonal elements must be non-zero. Otherwise,
the set of equations must be re-ordered (re-cast) so that this is satisfied.
Convergence is ensured when this is satisfied for every row (equation). There
may or may not be convergence if this is not satisfied for one or more rows.
Start with an initial set of guesses for the unknowns, x
1
, x
2
, and carry out an
iterative procedure similar to that of fixed-point iteration.
Use the new values of the unknowns as soon as they are available at every
iteration step.
The criterion for convergence is:
a a for each row i
i i i j
i j
>
=
Gauss-Seidel Method
Example
1 2 3
1 2 3
1 2 3
x x 3 x 1
x 2 x 5 x 2
3 x x x 1
+ + =
+ + =
+ + =
Re-write the equations in the form
1 2 3
2 1 3
3 1 2
x 1 - x - 3 x
1 5
x 1 - x - x
2 2
x 1 - 3 x - x
=
=
=
Show that convergence criterion is not satisfied in any of the rows. Hence,
the iterations will diverge.
Convergence Criterion
1 < 1 + 3
2 < 1 + 5
1 < 3 + 1
x 3 - x - 1 x
0
3
0
2
1
1
= x
2
5
- x
2
1
- 1 x
0
3
1
1
1
2
= x - x 3 - 1 x
1
2
1
1
1
3
=
1 0 - 0 - 1 x
1
1
= =

2
1
(0)
2
5
- (1)
2
1
- 1 x
1
2
= =
2
5
-
2
1
- (1) 3 - 1 x
1
3
= =
8.0 x
2
1
= 3.25 x
2
2
= 26.25 - x
2
3
=
Hence, the iterations are diverging
Re-cast the equations such that the criterion is satisfied for at least most of the rows.
Gauss-Seidel Method
Example with re-casting
1 x 3 x x
2 x x 2 x 5
1 x x x 3
3 2 1
3 2 1
3 2 1
= + +
= + +
= + +
Re-write the equations in the form
3 / ) x - x - (1 x
2 / ) x - x 5 - (2 x
3 / ) x - x - (1 x
2 1 3
3 1 2
3 2 1
=
=
=
Show that convergence criterion is not satisfied in the second row. However,
the iterations will converge.
Convergence Criterion
3 > 1 + 1
2 < 5 + 1
3 > 1 + 1
Gauss-Seidel Iterations
n x
1
x
2
x
3
0 1.00000 1.00000 1.00000
1 -0.33333 1.33333 0.00000
2 -0.11111 1.27778 -0.05556
3 -0.07407 1.21296 -0.04630
4 -0.05556 1.16204 -0.03549
5 -0.04218 1.12320 -0.02701
6 -0.03206 1.09366 -0.02053
7 -0.02438 1.07121 -0.01561
8 -0.01853 1.05414 -0.01187
9 -0.01409 1.04116 -0.00902
Under Relaxation and Over Relaxation with Gauss-Seidel
m+1 * m m
1 1 1 2 2 1 n n 1 2 3 n
1 1
m+1 * m+1 m
2 2 2 1 2 2 n n 2 1 3 n
2 2
m+1 * m+1
n n n 1 1
n n
1
(x ) (c - a x - ... - a x ) g (x , x , ..., x )
a
1
(x ) (c - a x - ... - a x ) g (x , x , ..., x )
a
. . . . .
1
(x ) (c - a x
a
= =
= =
=
m+1
n n-1 n-1 n 1 2 n-1
- ... - a x ) g (x , x , ..., x ) =
( )
m
n
*
1 m
n
1 m
n
x ) - (1 x x + =
+ +
Define Relaxation Factor,
= 1 No relaxation
< 1 Under relaxation : To change a diverging case to a converging one
> 1 Over relaxation : To increase the rate of convergence
Gauss-Seidel Iterations with over relaxation
= 1.20
n x
1
x
2
x
3
0 1.00000 1.00000 1.00000
1 -0.60000 2.20000 -0.44000
2 -0.18400 1.57600 -0.06880
3 -0.16608 1.42432 -0.08954
4 -0.10070 1.27095 -0.05019
5 -0.06816 1.18042 -0.03486
6 -0.04459 1.11860 -0.02263
7 -0.02947 1.07827 -0.01499
8 -0.01942 1.05159 -0.00987
9 -0.01280 1.03402 -0.00651
Given : A parabola f
1
(x , y) = x
2
- 2 x - y + 0.5 = 0
and an ellipse f
2
(x , y) = x
2
+ 4 y
2
- 4 = 0
NON-LINEAR SYSTEM OF EQUATIONS
Solution with fixed-point iterations:
y) , x ( g
2
0.5 y - x
x
1
2
=
+
=
y) , x ( g
8
4 y 8 y 4 - x -
y
2
2 2
=
+ +
=
Convert to
-1
-0.5
0
0.5
1
1.5
2
2.5
3
3.5
-1 -0.5 0 0.5 1 1.5 2
sqrt(4-x*x)/2
-sqrt(4-x*x)/2
x*x-2*x+0.5
Start with (0,1)
n x
n
y
n
0 0 1
1 - 0.25 1
2 - 0.21875 0.99219
3 - 0.22217 0.99399
4 - 0.22231 0.99381
5 - 0.22219 0.99380
6 - 0.22222 0.99381
7 - 0.22221 0.99381
Start with (2,0)
n x
n
y
n
0 2 0
1 2.25 0
2 2.78125 - 0.13228
3 4.18408 - 0.60855
4 9.30755 - 2.48204
5 44.80623 - 15.89109
6 1011.995 - 392.6042
7 512263.2 - 205477.8
Theorem on convergence in fixed-point iteration:
Let the functions, g
1
(x,y) and g
2
(x,y) and their first partial derivatives be
continuous on a region that contains the fixed point. Assume that the starting point
is chosen sufficiently close to the fixed point and
1 2
g (x,y) g (x,y) < 1 and
x x
c c
+
c c
1 2
g (x,y) g (x,y) < 1
y y
c c
+
c c
then, the iterations converge to the fixed point.
Solution with Newton-Raphson:
Given : A catenary f
1
(x,y) = y 0.5 (e
x / 2
+ e
- x / 2
) = 0
and an ellipse f
2
(x,y) = 9 x
2
+ 25 y
2
225 = 0
Use Newtons method to determine the point of intersection of the curves that
resides in the first quadrant of the coordinate system. Starting with the initial
guess, (x
1
,y
1
) = (2.5,2.0),
0
1
2
3
4
5
6
7
-4 -2 0 2 4
sqrt((225-9*x*x)/25)
0.5*(exp(x/2)+exp(-x/2))
1 1
1 1
1 1
1 1
1 1
1 1 1
x ,y
x ,y
2 2
2 1 1
x ,y
x ,y
f f
x + y = - f (x ,y )
x y
f f
x + y = - f (x ,y )
x y
c c
A A
c c
c c
A A
c c
1 1
1 1
1 1
1 1
1 1
x ,y
x ,y
1 1 1 1 1 1
1 2
2 1 1 2 1 1
2 2
x ,y
x ,y
f f

x y
- f (x ,y ) - f (x ,y ) x x
= or J(f ,f ) =
y - f (x ,y ) y - f (x ,y )
f f

x y
| |
c c
|
c c
A A | | | | | | | | |
| | | | |
A A
c c
\ . \ . \ . \ .
|
|
c c
\ .
or
Expand both non-linear functions of two variables in Taylor series around a chosen
point (x
0
,y
0
) (as close to the real solution as possible).
( )
x/2 -x/2
1 1
f f 1
= - e - e = 1
x 4 y
c c
c c
2 2
f f
= 18 x = 50 y
x y
c c
c c
| |
| |
1 1 1 1
1 1 1 1
2 1
1 1 1 2 1 1
x ,y x ,y
1 2
1 2
2 1 1 1 1 1
x ,y x ,y
1 2
f f
- f (x ,y ) + f (x ,y )
y y
x =
det J(f ,f )
f f
- f (x ,y ) + f (x ,y )
x x
y =
det J(f ,f )
c c
c c
A
c c
c c
A
One may use Cramers rule to solve the two equations:
where
| | ( )
1 1
x /2 x /2
1 2 1 1
1
det J(f ,f ) = - e - e 50 y - 18 x
4
x
n
y
n
f
1
(x,y) f
2
(x,y) df
1
/dx df
1
/dy df
2
/dx df
2
/dy det(J) x y
2.5000 2.0000 0.11157 -68.7500 -0.8009 1.0000 45.0000 100.0000 -125.095 0.6387 0.4000
3.1387 2.4000 -0.10588 7.67331 -1.1488 1.0000 56.4978 120.0026 -194.366 -0.1048 -0.0145
3.0339 2.3854 -0.00338 0.10425 -1.0847 1.0000 54.6105 119.2737 -183.991 -0.0027 0.0003
3.0311 2.3858 -0.00000 0.00007 -1.0830 1.0000 54.5608 119.2932 -183.766 -0.0000 0.0000
3.0311 2.3858 -1.0E-12 3.3E-11 -1.0830 1.0000 54.5607 119.2932 -183.766 -8.E-13 1.E-13

ME 361-System of Equations-26 Sept 2013

Cargado por

Información del documento

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

ME 361-System of Equations-26 Sept 2013

Cargado por

Copyright:

Formatos disponibles

ME 361 NUMERICAL METHODS FOR ENGINEERS

Prof. Dr. Faruk Arn Fall 2013

También podría gustarte