Compufers & Chemical Engineering Vol. 8, No. 1, pp. l-7, 1984 00991354184 s3.w + .

w
Printed in Great Britain. Pergamon Press Ltd.

ON SOLVING LARGE SPARSE NONLINEAR

EQUATION SYSTEMS

HERN-SHANNCHEN~ and MARK A. STADTHJXRR*

Chemical Engineering Department, University of Illinois, Urbana, IL 61801, U.S.A.

(Received 24 September 1982; received for publication 4 May 1983)

Abstract-An approach toward solving large sparse nonlinear equation systems using a
modified Powell’s dogleg method is described. Schubert’s update is used either to update the
Jacobian, or to update the U factor of the Jacobian while holding the L factor constant. This
use of Schubert’s update differs from others since it is used in connection with a dogleg method
rather than the usual quasi-Newton method. Results on a set of standard test problems
indicate that this leads to an efficient and reliable method for sparse nonlinear problems.

Scope-There are many applications in which a large sparse system of nonlinear equations
must be solved. Some relevant applications in chemical engineering include the numerical
solution of partial differential equations by the finite difference approach, the simulation of
multi-stage separation processes by linearization [ 11,and process flowsheeting by the equation-
based approach [2]. This work has been motivated primarily by applications of the last two
types.
As discussed in more detail by Stadtherr & Hilton [2], a conimon concern in equation-based
process flowsheeting is supplying an initial guess good enough to assure convergence to the
solution. One way to ease such concerns is to use a sparse nonlinear equation solver that is
both efficient and has excellent global convergence properties. The work described here
represents progress toward this goal.
For full matrix problems, there have been several techniques [3-91 proposed to promote
convergence from a poor initial guess. All except Powell’s hybrid, or dogleg, method [9] are
variqtions of the continuation method and require the solution of a series of subproblems, even
in the case of a good initial guess. Powell’s method combines the good global convergence
properties of the steepest descent methods and the good local convergence properties of
Broyden’s quasi-Newton method [lo]. The result is a reliable and efficient method that has
proven quite successful [1 I].
Recently Chen & Stadtherr [12] described a modification of Powell’s method that makes
it even more efficient and reliable. In this paper we describe extensions of this modified Powell’s
method to the case in which a large sparse. problem must be solved. Of particular interest is
the method used to update the Jacobian. Can Schubert’s update, which has proven somewhat
unreliable when used in connection with a quasi-Newton method [13, 141, be used more
successfully in connection with a dogleg approach?

Conclusions and Sigticanc~The modified Powell’s dogleg method described by Chen &
Stadtherr [12] for full matrix problems can be effectively extended to sparse problems using
Schubert’s update either to update the Jacobian or to update the U factor of the Jacobian.
This indicates that while Schubert’s update may be unreliable when used as part of a
quasi-Newton solution method, as indicated by the results of Perkins & Sargent [13] and Mah
& Lin [14], it. can be used much more successfully in connection with the modified Powell’s
method. This is possible because the modified Powell’s method, a combination of ideas from
the quasi-Newton approach and the steepest descent approach, is much more reliable than the
quasi-Newton approach alone. Since the modified Powell’s method is efficient and has very
good global convergence properties, its application in connection with equation-based process
flowsheeting seems particularly attractive.

TPresent address: Union Carbide Corporation, South Charleston, WV 25303, U.S.A.

*Author to whom correspondence should be addressed.
CACE Vol. 8, No. I-A
1
2 H.-S. CHEN and
BACKGROUND
A detailed description of the modified Powell’s
method employed here is given by Chen &
Stadtherr[l2]. Before discussing its extension to
sparse problems, we briefly outline this full matrix
algorithm.
Given an initial estimate x,, of the solution, a
diagonal variable scaling matrix D,,an initial step
bound A, a convergence parameter 6, and a function
f(x) = 0 to be solved, proceed as follows:
1. Calculate the Jacobian B by forward difference.
2. Calculate a diagonal function scaling matrix D,,
and scale the Jacobian. D, is calculated so that the
largest element in each row of D,BD,-'has a mag-
nitude of 1.
3. Decompose the scaled Jacobian into its L and
U factors.
4. Calculate the search step pk and adjust pk if
necessary. Powell’s dogleg formula[9] is used to com-
pute pk. The initial value of the step bound A used in
this computation can be specified directly by the user,
or calculated from another user-provided parameter
as described in [12]. For problems in which all
variables must be nonnegative, pk is adjusted as
necessary to maintain the nonnegativity of the vari-
ables. This is a user-selected option.
5. Calculate f(x, + pk) and check for convergence.
Accept xk + pk as a solution if the convergence cri-
teria are satisfied.
6. Check for very slow convergence or non-
convergence. If little or no progress toward the
solution is detected the algorithm stops.
7. Check whether it is necessary to calculate a new
Jacobian. The Jacobian is re-evaluated occasionally
when the rate of convergence begins to slow consid-
erably. If this is necessary, return to Step 1.
8. Define xk+ , and update the step bound A. The
next iterate x~+, is defined so as to maintain a
monotonical decrease in the norm of the scaled
function Ofl.
9. Update the L and U factors and return to Step
4. Broyden’s update formula[lO] is used, along with
Bennett’s method[l5] for actually computing the
modified triangular factors L and U.
The advantages of the modified Powell’s method
are considered in detail in [12]. A recent study by
Hiebert [1 1] (see [16] for more detailed conclusions
than presented in [1 11) compared several nonlinear
equation solvers on a set of standard test problems,
and found HYBRD, which implements a version of
Powell’s method, and is part of Argonne’s MIN-
PACK library, to be the best performer overall. The
numerical results presented in [ 121for the same set of
test problems show that our implementation NEQLU
of the algorithm outlined above outperforms HY-
BRD both in terms of efficiency and reliability. A
listing of the source code for NEQLU, along with
some sample problems, is available [ 171.
At this point we would like to divert briefly to a
discussion of the usual implementation of nonlinear
equation solvers. In a typical implementation, such as
in [ 171,a calling program is used to call the nonlinear
equation solver and to pass as arguments external
routines for function and Jacobian evaluation. The
nonlinear equation solver then takes full control and
performs calculations until it finds a solution or it
fails. The drawbacks of the scheme are: (1) If the
N. A. STADTHFRR
function and/or Jacobian routines have to share some
parameters with the calling program then these
parameters must be passed via COMMON state-
ments, which is inconvenient and often a source of
error; (2) The calling program has no controi over the
ranges of acceptable values of the unknowns; (3) If
the function evaluation requires the solution of an-
other system of nonlinear equations then another
copy of the nonlinear equation solver with a different
subroutine name is needed; (4) The function may
become undefined for some values of the unknowns.
These problems can be solved by using the reverse
communication technique. The basic idea of the
technique is that the nonlinear equation solver only
suggests estimates of the solution, and when function
evaluations are required, it returns control to the
calling program. Detailed discussions of the tech-
nique can be found in Krogh[l8] and Connet [19].
When applying the modified Powell’s method to solve
process flowsheeting problems, we find an imple-
mentation that uses the reverse communication tech-
nique to be much more convenient to use than the
usual implementation.
SOLUTION OF SPARSE NONLINEAR SYSTEMS
Consider the solution of the n x n system of non-
linear equations f(x) = 0, where n is large and the
Jacobian is sparse. When such large sparse nonlinear
equation systems arise in equation-based flowsheeting
they are typically solved used the Newton-Raphson
(NR) method,.often in connection with some step-size
relaxation scheme. The numerical studies of Stadtherr
& Hilton [2], using the prototype equation-based
flowsheeting system SEQUEL, also described in [2],
indicate that NR with step-size relaxation is a reason-
ably reliable approach, at least when a relatively good
initial guess is supplied. An approach to improving the
global convergence properties of the NR method has
been suggested by Gorzcynski & Hutchison [20]. They
suggest that in computing a linearization during each
iteration, a first-order linearization be blended in some
proportion with the standard second-order NR linear-
ization. In general, the proportion of first-order linear-
ization used is decreased as one proceeds toward the
solution. The idea is that the first-order linearization
will provide reliability at the expense of speed when far
from the solution, while the more accurate second-
order linearization will provide greater speed as the
solution is approached. The numerical studies in [2]
show that this so-called quasilinear approach does
improve the success rate from poor initial guesses, but
that considerable improvement is still needed.
As emphasized by Shacham et al. [21] a serious
limitation of the NR method in connection with
equation-based flowsheeting is the need to calculate a
new Jacobian at each iteration. For some problems it
is feasible to calculate the Jacobian analytically, al-
though coding the analytical partial derivatives can be
a very time consuming and error prone process. For
other problems it may be necessary to calculate the
Jacobian using finite difference. The structure of the
sparse Jacobian for flowsheeting problems is such
that this can be done without too much difficulty
using the technique of Curtis et al. [22]. The idea
involved in this technique can be briefly demonstrated
using the occurrence matrix in Fig. 1. If we perturb
variable x, we will change functions f, and f4. If we
 On solving large sparse nonlinear equation systems
x x x x
I2 3 4
F X X following problems arise:
x x
F x x can easily be resolved using the finite difference scheme
3 discussed above of Curtis et al. [22] to calculate the
f x X
4 sequently. The user may also be able to supply an
Fig. 1. Occurrence matrix for example in text.
perturb xX we will change functions fiand f3. Since
these two variables have no equations in common, we
can perturb variables x, and xg at the same time and
evaluate columns 1 and 3 of Jacobian using one func-
tion evaluation. Using this idea, a tridiagonal Ja-
cobian can be evaluated using only three function
evaluations. For process flowsheeting problems, the
number of function evaluations required to evaluate a
Jacobian by this approach is typically a few multiples
of the number of components in the system.
Because of the possible difficulty involved in calcu-
lating the Jacobian for flowsheeting problems there
has been considerable interest in the use of quasi-
Newton methods [21]. In this case rather than re-
evaluate the Jacobian at each iteration, an approxi-
mate Jacobian is updated. For full matrix problems,
the most popular quasi-Newton method is Broyden’s
[lo]. His update formula for approximating the Ja-
cobian can be applied directly to the Jacobian, or it
can be put in a form allowing direct updating of the L
and U factors of the Jacobian, thus avoiding the need
to factor the Jacobian at each iteration. Broyden’s
update is not applicable to large sparse systems, how-
ever, because the approximate Jacobian it generates
will in general be full.
For sparse problems, the Broyden-type update pro-
posed by Schubert [23] is well-known. In this case the
Jacobian is updated under the constraint that all
known constant coefficients be actually held constant.
This is often implemented so that only the zero ele-
ments are actually held constant. A drawback in using
Schubert’s update is that, unlike Broyden’s update, it
cannot be applied directly to the factorization of the
Jacobian. Thus when Schubert’s update is used, the
approximate Jacobian must be resolved at each iter-
ation. Some approaches to avoiding this drawback are
discussed below. It should be noted that Perkins &
Sargent [ 131, as well as Mah & Lin [14], have found
quasi-Newton methods using Schubert’s update to be
unreliable in comparison to the NR method. The work
presented here differs from this earlier work in that
here Schubert’s update is incorporated in a dogleg
method.
We now proceed to discuss methods for extending
the modified Powell’s method described in [ 121to the
sparse matrix case. As in full matrix problems, the goal
is to combine the desirable features of the quasi-
Newton approach discussed above with the desirable
features of the steepest descent approach, thus obtain-
ing a method that is both efficient and very reliable.
EXTENDING THE MODIFIED POWELL’S ALGORITHM
The modified Powell’s algorithm outlined above
3
can be extended directly to the large, sparse matrix
case except for Steps 1, 3 and 9. For these steps the
1. It is no longer feasible in Step 1 to evaluate the
Jacobian by a simple finite difference scheme, because
the n function evaluations required would be too ex-
pensive for large n. For sparse problems, this difficulty
Jacobian initially and whenever required sub-
analytical Jacobian.
2. It is no longer feasible in Step 3 to solve the linear
system of equations by full-matrix LU decomposition,
because it requires n2 computer storages and 0(n3)
arithmetic operations.
3. Broyden’s method for updating the Jacobian is
not directly applicable in Step 9 because it will gener-
ate a full approximate Jacobian and thus requires ,*
computer storages.
The techniques used to overcome the first problem
have already been discussed above. We now proceed
to discuss the second and third problems in more
detail.
Solving large sparse linear systems
To solve a large system of sparse linear equations
efficiently one must of course use sparse matrix tech-
niques. The basic ideas of sparse matrix techniques are
to only store and operate on nonzero elements and to
avoid too many new nonzeros (fill-ins) during Gaus-
sian elimination. For more background on the solu-
tion of large sparse linear systems, especially those
arising in process flowsheeting problems, the reader is
referred to Stadtherr & Wood [24,25]. An extensive
survey of sparse matrix techniques and their applica-
tions is given by Duff [26].
As discussed in more detail in [24], an important
decision in designing a sparse matrix program is
whether to use the one-pass approach or the two-pass
approach. In the one-pass approach, the program
alternates between selecting one pivot and performing
one Gaussian elimination step. The most popular
method of this type is the Markowitz approach with
threshold pivoting, a popular implementation of
which is the well known code MA28 [27], a part of the
Harwell subroutine library. In the two-pass approach,
a complete tentative pivot sequence is first selected
based on sparsity considerations alone, and this ten-
tative pivot sequence is then used, perhaps with some
modifications, in performing the actual Gaussian
elimination. The most popular reordering method for
selecting the initial tentative pivot sequence is proba-
bly the P4 algorithm of Hellerman & Rarick [28].
In a recent study by Stadtherr & Wood [24], it was
found that for process flowsheeting matrices neither
the Markowitz nor the P4 technique works well. This
appears to be because some equations (material bal-
ance equations) contain only a few variables while
other equations (equilibrium and energy balance
equations) may contain several more variables. For
such matrices, the usual sparse matrix techniques
tend to first choose pivots from the equations with
only a few elements, thus destroying the block struc-
ture inherent in these problems by taking successive
pivots from several different blocks. Stadtherr &
Wood [24] suggest a block reordering algorithm to
4 H.-S. CHEN and M. A. STAD~R~~R
exploit the structure of process flowsheeting matrices.
They first construct a block-occurrence matrix and
apply a P4-type reordering algorithm (SPKZ) to reor-
der the block matrix. Then a simple reordering algo-
rithm (SPKl) is applied to reorder the individual
equations and variables, while restricting the spike
selection to avoid destroying the block reordering
originally identified. This algorithm (BLOKS) was
found to be very efficient and to give consistently good
reorderings for flowsheeting problems.
In a related study [25] Stadtherr & Wood also
compared some algorithms for performing the actual
elimination. A modified version of the NSPIV pro-
gram [29], LUlSOL, was found to be among the
most effective codes. It should be noted that the
version of LUlSOL used in [25] did not attempt to
exploit the presence of irreducible blocks in the
matrix and thus failed to solve some of the larger test
problems due to storage limitations. An updated
version of LUlSOL does exploit this type of struc-
ture, limiting fill-in to within the irreducible blocks,
and thus in general permitting the solution of larger
problems. In NSPIV partial pivoting is used to
maintain numerical stability. However, it does not
take advantage of the a priori reordering scheme
used. In LUlSOL we change the pivoting strategy
from partial pivoting to threshold pivoting to take
advantage of the a priori reordering. The per-
formance of the two-pass codes NSPIV and LUl-
SOL, using the BLOKS reordering algorithm, as well
as the one-pass code MA28, on some process
flowsheeting matrices is shown in Table 1. The results
here are taken from Stadtherr & Wood [25] except for
the last run, which uses the update of LUlSOL
discussed above. It is clear from this table that we can
solve fairly large systems of linear equations with
reasonable CPU time. And since the largest problem
can be solved with LUlSOL in roughly 100,000
words of core, storage requirements are also reason-
able. As a final note we add that if the structure of
the L and U factors is saved when performing the first
factorization of the Jacobian, then this information
can be used to reduce the solution time required for
subsequent factorizations of Jacobians with the same
structure. For instance, subsequent factorizations of
the 4633 equations problem require only 2.013 set
with LUISOL.
Updating the sparse Jacobian
The problem here is to maintain the sparsity of the
Jacobian when it is updated. There are at least three
approaches for solving this problem:
1. Use Schubert’s update, as described above.
Some characteristics of this approach are: (i) An LU
factorization must be performed at each iteration; (ii)
Table 1. Solution times for some sparse nonlinear equation
solvers on process flowsheeting problems. N is the number
of variables; NZ is the number of nonzero elements. Failures
were due to instieient core storage. Except for the largest
problem, data is taken from Stadtherr & Wood [25]
SIZE SOLUTION TIME (SEC1
MA28 NSPIV LUISOL
3Y2 I.633 0.548 6.384
584 3963 I.187 1.698 0.509
I@68 6254 3.700 0.775 0 546
1564 9369 6.002 1.445 0 981
4633 33443 FAIL FAIL 4 681
During LU factorization the approximate Jacobian
and its U factor must be saved; (iii) The convergence
of Schubert’s update is q-superlinear [30]. The follow-
ing two approaches avoid the drawback that an LU
factorization is needed at every iteration.
2. In this case, Broyden’s update is used to update
the Jacobian, but in order to maintain sparsity the
rank one updates used are stored separately, as
suggested by Gallun & Holland [31]. Some character-
istics of this approach are: (i) An LU factorization is
only performed when a new Jacobian is calculated;
(ii) During LU factorization both the L and U factors
of the Jacobian must be saved, (iii) Two extra vectors
of length n are needed each time the Jacobian is
updated; (iv) The number of arithmetic operations
required to obtain the correction step will increase
with the number of rank one updates stored; (v) The
convergence of Broyden’s update is q-superlinear
1321.
3. In this case, Schubert’s update is used on the U
factor of the Jacobian while keeping the L factor
fixed, as suggested by Dennis & Marwil [33]. Some
characteristics of this approach are: (i) An LU fac-
torization is only performed when a new Jacobian is
calculated; (ii). Both L and U factors of the Jacobian
must be saved; (iii) The storage and arithmetic oper-
ations will not be affected by the number of Jacobian
updates, unlike the previous case; (iv) The con-
vergence of this update is also q-superlinear [33].
We have employed both the first and third ap-
proaches discussed above. The performance of these
two approaches is compared below.
Before proceeding to some numerical results, we
summarize the actions taken to extend our modified
Powell’s method to the solution of large sparse
problems.
1. The finite difference approximation scheme of
Curtis et al. is used to evaluate the Jacobian initially
and whenever required subsequently.
2. A two-pass approach is used to perform the LU
factorization required. For flowsheeting problems we
recommend the algorithm BLOKS [24] for the reor-
dering phase.
3. Schubert’s update is used either to update the
Jacobian or its U factor.
NUMERICAL RESULTS
In order to study the reliability and efficiency of the
modified Powell’s method on sparse problems, we
have used a number of relatively small test problems,
and apply three versions of the method, differing in
how the Jacobian is updated. Perkins & Sargent [13]
have applied two versions of the NR method and
three versions of the quasi-Newton method using
Schubert’s update to these same problems. Thus we
can compare the reliability of these approaches to the
modified Powell’s method. We also make comparisons
based on the efficiency and reliability of the different
methods used to update the Jacobian in the modified
Powell’s method.
Test problems
Five standard test problems, commonly used in
comparisons of nonlinear equation solvers, are con-
sidered:
1. Discrete boundary value problem.
 On solving large sparse nonlinear equation systems
2. Broyden’s tridiagonal function.
3. Broyden’s banded function.
4. Extended Rosenbrock function.
5. Extended Powell’s singular function.
Perkins and Sargent also use a sixth, less standard
problem. These are all variably dimensioned prob-
lems, and each problem was tried with three different
dimensions: 50,100 and 200 variables. Also in order to
test the performance of the methods considered on
poor initial guesses, three starting points are used,
x,, = x, (the “standard” starting point for each prob-
lem), x,, = lox,, and x, = 100x,.
Methods compared
The five methods considered by Perkins and Sargent
are:
1. Finite-difference Newton-Raphson with per-
turbation sizes proportional to variable values.
2. Finite difference Newton-Raphson method with
perturbation sizes proportional to function values.
3. Quasi-Newton method with non-constant ele-
ments of the sparse Jacobian updated by Schubert’s
update.
4. Quasi-Newton method with nonzero elements of
the sparse Jacobian updated by Schubert’s update.
5. Same as 4 except the Jacobian is recalculated
when convergence becomes sufficiently slow.
Tne new methods considered are:
6. Modified Powell’s method with Jacobian up-
dated by Broyden’s method (full matrix update).
7. Modified Powell’s method with nonzero ele-
ments of the sparse Jacobian updated by Schubert’s
method.
8. Modified Powell’s method with the sparse upper
triangular factor of the Jacobian matrix updated by
Schubert’s method (L factor held constant).
Reliability
The reliability of the eight methods tested is sum-
marized in Table 2, which shows the number of fail-
ures of each method on these test problems. The re-
sults for methods l-5 are from Perkins & Sargent [13],
as adjusted to account for the fact that they include a
sixth test problem. To assure a conservative adjust-
ment, we assumed that methods 3-5 always failed in
the sixth test problem and then reduced the number of
failures reported in [13] accordingly. As a result the
number of failures shown in Table 2 for methods 3-5
on these five problems is probably somewhat under-
stated. Regarding these results, the following com-
ments are in order:
Table 2. Number of failures on test problem set. Data for methods l-5 are taken from Perkins & Sargent
[13], adjusted as explained in the text
___WZ_
METHOD: 1
S.P.: X 8 0 3
10x 8 0
IBOX B 0
OVERALL: 0 0
5
1. Both versions of the NR method (methods 1
and 2) proved reliable on this set of test problems.
2. All three versions of Perkins and Sargent’s
implementation of the quasi-Newton method using
Schubert’s update (methods 3-5) failed on a large
number of problems. This is not inconsistent with the
results of Mah & Lin [14], who also found that their
implementation of Schubert’s method became
inefficient and unreliable when poor initial guesses
were used.
3. The version of the modified Powell’s method
that does not take the sparsity of the Jacobian into
account when updating the Jacobian (method 6)
failed on problems 4 and 5. This is not surprising and
is partially because Broyden’s update treats these as
problems of dimensions 50-200, while actually these
problems are made of smaller subproblems of dimen-
sion 2 or 4.
4. Both versions of the modified Powell’s method
that use Schubert’s method, either to update the
Jacobian (method 7) or to update the U factor of the
Jacobian (method 8) solved all of the problems. This
suggests that the low reliability of methods 3-5 is not
due to the use of Schubert’s update, but is due to the
method used to determine the correction step, or
perhaps due to the implementation itself. The results
here indicate that the modified Powell’s method,
which combines ideas from the quasi-Newton ap-
proach and the steepest descent approach, is much
more reliable than the quasi-Newton approach alone,
thereby permitting a more successful application of
Schubert’s update than has been reported elsewhere
[13, 141.
E$ect of Jacobian updating scheme
Table 3 shows how the Jacobian updating scheme
used in the modified Powell’s method affects the
number of iterations and the number of Jacobian
evaluations required for each problem. The following
comments are in order:
1. The dimensionality of the problem has very
little effect on the number of iterations or Jacobian
evaluations required to solve the problems.
2. The method that updates the Jacobian by Schu-
bert’s method usually gives convergence in the fewest
number of iterations. However, in some applications,
the method that updates the U factor of the Jacobian
may still be preferred because it requires far fewer LU
factorizations of the Jacobian.
3. In our experience the NR method usually re-
quires roughly half as many iterations as the modified
_~-___Q-+__5- _S-__-M;P_-_:6_
3 3 6 @ 0
6 6 6 6 0 0
6 9 6 6 8 B
15 18 15 18 8 0
6 H.-S. CH~J and M. A. STADTHERR

Table 3. Efficiency of different approaches for updating the Jacobian in the modified Powell’smethod.
Results are given in terms of number of iterations followed by number of Jacobian evaluations
N: ______58 __-__- ____-_,a8 __-_- ______200_____
HETHOD:6 7 8 6 7 8 6 7 *

PRGSLEH: S.P.:

I X 4/I 4/i .5/l 4/I 4/l 5/l 4/1 4,) 5/f

10X 7/l 6/l 3/l 7/l S/l I,/, 7/l 5/l 12/l
,00x 23/2 16/1 22/2 2312 14/l 2913 23/2 13/l 25/3

2 X 12/I 7/1 8/l 12/l 7/l 8/l 12/l 7/1 8/l

10X 22/2 16,2 17/2 2112 l6/2 1712 22/2 ,6/2 17/2
,e0x 25/2 21/2 20/2 25/2 20/2 2112 25/2 2012 2!/2

3 X 18/l 16/l IS/2 18/l ,4/I 15/2 1*/t 14/l 15/2

,0X 24/2 21/2 25/2 2w2 22/2 23,2 25/2 23/2 2913
,00x 35/2 30/2 35/2 34/2 31/2 38/2 34/2 3117 36/3

4 X FAIL 8/l 9/2 FAIL 8/l 912 FAIL 8/l 912

10X FAIL 1312 Q/2 '3'2 9/2 '3'2 Q/2
IKlX FAIL 912 912 :::: Q/2 Q/2 :::: Q/2 9/L

5 I0E FAIL 38/2 53/4 FAIL 38/2 61,s FAIL 39/2 62,s
FAIL 4212 7216 FAIL 43/2 74,6 FAIL 4412 78/6
lE0X FAIL 47/2 74/S FAIL 48/2 78/6 FAIL 4912 7616

Powell’s method. If one were to try to choose between
the NR method and the modified Powell’s methods
(methods 7 and 8) on the basis of overall numerical
efficiency on large sparse problems, the choice would
depend on the relative expense of function evalu-
ation, Jacobian evaluation, and LU factorization of
the Jacobian. If both the evaluation and the fac-
torization of the Jacobian are relatively cheap, then
the NR method would Seem preferable, assuming
that no convergence difficulties arise. If there are such
difficulties, it may be desirable to use the modified
Powell’s method, which may be implemented so that
the Jacobian is simply re-evaluated at each iteration,
rather than updated with some quasi-Newton for-
mula. This is feasible if the Jacobian is easy to
evaluate, and in general reduces the number of
iterations required by the modified Powell’s method.
If the Jacobian is relatively hard to evaluate but
cheap to factor, then the modified Powell’s method
using Schubert’s update on the Jacobian (method 7)
looks attractive. If the Jacobian is relatively hard to
factor, then the modified Powell’s method using
Schubert’s update on the U factor of the Jacobian
while holding the L factor constant [33] may be
preferred. While it is not evident in this set of test
problems, the modified Powell’s method in general
exhibits much better global convergence properties
than the NR method. This must be of course also be
considered in selecting a nonlinear equation solver.
The modified Powell’s method, as extended here to
sparse problems, has proved to be a reliable and
efficient approach on the initial set of test problems
considered here. Numerical studies are currently un-
derway, using the equation-based flowsheeting sys-
tem SEQUEL [2], to determine the performance of
the modified Powell’s method on realistically large
flowsheeting problems.
Acknowledgemenrs-This work has been supported by the
National Science Foundation under Grant CPE 80-12428.
This work was also presented at the AIChE Annual Meet-
ing, Los Angeles, November, 1982.
REFERENCES
1. M. A. Stadtherr & M. A. Malachowski, On efficient
solution of complex systems of interlinked multistage
separators. Comput. Chem. Engng 6, 121 (1982).
2. M. A. Stadtherr 8 C. M. Hilton, Development of a new
equation-based process flowsheeting system: Numerical
studies. In Selected Topics on Computer Aided Process
Design and Analysis (Edited by R. S. H. Mah & G. V.
Reklaitis), AIChE Symposium Series Vol. 78(214), p.
12(1982).
3. C. G. Broyden, A new method of solving simultaneous
nonlinear equations. Comput. .Z. 12, 94 (1969).
4. W. E. Borsage, Infinite dimensional iterative methods
and applications. Publication 320-2347, IBM Scientific
Center, Houston, Texas (1968).
5. P. T. Boggs, The solution of nonlinear systems of equa-
tions by A-stable integration techniques. SIAM J. Nu-
mer. Anal. 8, 767 (1971).
6. S. N. Chow, J. Mallet-Paret & J. A. Yorke, Finding
zeros of maps: homotopy methods that are constructive
with probability one. Math. Comput. 32, 887 (1978).
7. D. F. Davidenko, On a new method of numerical solu-
tion of systems of nonlinear equations. Doklady Akad.
Nauk SSSR (N.S.) 88, 611 (1953).
8. J. Davis, The solution of nonlinear operalor equations
with critical points. Ph. D. Thesis. Oregon State Univer-
sity, Corvalis (1966).
9. M. J. D. Powell, A hybrid method for nonlinear equa-
tions. In Numerical Methods for Nonlinear Algebraic
Equations (Edited by P. Rabinowitz). Gordon & Breach,
New York (1970).
IO. C. G. Broyden, A class of methods for solving nonlinear
simultaneous equations. Math. Comput. 19, 577 (1965).
11. K. L. Hiebert, An evaluation of mathematical software
that solves systems of nonlinear equations. Trans. Math.
software, 8, 5 (1982).
12. H. S. Chen & M. A. Stadtherr, A modification of
Powell’s dogleg method for solving systems of nonlinear
equations. Comput. Chem. Engng 5, 143 (1981).
13. J. D. Perkins & R. W. H. Sargent, SPEEDUP: A com-
puter program for steady-state and dynamic simulation
and design of chemical processes. In Selected Topics on
Computer-Aided Process Design and Analysis (Edited by
R. S. H. Mah & G. V. Reklaitis), AIChE Symposium
Series Vol. 78(214), p. 1 (1982).
14. R. S. H. Mah & T. D. Lin, Comparison of modified
Newton’s methods. Compur. Chem. Engng 4, 75 (1980).
15. J. M. Bennett, Triangular factors of modified matrices.
Numer. Math. 7, 217 (1965).
16. K. L. Hiebert, A comparison of software which solves
systems of nonlinear equations, Sandia Tech. Rep.
SAND 80-018 1. Sandia National Laboratories, Albu-
querque, New Mexico (1980).
17. H. S. Chen & M. A. Stadtherr, NEQLU-A Fortran
subroutine for solving systems of nonlinear equations.
Submitted for publication (1982).
 On solving large sparse nonlinear equation systems
18. F. T. Krogh, An integrator design. JPL Tech. Memo-
randum 33-479, Pasadena (1971).
19. G. H. Connet, On the structure of zero finders. BIT, 17,
170 (1977).
20. E. W. Gorczynski & H. P. Hutchison, Towards a quasi-
linear process simulator: I. Fundamental ideas. Compur.
Chem. Engng 2, 189 (1978).
21. M. Sacham, S. Macchietto, L. F. Stutzman & P. Ba-
bcock, Equation oriented approach to process
flowsheetine. Comout. Chem. En~nrr 6, 79 (1982).
22. A. R. Cur&, M. J. D. Powell & J. K. Reid, Gn the
estimation of sparse Jacobian matrices. J. Inst. Math.
Appl. 13, 117 (1974).
23. L. K. Schubert, Modification of a quasi-Newton method
for nonlinear equations with a sparse Jacobian. Math.
Comput. 25, 27 (1970).
24. M. A. Stadtherr & E. S. Wood, Sparse matrix methods
for equation-based chemical process flowsheeting-I.
Reordering phase. Compur. Chem. Engng 8, 9 (1984).
25. M. A. Stadtherr & E. S. Wood, Sparse Matrix methods
for equation-based chemical process flowsheeting-II.
Numerical phase. Comput. Chem. Engng 8, 19 (1984).
26. I. S. Duff, A survey of sparse matrix research. Proc.
IEEE, 65, 500 (1977).
7
27. I. S. Duff&J. K. Reid, Some design features of a sparse
matrix code. Trans. Math. Sofrw&e, 5, 18 (1979).-
28. E. Hellerman & D. Rarick. The uartitioned ureassinned
pivot procedure (P4). In’ Spa&e Matrices- and ?heir
Applications (Edited by D. J. Rose & R. A. Willoughby).
Plenum Press, New York (1972).
29. A. H. Sherman, Algorithm 533. NSPIV, A Fortran
subroutine for sparse Gaussian elimination with partial
pivoting. Trans. Math. Software, 4, 391 (1978).
30. E. Marwil, Convergence results for Schubert’s method
for solving sparse nonlinear equations. SIAM J. Numer.
Anal. 16, 488 (1979).
31. S. E. Gallun & C. D. Holland, A modification of Bro-
yden’s method for the solution of sparse systems-with
application to distillation problems described by non-
ideal thermodynamic functions. Comput. Chem. Engng
4, 33 (1980).
32. C. G. Broyden, J. E. Dennis, Jr. & J. J. More, On the
local and superlinear convergence of quasi-Newton
methods. J. Inst. Math. Appl. 12, 223 (1973).
33. J. E. Dennis, Jr. $ E. S. Marwil, Direct secant update of
matrix factorizations. Math. Comput. 38, 459 (1982).