7 vistas

Cargado por alecandro_90

- Jacobi An
- Solution Set Math II
- MapleProject07_m032
- Economic Operations
- SVM3
- Dynamic Simulation of Polymerization Process
- hsc-commerce-2014-october-maths1.pdf
- 2004b-Lin and Miller- Tabu Search Algorithm for Chemical Process, 2287-2306
- optimization
- Quiz 2 Arithmetic Sequence and Means
- Year 3 Choices 13 14(1)
- Qe Trick@Thats My Way
- SVM tutorial
- alg1_ch04_extrapractice
- Distributed Joint Resource Allocation in Primary and Cognitive Wireless Networks
- Retest
- 34
- Dynamic Error
- Math 151-16 Exam 3 Practice Solutions
- Inequalities

Está en la página 1de 9

Introduction

We turn now to the study of minimization with constraints. More specically, we will tackle the following problem: minimize f (x) subject to h1 (x) = 0 . . . hm (x) = 0 where x and the functions f , h1 , . . . , hm are continuous, and usually assumed to be in C 2 (i.e., with continuous second partial derivatives). When f and hj s are linear, the problem is a linear programming one and can be solved using the simplex algorithm [1]. Hence we would like to focus on the case that these functions are nonlinear. In order to gain some intuition, let us look at the case where n = 2 and m = 1. The problem becomes minimize f (x, y ) subject to h(x, y ) = 0, x, y R The constraint h(x, y ) = 0 denes a curve. Dierentiate the equation with respect to x: h h dy + = 0. x y dx

dy h ). And the gradient of the curve is h = ( h The tangent of the curve is T (x, y ) = (1, dx x , y ). So the above equation states that T h = 0;

Rn ,

namely, the tangent of the curve must be normal to the gradient all the time. Suppose we are at a point on the curve. To stay on the curve, any motion must be along the tangent T .

h T h(x, y ) = 0

In order to increase or decrease f (x, y ), motion along the constraint curve must have a component along the gradient of f , that is, f T = 0.

h(x, y ) = 0 f

T f

At an extremum of f , a dierential motion should not yield a component of motion along f . Thus T is orthogonal to f ; in other words, the condition f T = 0 must hold. Now T is orthogonal to both gradients f and h at an extrema. This means that f and h must be parallel. Phrased dierently, there exists some R such that f + h = 0. (1)

(x2 , y2 )

h(x, y ) = 0 (x1 , y1 ) f (x , y ) h f = c5

f = c4 f = c1 f = c3 f = c

The gure above explains condition (1) by superposing the curve h(x, y ) = 0 onto the family of level curves of f (x, y ), that is, the collection of curves f (x, y ) = c, where c is any real number in the range of f . In the gure, c5 > c4 > c3 > c > c1 . The tangent of a level curve is always orthogonal to the gradient f . Otherwise moving along the curve would result in an increase or decrease of the value of f . Imagine a point moving on the curve h(x, y ) = 0 from (x1 , y1 ) to (x2 , y2 ). Initially, the motion has a component along the negative gradient direction f , resulting in the decrease of the value of f . This component becomes smaller and smaller. When the moving point reaches (x , y ), the motion is orthogonal to the gradient. From that point on, the motion starts having a 2

component along the gradient direction f so the value of f increases. Thus at (x , y ), f achieves its local minimum. The motion is in the tangential direction of the curve h(x, y ) = 0, which is orthogonal to the gradient h. Therefore, at the point (x , y ) the two gradients f and h must be collinear. This is what equation (1) says. Let c be the local minimum achieved at (x , y ) . It is clear that the two curves f (x, y ) = c and h(x, y ) = 0 are tangent at (x , y ). Suppose we nd the set S of points satisfying the equations h(x, y ) = 0, f + h = 0, for some .

Then S contains the extremal points of f subject to the constraints h(x, y ) = 0. The above two equations constitute a nonlinear system in the variables x, y, . It can be solved using numerical techniques, for example, Newtons method.

Lagrangian

It is convenient to introduce the Lagrangian associated with the constrained problem, dened as F (x, y, ) = f (x, y ) + h(x, y ). Note that F =

f x f y

+ h x h

+ h y

= (f + h, h) .

Thus setting F = 0 yields the same system of nonlinear equations we derived earlier. The value is known as the Lagrange multiplier. The approach of constructing the Lagrangians and setting its gradient to zero is known as the method of Lagrange multipliers.

Example 1 Find the extremal values of the function f (x, y ) = xy subject to the constraint h(x, y ) = x2 y2 + 1 = 0. 8 2

y+

2

x 4 x + y

2

= 0, = 0, = 8.

x + 4y

Eliminate x from (2) and (3): 2 y = 0, 4 which, since y = 0 (otherwise x = 0 would hold by (2)), leads to y 2 = 4 and = 2.

So there are four extremal points of f subject to the constraint h: (2, 1), (2, 1), (2, 1), and (2, 1). The maximum value 2 is achieved at the rst two points while the minimum value 2 is achieved at the last two points.

c = 2

c=2

c = 1

c=1

Graphically, the constraint h denes an ellipse. The level contours of f are the hyperbolas xy = c, with |c| increasing as the curves move out from the origin.

General Formulation

Now we would like to generalize to the case with multiple constraints. Let h = (h1 , . . . , hm )T be a function from Rn to Rm . Consider the constrained optimization problem below. minimize f (x) subject to h(x) = 0. Each constraint equation hj (x) = 0 denes a constraint hypersurface S of generally nm dimensions in the space Rn . And this surface is smooth provided hj (x) C 1 . A curve on S is a family of points x(t) S with a t b. The curve is dierentiable if x = dx/dt exists, and twice dierentiable if x = d2 x/dt2 exists. The curve passes through a point x if x = x(t ) for some t , a t b. 4

h2 (x) = 0

h1 (x)T

h(x )T

h2 (x)T x S

x h(x) = 0 S x(t) = 0

T

(a) (b)

h1 (x) = 0

Figure 1: Tangent space at x on a surface dened by (a) one constraint h(x) = 0 (shown as a tangent plane) and (b) two constraints h1 (x) = 0 and h2 (x) = 0 (shown as a tangent line).

The tangent space at x is the subspace of Rn spanned by the tangents x (t ) of all curves x(t) on S such that x(t ) = x . In other words, the tangent space is the set of the derivatives at x of all surface curves through x . Denote this subspace as T . See Figure 1. A point x satisfying h(x) = 0 is a regular point of the constraint if the gradient vectors h1 (x), . . ., hm (x) are linearly independent. From our previous intuition, we expect that f v = 0 for all v T at an extremum. This implies that f lies in the orthogonal complement T of T . We would like to claim that f can be composed from a linear combination of the hi s. This is only valid provided that these gradients span T , which is true when the extremal point is regular. Theorem 1 At a regular point x of the surface S dened by h(x) = 0, the tangent space is the same as { y | h(x)y = 0 }, where the matrix h1 . h = . . . hm

The rows of the matrix h(x) are the gradient vectors hj (x), j = 1, . . . , m. The theorem says that the tangent space at x is equal to the null space of h(x). Thus, its orthogonal complement T must equal the row space of h(x). Hence the vectors hj (x) span T .

Example 2. Suppose h(x1 , x2 ) = x1 . Then h(x) = 0 yields the x2 axis. And h = (1, 0) at all points. So every x R2 is regular. The tangent space is also the x2 axis and has dimension 1.

If instead h(x1 , x2 ) = x2 1 then h(x) = 0 still denes the x2 axis. On this axis, h = (2x1 , 0) = (0, 0). Thus, no point is regular. The dimension of T , which is the x2 axis, is still one, but the dimension of the space { y | h y = 0 } is two.

Lemma 2 Let x be a local extremum of f subject to the constrains h(x) = 0. Then for all y in the tangent space of the constraint surface at x , f (x )y = 0. The next theorem states that the Lagrange multiplier method is a necessary condition on an extremum point. Theorem 3 (First-Order Necessary Conditions) Let x be a local extremum point of f subject to the constraints h(x) = 0. Assume further that x is a regular point of these constraints. Then there is a Rn such that f (x ) + T h(x ) = 0. The rst order necessary conditions together with the constraints h(x ) = 0 give a total of n + m equations in n + m variables x and . Thus a unique solution can be determined at least locally.

Example 3. We seek to construct a cardboard box of maximum volume, given a xed area of cardboard. Denoting the dimensions of the box by x, y, z , the problem can be expressed as maximize subject to xyz c , xy + yz + xz = 2

c where c > 0 is the given area of cardboard. Consider the Lagrangian xyz + (xy + yz + xz 2 ). The rst-order necessary conditions are easily found to be

yz + (y + z ) = 0, xz + (x + z ) = 0, xy + (x + y ) = 0, together with the original constraint. Before solving the above equations, we note that their sum is (xy + yz + xz ) + 2(x + y + z ) = 0, which, under the original constraint, becomes c/2 + 2(x + y + z ) = 0.

Hence, it is clear that = 0. Neither of x, y, z can be zero since if either is zero, all must be so according to (5)(7). To solve the equations (5)(7), multiply (5) by x and (6) by y , and then subtract the two to obtain (x y )z = 0.

Operate similarly on the second and the third to obtain (y z )x = 0. Since no variables can be zero, it follows that x=y=z= c 6

is the unique solution to the necessary conditions. The box must be a cube.

We can derive second-order conditions for constrained problems, assuming f and h are twice continuously dierentiable. Theorem 4 (Second-Order Necessary Conditions) Suppose that x is a local minimum of f subject to h(x) = 0 and that x is a regular point of these constraints. Denote by F the Hessian of f , and by Hi the Hessian of hi , for 1 i m. Then there is a Rm such that f (x ) + T h(x ) = 0. The matrix L(x ) = F (x ) +

i=1 m

i Hi (x )

(8)

is positive semidenite on the tangent space { y | h(x )y = 0 }. Theorem 5 (Second-Order Sucient Conditions) Suppose there is a point x satisfying h(x ) = 0, and a such that f (x ) + T h(x ) = 0. Suppose also that the matrix L(x ) dened in (8) is positive denite on the tangent space { y | h(x )y = 0 }. Then x is a strict local minimum of f subject to h(x) = 0.

Example 4. consider the problem minimize subject to The rst order necessary conditions become x2 + x3 + x1 + x3 + x1 + x2 + = 0, = 0, = 0. x1 x2 + x2 x3 + x1 x3 x1 + x2 + x3 = 3

We can solve these three equations together with the one constraint equation and obtain x1 = x2 = x3 = 1 Thus x = (1, 1, 1)T . and = 2.

Now we need to resort to the second-order sucient conditions to determine if the problem achieves a local maximum or minimum at x1 = x2 = x3 = 1. We nd the matrix L(x ) = = F (x ) + H (x ) 0 1 1 0 1 0 1 + 0 1 1 0 0 0 1 1 1 0 1 1 1 0 0 0 0 0 0 0

is neither positive nor negative denite. On the tangent space M = { y | y1 + y2 + y3 = 0 }, however, we note that y T Ly = y1 (y2 + y3 ) + y2 (y1 + y3 ) + y3 (y1 + y2 )

2 2 2 = (y1 + y2 + y3 ) < 0,

Thus L is negative denite on M and the solution 3 we found is at least a local maximum.

Inequality Constraints

minimize f (x) subject to h(x) = 0 g (x) 0

where h = (h1 , . . . , hm )T and g = (g1 , . . . , gp )T . A fundamental concept that provides a great deal of insight as well as simplies the required theoretical development is that of an active constraint. An inequality constraint gi (x) 0 is said to be active at a feasible point x if gi (x) = 0 and inactive at x if gi (x) < 0. By convention we refer to any equality constraint hi (x) = 0 as active at any feasible point. The constraints active at a feasible point x restrict the domain of feasibility in neighborhoods of x. Therefore, in studying the properties of a local minimum point, it is clear that attention can be restricted to the active constraints. This is illustrated in the gure below where local properties satised by the solution x obviously do not depend on the inactive constraints g2 and g3 . Assume that the functions f , h = (h1 , . . . , hm )T , g = (g1 , . . . , gp )T are twice continuously dierentiable. Let x be a point satisfying the constraints h(x ) = 0 and g (x ) 0, and let J = { j | gj (x ) = 0 }. Then x is said to be a regular point of the above constraints if the gradient vectors hi (x ), gj (x ), 1 i m, j J are linearly independent. Now suppose this regular point x is also a relative minimum point for the original problem (1). Then it can be shown that there exists a vector Rm and a vector Rp with 0 such that f (x ) + T h(x ) + T g (x ) = 0; T g (x ) = 0. 8

x = 0

feasible region

g3 (x) = 0

g1 (x) = 0 g2 (x) = 0

Since 0 and g (x ) 0, the second constraint above is equivalent to the statement that j = 0, for each 1 j p, only if gj is active. To nd a solution, we enumerate various combinations of active constraints, that is, constraints where equalities are attained at x , and check the signs of the resulting Lagrange multipliers. There are a number of distinct theories concerning this problem, based on various regularity conditions or constraint qualications, which are directed toward obtaining denitive general statements of necessary and sucient conditions. One can by no means pretend that all such results can be obtained as minor extensions of the theory for problems having equality constraints only. To date, however, their use has been limited to small-scale programming problems of two or three variables. We refer you to [1] for more on the subject.

References

[1] D. G. Luenberger. Linear and Nonlinear Programming. Addison-Wesley, 2nd edition, 1984. [2] T. Abell. Lecture notes. The Robotics Institute, School of Computer Science, Carnegie Mellon University, 1992.

- Jacobi AnCargado porDương Hà HT
- Solution Set Math IICargado porChristian Lazatin Sabadisto
- MapleProject07_m032Cargado porQui Danh Le
- Economic OperationsCargado poruplbseles
- SVM3Cargado porAmer Mohib
- Dynamic Simulation of Polymerization ProcessCargado porAseem Kashyap
- hsc-commerce-2014-october-maths1.pdfCargado porShradha Rohan Bayas
- 2004b-Lin and Miller- Tabu Search Algorithm for Chemical Process, 2287-2306Cargado pormaxpower2710
- optimizationCargado porKunal Anurag
- Quiz 2 Arithmetic Sequence and MeansCargado porMarjorie Rodriguez
- Year 3 Choices 13 14(1)Cargado porjake8098
- Qe Trick@Thats My WayCargado porPrakhar Singh
- SVM tutorialCargado porriyadh_pro
- alg1_ch04_extrapracticeCargado porwenatchee25
- Distributed Joint Resource Allocation in Primary and Cognitive Wireless NetworksCargado porMotasim Mustafa
- RetestCargado pormert
- 34Cargado porTünde Kiss
- Dynamic ErrorCargado porSuvra Pattanayak
- Math 151-16 Exam 3 Practice SolutionsCargado porUMBCCalculus
- InequalitiesCargado porChengLonkhin
- AhpCargado porHerlina
- Exercises Handout 1Cargado porChristelle Africa
- MTH 1112 Sem III, 2004-05Cargado porJEJUNG
- 1.8 Piecewise Functions Note & HwCargado porkcykim4
- Tutorial 3Cargado portheko
- dopeCargado porArpan Chopra
- 考古題 IIICargado porShellyChen
- New File 1Cargado porcox_j_sam
- Ch7ReviewCargado porTheGreatPostulio
- HBSE 12th Mathematics a 2018 PaperCargado pornayeem

- Sauer Danfoss HydraulicsCargado poralecandro_90
- TruServices Ersatzteilkatalog 2014Cargado poralecandro_90
- TIMPKEN BearingsCargado poralecandro_90
- BN_84512Cargado poralecandro_90
- Knife FlierCargado poralecandro_90
- Snd 6084r Spec SheetCargado poralecandro_90
- Brochure TRASH reduction John DeereCargado poralecandro_90
- Astm a240 StainlessCargado porTiago Mozart
- Astm a240 StainlessCargado porTiago Mozart
- Gear Pumps Technical InformationCargado poralecandro_90
- Harvestec_6000_Parts_CatalogCargado poralecandro_90
- snp_3750_3350_spec_sheetCargado poralecandro_90
- Environmentally Friendly Zirconium Oxide PretreatmentCargado poralecandro_90
- Environmentally Friendly Zirconium Oxide PretreatmentCargado poralecandro_90
- Environmentally Friendly Zirconium Oxide PretreatmentCargado poralecandro_90
- 1929065493Cargado poralecandro_90
- Lagrange MultiplierCargado poralecandro_90
- Highlighted SUMMARY - Competing for the Future by Gary Hamel & CK Prahalad 1994Cargado porsaranyabio
- US7886511 (Draper Head With Flexible Cutterbar Having Rigid Center Section)Cargado poralecandro_90
- FO Distribution OFNPCargado poralecandro_90
- Ficha 12-La Leche Producida en SistemasCargado poralecandro_90
- EP2306801B1 (Draper Head With Flexible Cutterbar Having Rigid Center Section)Cargado poralecandro_90
- Jag_vs_NHCargado poralecandro_90
- LankotaCatalog2013 WebCargado poralecandro_90
- Ficha Tecnica 25Cargado poralecandro_90
- 56559195 Curso de Fisica Teorica Landau y Lifshitz Vol 1 MecanicaCargado porderechoéizquierdou
- Us 20120047866Cargado poralecandro_90
- Metric Bolt and Cap Screw Torque ValuesCargado poralecandro_90
- CAP 01 Propiedades FluidosCargado poralecandro_90

- quenchants.pdfCargado porwienslaw5804
- Communicative competence and communicative language teachingCargado porX-fans Glazov
- Practical HynosisCargado porDr. Newland Gudu Komla
- Capabilities Marine RisersCargado pormesserklinge
- Research PaperCargado porYusuf Sarıgöz
- (Theoretical Perspectives in Architecture) Massimo Cacciari, Stephen Sartarelli (Transl.), Patrizia Lombardo (Introduction)-Architecture and Nihilism_ on the Philosophy of Modern Architecture-Yale UniCargado porJadestin
- AFV.29.spread_02Cargado porPatrick Lebecq
- Modeling Rainfall Prediction Using Fuzzy LogicCargado porFrancisco Suárez
- C7 - Array I (Students)Cargado porSyakira Wahida
- John Milbank, Only Theology Saves Metaphysics: On the Modalities of TerrorCargado poram050804
- FoxPro DatabaseCargado porGanapatibo Ganapati
- Ch7_G4010-R1Cargado porMosh
- Best Practice BW Netweaver ConnectionCargado porgfavaro
- What is LinguisticsCargado porAshar Ahmad
- B. Adam et alCargado porboanerges wino patty
- Learning to LEEDCargado porMohit Mulchandani
- PhiloCargado por'mCharity Tonii
- LM hiwinCargado porRaja Shankar Jeeva
- reverse motorized pendulumCargado porapi-285757994
- practice1-1Cargado porgwzgl
- EVSCargado porSock Rex
- Tuning and Optimizing Red Hat Enterprise Linux for Oracle 9i and 10g DatabasesCargado porPranabKanojia
- Post Structuralism in IRCargado porlwebster23
- NDS Drainage Catalog Apr 2010Cargado porJason Brown
- Fundamentals of Protein ChemCargado pormailtorubal
- LEAN CONSTRUCTION.docxCargado porThaliaTicllaRios
- RHCE Bootcamp BookCargado porNicușor Vatră
- IRB460Cargado porLucas Vinícius Costa
- SR 002 Panel Test ConnectionCargado porTuroy
- 86189660 Protection Reversal MagickCargado poramber43228