The Created Response Surface Technique for Optimizing Nonlinear Restrained Systems
Author(s): Charles W. Carroll and Anthony V. Fiacco
Source: Operations Research, Vol. 9, No. 2 (Mar. - Apr., 1961), pp. 169-185
Published by: INFORMS
Stable URL: .
Accessed: 08/05/2014 16:42
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
/>
.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact

.

INFORMS is collaborating with JSTOR to digitize, preserve and extend access to Operations Research.



This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


THE CREATED RESPONSE SURFACE TECHNIQUE
FOR OPTIMIZING NONLINEAR, RESTRAINED
SYSTEMS
Charles W. Carrollt
The Institute of Paper Chemistry, Appleton, Wisconsin
(Received April 25, 1960)
A different approach to the general problem of optimization of nonlinear,
restrained systems is presented.

This approach, called the Created Response
Surface Technique, is based, essentially, on steepest ascents up a succession
of created response surfaces within the solution space. The most important characteristic of the approach is that it automatically avoids restraint
boundary violations during the optimization.
This article discusses application to fully developed steady-state mathematical models. By the use
of appropriate experimental designs, the method could be used for the
characterization and optimization of mathematically incomplete, nonlinear,
restrained systems.
In addition, it seems possible that the Created Response Surface concept could be applied to attain dynamic process control
in certain nonlinear, restrained control situations.
However, the mathematical implications of the new approach must be studied further.

FOR PURPOSES of optimization, complex industrial situations often
can be described quantitatively by mathematical models. Such
mathematical models frequently consist of a continuous effectiveness function (measuring how effectively the operating objectives for a simulated
system are met), along with a number of continuous, realistic restraints.
The effectiveness function and restraints must be expressed, essentially,
in terms of the model's independent variables. These variables often correspond to certain controllable operating variables in the counterpart industrial system. Optimization consists of determining the values of the
independent variables that maximize the system's effectiveness, subject to
nonviolation of the restraints applying.
Such optimizations can be accomplished in a number of ways, depending
on the particular nature of the effectiveness function, the presence or absence of restraints, the nature of the restraints when present, the dimensions and complexity of the problem, the accuracy desired in the final solution, etc. (In addition, when computers are used, computer speed and
storage capacity are considerations.)
t This paper was prepared while the author was a graduate student, doing research at The Institute of Paper Chemistry; he is now with the International Business
Machines Corporation, Green Bay, Wisconsin.
169

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions



Cr

170

arles W. Carroll

Thus, when the effectiveness function and any restraints involved are
linear, standard methods of linear programming can be used to seek an
optimum. On the other hand, a general case for optimization occurs when
the effectiveness function and at least some of the restraints are relatively
complicated, nonlinear expressions. Such nonlinear situations have considerable industrial importance.
SOME APPROACHES TO THE OPTIMIZATION OF
NONLINEAR SYSTEMS

effectiveness function is nonlinear and subject to no restraints
(this probably seldom happens in entirely realistic situations), the approximate gradient technique of the method of steepest ascents can be used to
seek an optimum. [1-41 (If dimensionality is not too great, ordinary calculus

WHEGNTHE

20

X2

Fig. 1. Contour diagram of ES=E(x,, X2) showing paths of ascent and
restraints.

can be used.) The method of steepest ascents allows for the approximate
calculation of the steepest path up to the top of the effectiveness surface,

so that the maximum is arrived at very efficiently. The thin, solid line in
Fig. 1 indicates, for a three-dimensional case tE=E(X1, X2)], this path

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


Created Response

Surface

Technique

171

(II 4, 5, 2, 1) that, ideally, is always perpendicular to the contour lines (lines
of constant E). Point I is an arbitrarily chosen starting point.
When equality and inequality restraints apply, Lagrange multipliers

may be used in smallerproblems.[5

6, 7,

9,16]

For largerproblems,the

Lagrange multiplier technique may be quite unwieldy. One approach to
such problems is to apply the Method of Steepest Ascents until a limiting
value of a restraint is reached. From this point on, an efficient procedure

depends upon the nature of the model-in particular, the nature of the
restraints. Thus, when a single restraint is involved, like R' (which represents a maximum value for the independent variable XI) in Fig. 1, the
optimization may be completed by moving in the direction of steepest
ascent with this variable held constant at its limiting value. The over-all
path of the optimization then becomes I, 4, 5, 2, 3. (The arrows from the
dashed restraint boundaries in Fig. 1 point into the region of allowable
solution.)
However, suppose a single restraint, R" (a more complicated functional
restraint), is involved (again, see Fig. 1). Following the path of steepest
ascent soon results (at point 4) in a violation of this boundary. Using the
approximate gradient method alone, one might proceed farther up the path
of steepest ascent in hopes that the restraint would become noncritical at
the optimum (as actually happens in the illustration). But, in a more
complex problem, since neither the nature of the effectiveness surface nor
that of the restraints may be known, this is risky.
As another approach to the above situation, it is possible by means of
a series of linear approximations to the nonlinear effectiveness and restraint
surfaces, to apply linear programming sequentially.?101 Still other techniques, such as the Multiplex Method[T] and various other methods based
on gradient concepts (e.g., see references 16 and 17) and the Monte Carlo
technique have been suggested or used in certain complex, nonlinear situations.
THE CREATED RESPONSE SURFACE TECHNIQUE: A NEW APPROACH TO
THE OPTIMIZATION OF NONLINEAR, RESTRAINED SYSTEMS

A DIFFERENT approachto the generalproblemof optimizationof nonlinear,
restrained systems was developed in connection with a complex economic

optimizationof a paperindustryprocess.[",2 13 This new approach,called
the Created Response Surface Technique (CRST), converts a restrained
optimization problem into a series of nonlinear, unrestrained optimization
problems. With both restraints, R' and R", applying in Fig. 1, the CRST

takes a modified path of ascent (heavy, solid line) that, while maximizing
E as quickly as possible, at the same time automatically steers clear of the

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


172

Charles W. Carroll

restraints and thus prevents their violation. Near the very optimum,
however, solutions are allowed on or very close to critical restraint surfaces.
The Approach
The immediate object of any optimization is to find in some way the
values of the independent variables that maximize the effectiveness of the
system subject to nonviolation of restraints at the optimum. As previously
indicated, the violation of restraints during an optimization may be only
temporary and thus not crucial at the optimum. In general, though, it is
risky to rely on this happening because the nature of the model is often
impossible or impractical to assess beforehand, and because some restraints
are usually operative so that optimum solutions exist on these critical restraint surfaces.
If, however, the restraints are never allowed to be violated during an
optimization, then the resulting optimum solution is sure to be a feasible
one (that is, one where no restraints are violated).t Employing this concept, one can consider two competingrequirementsthat are to be simultaneously, though somewhat compromisingly, met during optimization: (1)
maximization of the effectiveness function as quickly as possible while, at
all times, (2) positively avoiding violation of the restraints.
The second requirement can be satisfied by devising a penalty, becoming increasingly severe, as restraint boundaries are approached. This
penalty, or cost for being too close to the restraints and thereby threatening
their violation, can be subtracted from the value of the effectiveness function to give a new 'control' function in exactly the same way as dollar costs

are subtracted from gross dollar profits to give 'net profit' as a sound measure of economic effectiveness.
The resulting new function, which is really a createdresponse function,
will be called A. Let E represent the effectiveness function in a form suitable for maximization. Then, if all restraints (say there are m of them)
are expressed in the form: Ri>O, an A-function compatible with the requirements of the new approach can be written:
A =EmrEi-Z

Wd/Ri,

where the sum represents the 'penalty.' Assume a unique, finite optimum
can be attained without complications. Then, as any restraint, Ri, approaches its limiting value (zero), A approaches negative infinity. In this
way, a progressively severe penalty is imposed as the boundary of a restraint
t This assumes that an initial, feasible starting point is known, as, for example,
would be the case if the improvement of effectiveness in an existing operation were
sought. See the discussion by FIACCO following this paper for a suggested approach
to determining an initial, feasible starting condition when such is not available.

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


Created Response

Surface

Technique

173

is approached. [The Wi's (in this paper, the Wi's are always >0) weight
the individual penalties amongst themselves while the r (always ?0)

weights the sum of these penalties in relation to E.] Since the optimum
solution lies on any restraint boundaries that are critical (critical R's= 0),
and conceivably very close to other near-critical restraint boundaries, it
must be possible to eventually set r = 0 so that optimal solutions can reside,
penalty-free, on or close to such limiting boundaries. At the same time,
since A would then equal E, the true effectiveness function would be
maximized.
Such an approach to restrained optimizations can be thought of as resulting, for visual purposes, in the creation of one dome-shaped A-surface,
itself having a maximum, for each nonzero value of r. The modified path
of ascent in Fig. 1 is really the continuous projection onto the E-surface
of the maxima of the infinite set of A-surfaces created as r is continually
relaxed from some positive, finite, initial value to zero. The feasible portions of the A-surfaces all lie beneath the E-surface for r>O. The larger
the value of r, the greater the penalty in relation to E, and thus the farther
below the E-surface the corresponding A-surface lies. All the A-surface
portions of interest exist entirely within the solution space defined by the
limits of the restraints.
Suppose from some feasible starting point the Method of Steepest
Ascents is used to determine the maximum on the lowest A-surface. The
values of the independent variables corresponding to this maximum can be
used to determine the starting point of a second maximization on the next
higher A-surface (the one characterized by the next smaller value of r) and
this process can be repeated until r= 0(A = E). Then, presumably, a final
short ascent can be made on the E-surface until critical restraints are met
in the neighborhood of the true optimum. In this way not only are the restraint boundaries automatically avoided during the optimization, but successive maxima tunnel up through the solution space, in a sense being continually guided in the direction of the desired optimum.*
Essentially then, this new approach to the optimization of restrained
systems is based on steepest ascents up a succession of createdresponse surfaces, within the natural solution space. Consequently, it has been entitled the CreatedResponse Surface Technique (CRST).
ILLUSTRATIVE

EXAMPLES


Two simple applications of the CRST will illustrate more concretely the
nature of the approach and the type of computations required.
* Movement through the solution space by quite a different procedure is also a
feature of Frisch's Multiplex Method. For a description of this method, see reference
11.

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


Charles W. Carroll

174

Example I
Maximize AE= E (x) =x for Ox! sai .
Putting the restraints in the required form (Ri>!:O), the problem can be rewritten:
Maximize E=E( x) = x
subject to

R2= (1-X)

R1=x>O,

_O.

+1.0

+0.8


/

PROBLEM: MAXIMIZE

I E=E(X)=X
SUBJECT TO

\
E= E(X)

I OR
s
OXs
R,=XtO AND
R2= 1-X0

/

X

I

40.6

40.4

4+0.2-

/t-O.01 /


__-ss

/

-0.2
-04

g-O0iI

DOME-SHAPED
SURFACES ARE

|

|

|

|R-

\

2)
ta(T
WHERE W. =0.3
WAND W2=0.7

r0.3
_o-0.6014


0

0.2

0.4

016

D[i

l

0.6

0.8

1.0

X OR STEP NO.

Fig. 2. Effectiveness surface and created response surfaces for Example I.

Figure 2 illustrates the E-surface, restraints, and several A-surfaces corresponding to the indicated values of r. The weighting factors, W1 and
W2, have been somewhat arbitrarily taken to be 0.3 and 0.7, respectively,
making the penalty contribution of each restraint approximately equal at
the start. These weighting factors can remain fixed during the optimization. Their rational sequential adjustment in a general problem could

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions



Created Response Surface Technique

175

represent a useful refinement to the method as presented in this paper, by
allowing for more efficient convergence to the optimum.
Let the initial feasible solution be x=0.275. The desirability of this
starting condition, considering its effectiveness and proximity to restraint
boundaries, is measured by the corresponding value on the lowest A-surface. This is represented by the lowest dot on the A-surface characterized
by r=0.5 in Fig. 2. Stepwise progress in the direction of steepest ascents
DATA

I

ASCEND UP LOWEST A-SURFACE
UNTIL A DECREASES*

REDUCE

._

-

ASCEND UP NEXT A-SURFACE
UNTIL A DECREASES*

NO

YES


(A = E)

ASCEND UP E -SURFACE
UNTIL NO FURTHER
PROGRESS

\B{E

|REDUCE
STEP SIZE
*

IS POSSIBLE

*

TAKEN

YE

IN TAKING STEPS IN ANY ASCENT RESTRAINTS MUST
NOT BE VIOLATED

Fig. 3. Simplified logic diagram compatible with the CRST.

(here, simply, the positive x-direction) gives rise to dots, each higher than
their predecessors, until finally a value of x (= 0.585) is reached causing A
to begin to fall off (see the cross mark). Preserving the value of x corresponding to the highest point (i.e., for the case immediately preceding the
one where A was found to decrease), r is reduced arbitrarily to 0.3 and the

corresponding point on the next higher A-surface becomes the starting
point for a similar ascent on this surface. Progress is made up this surface
to the approximate maximum, where another jump to the next higher
A-surface occurs, etc., until the jump is onto the E-surface (r =0, A =E).
(The number of A-surfaces involved here is arbitrary.) The guidance provided by the progress up through the solution space is relied upon to bring

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


Charles W. Carroll

176

the starting point on the E-surface very close to the true optimum (in the
present case, optimum E = 1.0).
In a general problem having more than two dimensions, final progress
is made in the direction of steepestascents on the effectivenesssurface and not
necessarily directly towards the optimum. However, if the distance
traveled is small, an optimum or near optimum will result, and critical restraints will be zero or very small. In order to press closer into the optimum
'corner,' reduction of step size at this final stage is very desirable.
Figure 3 is a simplified logic diagram showing how an automatic digital
computer program compatible with the Created Response Surface concept
can allow for ultimate optimization.
Example 11
Maximize E= [25- (x- 5)2-

subject to*

RI= (0.8)x-y>0,


(y-5)2]1/2

R2==8-(0.8)x-y>0.

Here the effectiveness function is the equation of a sphere having a radius
of 5 and having its center at x= 5, y= 5, E= 0. The restraints are planes,
running parallel to the E-axis and intersecting the sphere and each other.
Figure 4 is a contour diagram of the pertinent, strictly concave portion of
the effectiveness function; it also shows the restraints and the path ultimately followed in applying the CRST (heavy solid line through data
points). The arrows from the restraint surfaces point in the direction of
allowable solution. The solution space is seen to be a triangular wedge in
the lower portion of the figure.
The actual optimum occurs at the intersection of the two restraint surfaces on the effectiveness surface.t This maximal E can be easily determined (at x = 5, y = 4) to be 4.899. To illustrate further the nature of the
new approach, the CRST will again be used to seek an appropriate optimum.
In addition, this example will show the kind of computations that are required in typical, realistic applications.
The created response surface equations are given generally by
A =E - r Zi WiRi.

1

For this particular problem, let W, = 3.6 and W2= 0.4. These values make
the penalties associated with each restraint equal at the start, where (as
* The restraint functions as written measure the distance of a point in the solution
space from the restraint boundaries in the y-direction. They could have been expressed in terms of the x-direction. Also the functional forms of the restraint funcSuch choices relating to the manner of expressing
tions are somewhat arbitrary.
restraints could conceivably have important effects on the sensitivity of restraint,
and therefore penalty, response during optimization.
t Existence of maxima in constrained optimizations
TUCKER


in reference 8, and by

ARROW

AND HURWICZ

such as this is discussed by

in references 9 and 16.

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


Created Response

177

Technique

Surface

will be seen later) the initial values of x and y are taken to be 7 and 2 respectively. Thus,
A = [25- (x-5)2-

36

(y5)2]1/2r


0.4

+

}

(2)

The path of steepest ascents up a response surface is the one resulting
from progress always in the direction of the gradient of the function describ,0.0-

PROBLEM:

7.0-/i

\

"\

/

/\iX

<\0

E=
E

\


MAXIMIZE

25

(X-5)2-

SUBJECT

6.0

t '

X

X

\

\

\

(5)2

TO

R,=(0.8)X-yX

0


AND
0

R2=8 - (O8)X-Y1
STARTING AT

\

X=7.000

AND

y=2.000

.(ACTUAL MAXIMUM = 4.899
C RS T MAXIMUM = 4.886
&78ERROR = 0.27)

3C0

o~~~~~~
0

1.0

2.0

3.0

4.0


5.0

6.0

7.0

8.0

9.0

10.0

Fig. 4. Contour diagram, restraints, and ascent paths for Example II.

ing that surface (see, e.g., reference 14).
a function, its gradient is:

If +(x, y, z,

*.*)

represents such

VO=(801ax)T+ (aolay)! + (do/dz) ;**.
Here I, j, k, etc. are unit vectors in the x, y, z, * directions.
To move always in the direction of the gradient, it is necessary continually and simultaneously to adjust the independent variables by amounts
proportional (by the same proportionality constant) to their respective
components as given in the gradient expression. Thus, to move in the direction of steepest ascents in the ?-surface (that is, to increase 0 at the
maximum rate), x, y, z, etc. would each be changed simultaneously always

by an amount proportional to aq/Ox, a?/Oy, ao/Oz, etc., respectively.

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


Charles W. Carroll

178

In actual computations, the gradient is not continually calculated. At
a given starting point, the components of a base gradient are calculated or
approximated. Then, after choosing a proportionality constant, h, the
independent variables are adjusted by adding h times the appropriate
gradient component (or its approximation) to the base value of the corresponding variable. Thus, if the base value of x is Xb, the new value of x
the new value of each variable is calbecomes xb+h fl/xlxb, Yb' Zb, ""If
culated and a new p computed using these new values, p will be increasing
at approximately its greatest spatial rate.
The direction of steepest ascent so calculated is generally exact only at
the base point (and there only when the components of the gradient are
not approximated). Thus, continued steps in this direction may deviate
considerably from those corresponding to the true path of steepest ascent,
especially when the response surface is complicated. The choice of step
size and of how far to progress on a given ascent are somewhat arbitrary.
A good general rule on the extent of an ascent seems to be to progress in the
direction of a given base gradient until the value of the response starts to
*
Here one can go back to the best case (that is, the case for the
fall off.t?01
immediately preceding conditions) and calculate a new (and differently

directed) gradient using these conditions as a new base. A given response
surface can be ascended in this 'zig-zag' manner until, following the calculation of a new base case and gradient, not even one, small, first step can be
taken. At this point it may be assumed that the maximum of the response
surface is nearby. (This may not always be the case. The problems of
level surfaces, alternate optima, etc. are discussed in references 1 and 10.)
It is at this stage in the CRST that the value of r is reduced and a new
A-surface is ascended. When, after a number of ascents, r finally becomes
zero, a final, short ascent on the actual effectiveness surface should lead
to a solution very close to the true optimum.
In the present example, the components of the gradient can be easily
calculated by partial differentiation of the expression for A: equation (2). t
Thus,
AA/8x= (5-x)/E+r
[2.88/(AI )2_ 0.32/(R2)2]
and

3A/Oy= (5-y)/E-r

[3.6/(RI1)2+0.4/(R2)2].

Starting this illustrative optimization at, say, x=7, y=2 as an initial,
feasible solution,
* However, other criteria are possible.

See, e.g., reference 15.

t These components may be approximated too. Thus, the component, aAA/ax,
can be approximated by [(AA) /Ax],, where (AA)x is the change in A produced by a
small, arbitrary change in x (Ax), y being held constant at its base value. See references 10 and 12 for applications of this approximation method.


This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


Created Response

Surface

dA/Axjx=7,-2=-2.355,

179

Technique

y=2 =-1.912.*
A/dayjx=7,

For r= 1.0 and h= 0.1, a first ascent can be taken (see Table I) starting at
the initial conditions mentioned above. The new values of x and y for the
first step of this first ascent are
=7.000-0.236=6.764,

x=7.000+0.1

(-2.355)

y=2.000+0.1

(-1.912)=2.000-0.191=1.809.


From these values, RI, R2,E, the penalty, and A can be calculated and are
TABLE I
CALCULATIONSFOR EXAMPLE II(a)
ascent, r =

Ist

Initials
Cumulative step
no. (see Fig. 5)

I

.

y.

OA/Ax.

Base

2*

-

7.000

6.764

2.000


I.809

6.528
i.6i8

6.292
I*427

1.

6.528
i.6i8

1*

4th ascent,

i.o

r

_

=

1.0

Base*


5
-

6.o56
2.053

--

-

2.0

6.o56

5.890

5.724

5.890

5.832

2.053

2.oi8

*.983

2.OI8


2.054

1.0

5.584
2.488

---0

-o. i66

-0.472

+

.-I.9I2

Ri.
R ..............

Base

4
-

-2-355

AA/dy....

?


z*

3

2
0.1

.........-

x ...

I

3rd ascent, r =

2nd ascent, r = 1.0

I.o

->

+0*435

-

-0.029

-0.035


+0. OI8-

3.600

3.602

I.979

2.792

2.694

2.596

2.694

2.6I2

I*539

3.604
i.i6o

2.792

o0.780

3.604
i.i6o


3.607

0. 400

I.I02

I.045

I.I02

I.270

I*438

I.270

I.280

E .3.464

3.42I

3.35I

3.250

3.35I

3.899


4.284

3.899

3.9I4

3.92I

3.914

3.953

r Xi WilRi ......

2.000

I*5I2

I*344

I.258

I*344

I.652

2.202

I.652


i.65I

I.665

i.65I

i.69i

A.........

I*464

I.909

2.007

I.992

2.007

2.247

2.o82

2.247

2.263

2.256


2.263

2.262

(a)

See text for significance of starred and encircled step numbers.

shown in the table. Other steps for this first ascent are calculated in a
similar manner.
The value of A falls off from the previous value at step 3 (encircled) and
thus step 2 (marked by an asterisk) becomes the base for the calculation of
another ascent where h is now taken to be 1.0. In a similar manner, a
third ascent is taken. Finally, in attempting a further ascent, not even a
first step can be taken with h = 2.0 without A decreasing. It is assumed,
therefore, that the maximum of the lowest A-surface is close by, and thus
* Using Ax=Ay=O.O1,the approximate components of the gradient are
[( A).IAx ]y=- 2.398,

[ (A A)y/Ay .= - 1.979.

The exact calculations of the components of the gradients will be used here and
throughout the rest of this example.

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


Charles W. Carroll


180

TABLE II
CALCULATIONS FOR EXAMPLE II(a)
.

sascent,

ISt

Base

I

r

r

?

3

2*

2nd ascent,
=

|

O.OI


=

r = O.OI

r = O.OI

0

I*

Base

4th ascent,

3rd ascent,

0.01

I*

Base

Base*

(D

Cumulative

step no.

(see Fig.
5) .---

7

6

(5)

8

I0.

.........

9

I. .

0.2

_

I.0

0.5

I.0

__


X ..........

5.890

5.665

5.440

5.2I5

5.395

5.440

5.I75

4.910

5.I75

5.185

5.I90

5.I85

5.I52

y ..........


2.OI8

2.772

3.526

4.280

3.677

3.526

3.5I4

3.502

3.5I4

3.700

3.793

3.700

3.6I7

-0.225
......


,3A/Ox.
......
OiA/Oy

-

-o0.265

-----

?0.0I0

RI .........

2.694

I.760

0.826 -o. io8

O.639

o.826

O.626 0.426

R .........2

I.270


o.696

O.I22

0.007

O.I22

0.346

0.570

E. .........
rL WilRi.

A..........

00.

OI 2

___

-0.033

_

-_

o. i86

O.626

---

+0.754

-*

__

_

O.o83

o0448

0.3 59

0.448

0.505

0.346

0.I52

0.055

0.252


0.26i

3-9I4

4.427

4.758

4.806

4.758

4-77I

4.769

4-77I

4.825

4.848

4.825

4.803

O.OI7

0.026


0.076

o.628

0.076

o.o69

0.092

o.o69

0.I07

0.I73

0.I07

O.o87

3.897

4.40I

4.682

4.178

4.682


4.702

4.677

4.702

4.728

4.675

4.728

4.7I6

(a) See text for significance

of starred and encircled step numbers.

the value of r is reduced, arbitrarily, to 0.01. The first portion of Table II
indicates the conditions for and the results of the subsequent ascents.
[Notice that the third step of the first ascent leads to a restraint violation
so that the step size (h) is reduced for step 4, which is feasible.] The attempted fourth ascent indicates the need for a further reduction of r. This
TABLE III
CALCULATIONS FOR EXAMPLE II(a)
ist ascent,
r = o.ooI
Base

?


1*

Base

I*

4th ascent,
r = O.OOI

3rd ascent,
r = O.OOI

2nd ascent,
r = O.OOI
(D

I*

Base

(19

Base*

(i)

Final ascent,
Y = O; A = E
Base


I

2

Cumulative step
(see

no.

Fig. 5)
k.

(9)

.

X.

y .........

--

. . ......
.

lA/Ox .-o
OA/Oy. .

R


.......
R2. .
E.

WiRi
A.

_

IO

II
0.2

0.5

__

I2
0.3

0.7

0.3

(12)
0.2

I.0


5.074,5.073
3.928 33.939

5.i66

5.I58

5.i66

5.073

5.033

5.073

5.077

5.080

5.077

5.048

5.077

3.700

3.8I7

3.864


3.8I7

3.85I

3.865

3.8sI

3.882

3.903

3.882

3.873

3.882

-

-o0.38

-0.I33

0.02I4

__--

+o.o48


+0.

_

I04

I4

0.05

0.2

55.I85

+0.234

I3

--

-

-0.029

-

-o.Oi6

-----


-0.009

-

+0.229

-

-0.448

0.3i6

0.262

o-3I6

0. 207

o. i6I

0.207

o.i8o

o.i6i

o.i8o

o.I65


0.050

0.010

0.050

0.092

0.209

0.092

0.056

0.0,33

0.056

0.089

o.i8o
o.o56

0.I3

0.152

4.825


4.855

4.867

4.855

4.866

4.869

4.866

4.873

4.878

4.873

4.87I

4.873

4.883 4.886

0.O0II

0.019
4.836

0.054

4.8I3

O.OI9

0.022

0.026

0.022

0.027

0.026

4.844

4.843

4.844

4.846

0.0o34
4.844

0.027

4.836

4.846


4.845

4.8I4

(a) See text for significance of starred and encircled step numbers.

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions

0

4.873

10.II9

0.013

0

0.003

0

4.883 4.886


Created Response

Surface


Technique

181

time, r is set equal to 0.001 (see Table III). After four ascents on this
A-surface characterized by r = 0.001, r is allowed to equal zero and A =E.
The final ascent (right-hand portion of Table III) arrives at the approximate optimum of 4.886. This is only 0.27 per cent below the true optimum
of 4.899. By letting r become very, very small, by calculating the gradient
very accurately if an approximation is used, and by employing extremely
small values of h, it appears that the difference between the optimum obtained by means of the CRST and the true optimum can be made as small
as desired in this example.
Figure 4 indicates the path connecting the calculated (approximate)
5.0-

\orooi
4.5

<

r

-

3.0 -

2.5 -<

/


1.0

1.5_

CUMULATIVESTEP NUMBER

Fig. 5. E, A vs. cumulative step number for Example 11.

maxima for the various ascents (thick solid line through data points). The
thin solid line is the approximate path that would result if the A-surface
maxima were projected onto the E-surface as r was continuously relaxed
to zero. Note in Fig. 4 (also see Table I and Fig. 5 to be discussed in the
next paragraph) that at first E actually decreases to allow for an initial
movement away from restraint R2. This decrease in effectiveness is more
than compensated for by decreasing the penalty for being too close to
restraining surfaces, so that the net effect is an increase in A. [On the other
hand, in the first ascent where r=0.01 (see Table II), A finally falls off in
attempted step 4 because R2 is approached too closely.]
Figure 5 plots A. and E against cumulative step number. Here, of
course, in contrast to 'step no.' in Fig. 2 of the first illustrative example
where only x is changed, a given step results from the simultaneous change
of both independent variables involved.

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


182

Charles W. Carroll

RESULTS OF AN INDUSTRIAL APPLICATION

As mentioned previously, the CRST was developed as a part of an
economic optimization study of a complex paper industry process. Significant improvement in the simulated effectiveness of operation of this
process was accomplished using the CRST in a nearly automatic fashion
on an IBM-650 digital computer. A brief description of this application
can be found in reference 13. A detailed discussion appears in reference
12.

1.0

0.4

-

0.2 _

4O -

~~~~~CMLTIVE
STPNME

-6.0

12.0

H

16.
CUMULATIVE


STEP

NUMBER

Fig. 6. E, A vs. cumulativestep numberfor an industrialexample.
As in Example II, each step of this multidimensional industrial problem
resulted from the simultaneous change of the independent variables by an
amount proportional to their respective components in the current gradient.
When A and E are plotted against cumulative step number (see Fig. 6),
the results appear strikingly similar to those of Fig. 5 for the second illustrative example.
CONCLUSION
IT SEEMS possible that in addition to its application to the optimization of
mathematically complete nonlinear, restrained systems, the Created
Response Surface Technique could be effectively applied in certain other
optimization problems. For example, the method might be useful in determining approximate optima and critical restraints in some linear programming situations. t21 The linearity of such systems leads to significant com-

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


Created Response

Surface

Technique

183

putational simplifications. The technique may also be effective in the

experimental optimization of mathematically incomplete systems, including the analytical treatment of restrained evolutionary operation problems, and in certain process-control situations. Application of the technique to dynamic process control is being investigated at present.
However, the mathematical implications of the Created Response Surface Technique [existence and uniqueness of optimum solutions, the effect
of adjustment of the various parameters (h, Wt, r) on convergence, the
complexity of the A-surface in relation to the E and restraint surfaces, etc.]
must yet be fully explored. This task, along with additional experience
using the new method, is necessary in order to determine more assuredly
those situations where application of the Created Response Surface Technique could be expected to be more efficient than the use of previously
existing methods.
based on a portion of a thesis submitted in partial fulfillment of the requirements of The Institute of Paper Chemistry for the degree
of Doctor of Philosophy from Lawrence College, Appleton, Wis., June,
1959. This work was carried out under the direction of ANDREW C. BERRY,
whom the author wishes to thank for his continual interest, encouragement, and constructive criticism during the course of the CRST development.
THIS ARTICLE is

REFERENCES

1. 0. L. DAVIES(Editor), The Design and Analysis of IndustrialExperiments,
Chapter11, pp. 495-578, Hafner PublishingCompany,New York, 1956.
2. R. A. BRADLEY,"Mathematics and Statistics Fundamental to the Fitting of

ResponseSurfaces,"IndustrialQualityControl15, no. 1, 16-20 (1958).
3. J. S. HUNTER, "Determinationof OptimumOperatingConditionsby Experimental Methods:Part II- 1 2, 3, Models and Methods,"IndustrialQuality
Control15, no. 6, 16-24; 15, no. 7, 7-15; 15, no. 8, 6-14 (1958, 1959).
4. V. CHEW(Editor), ExperimentalDesigns in Industry, Part I, pp. 138-190,
Wiley, New York, 1956.
5. W. KAPLAN,Advanced Calculus, p. 128, Addison-Wesley Press, Cambridge,
Mass., 1952.
R. L. ACKOFF,ANDE. L. ARNOFF,Introduction to Opera6. C. W. CHURCHMAN,

tions Research,Wiley, New York, 1957.

7. B. KLEIN, "Direct Use of ExtremalPrinciplesin Solving CertainOptimizing
ProblemsInvolving Inequalities,"Opns.Res. 3, 168-175 (1955).
8. A. W. TUCKER,"Linear and Nonlinear Programming," Opns. Res. 5, 244-257

(1957).
9. K. ARROWANDL. HURWICZ,"Gradient Methods for Constrained Maxima,"

10. R.

Opns.Res. 5, 258-265 (1957).
W. SCHRAGE, "Optimizinga Catalytic CrackingOperationby the Method
of SteepestAscentsand LinearProgramming,"Opns.Res.6, 498-515 (1958).

11. C. J. VAN EIJK AND J. SANDEE, "Quantitative Determination of an Optimum

27, 1-13 (1959).
EconomicPolicy," Econometrica

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


184

Charles W. Carroll

12. C. W. CARROLL, "An Operations Research Approach to the Economic Optimization of a Kraft Pulping Process," Doctor's Dissertation, The Institute of
Paper Chemistry, Appleton, Wisconsin, 1959.
, "An Operations Research Approach to the Economic Optimization of
13.

a Kraft Pulping Process," Tappi 43, 305-313 (1960).
14. C. R. WYLIE,JR., Advanced Engineering Mathematics, p. 448, McGraw-Hill,
1951.
15. J. B. CROCKETT AND H. CHERNOFF, "Gradient Methods of Maximization,
Pacific J. Math 5, 33-50 (1955).
16. K. J. ARROW, L. HURWICZ, AND H. UZAWA, Studies in Linear and Nonlinear
Programming, Stanford University Press, Stanford, California, 1958.
17. A. V. FIACCO, N. M. SMITH, AND D. BLACKWELL, "A More General Method
for Nonlinear Programming" (a paper presented at the Seventeenth National Meeting of the OPERATIONS RESEARCH SOCIETY OF AMERICA, May 20,
1960, in New York, N. Y.)

COMMENTS ON THE PRECEDING PAPER
Anthony

V. Fiacco*

OperationsResearchOffice,The Johns Hopkins University, Bethesda, Maryland
SOLUTION to the problem of acquiring an acceptable starting point is given
in "A General Method for Nonlinear Programming," by ANTHONYV. FIACCO,
The method is described in terms
NICHOLASM. SMITH,AND DAVID BLACKWELL.t
of the particular gradient techniques employed, but their principle of entry into a
constrained space can be simply enunciated: the entry problem can be stated as a
programming problem; hence, a methodology that solves the programming problem
solves the entry problem. This principle is applied to Carroll's technique as
follows:
Assume we have the starting point x =xo, and that S {s; Rs(xo) ?0,
T= {t; Rt(xo) >0 1. It is trivial to satisfy Rt(xo) >0 for at least one value of t, so
we shall assume that T is not empty. We require x, such that Rj(x1) >0 for
j= 1, 2, *, k, where k is the number of restraints. Assume further that S is not

empty; otherwise, x0 =x1 and we already have an acceptable starting point.
Using the notation of the paper, we can maximize Es,1=IR,(x) for some s
s1GS by maximizing A,, =E8l -r ZtET TVW/Rt(x), using the technique described
in the paper.
The procedure is begun at x=x0 where, by hypothesis, R(x) >0. The computations continue until E,, =R, (x0) >0. Incidentally, following each iteration it
A

I

* EDITOR'S NOTE: Mr. Fiacco acted as referee for the preceding paper by Carroll, and submitted these comments as part of his report. Mr. Carroll suggested
that they be published in conjunction with his paper and Mr. Fiacco has agreed to
this publication.
t Bulletin of the OPERATIONS RESEARCH SOCIETY OF AMERICA, Volume 8, Supplement 1, page B-36 (1960). This is the paper referred to in reference 17 of the preceding paper.

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions


Comments

on the Preceding

Paper

185

is prudent to evaluate R,(x) for all s CS, since R8(x) >0 may obtain fortuitously
for some s after a given move.
In any event, once R,1(xol) >0, we have si C T when x =xoi. If S is not yet
empty, we have some S =82 ES and can now maximize E,2 by maximizing A,2=

E,2 -r ZtET Wt/Rt(x), given the acceptable point xOl,where we now have S
{s; R(xol) <0}, and T= It; Rt(xol) >O}.
The procedure is continued until the set S is empty for some x =x1, whereupon
t= 1, 2, .., k =j, and hence, R1(x1)>0 for all j. The desired starting point for
the original problem has been computed and the optimization routine can begin.

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 16:43:00 PM
All use subject to JSTOR Terms and Conditions