Re: A Two-Level SOLVER ??

$$ Dana;
1) >"Hi. I'd be very interested if you could send me your workbook."
I would be glad to do so. The difficulty, however, is multi-folds!! The
w/b is ~ 13. MB and trying to delete irrelevant stuff would likely disrupt
the data flow and renders the w/b unworkable!!
2) I may be able to send a truncated copy directly to you, if you don't mind!


$$ Mike;
3) >"Please educate me. When is a 6th degree polynomial appropriate?"
Polynomials are the most easy to use and the most difficult to predict!
In some cases, the deg of poly is a by-product of the analytical method,
while in other cases it is pre-selected depending on the application.
4) In my current application, I selected/ended up with a 6th degree
polynomials, and here's how.
I had numerous 5-parameter analytical data sets, and I needed an accurate
regression model to represent the data analytically for post processing.
5) I've successfully developed a Genetic Algorithm (GA) based on a reliable
Evolutionary Strategy (ES) technique and incorporating a non-linear
multi-variant regression model.
I've the option of specifying the variable's highest exponent in each
polynomial term. Based on my experience on handling similar regression
problems, I chose 6; the max integer exponent allowed in my current version
of the algorithm.
6) So if the regression equation is given by:
....E = fn(X, Y, Z, W, V)
and
....E = SUM(n=1, N) An.(X)^an .(Y)^bn. (Z)^cn. (W)^dn. (V)^en
then one can determine the exponents and the set of regression coefficients
such that the best-fit to the data is realized, i.e.; the Squared Error Sum
(SES) is less than a specified accuracy, say, < 1.E-6, which is a reasonable
engineering accuracy for my current application.


$$ Steve;
7) >"Say you have a simple parabola, y = x2 that you want to minimize.
Setting the first derivative = 0, gives an inflection point."
As you know, a zero 2nd derivative alone DOES NOT necessarily identify an
inflection point! The 1st derivative must pass smoothly through a "true"
inflection point. Another situation, if the change-of-sign of the 2nd
derivative is accompanied by a change-of-sign of the 1st derivative, then the
point is NOT an inflection point!
8) >"So the derivative by itself provides you with no value."
I respectfully disagree! One can learn a lot by
examining/studying/observing the behaviour of derivatives of complex
functions!!
9) Back to my OP and my subsequent reformulation of the optimization problem
by setting d(E)/d(J) = 0 instead of E = max.
I've tested the reformulated optimization 100s of times with different sets
of data and with different starting values. The results, so far, are
consistent and correct for every and each case tested.
10) I'll access the links to NLP and Arizona State (and others you kindly
provided) to see if I can realistically find a reasonable replacement to
Excel/Solver without major changes to my program.
I'll seriously consider any non-Frontline VBA Solver algorithm.

Thank you, Dana, Steve and Mike for your thoughtful contributions.
Monir

"SteveM" wrote:

On Feb 3, 12:29 am, "Mike Middleton" <m...@xxxxxxxxxxxxxxxxx> wrote:
monir -

Yes, a combination of a 6th deg poly and Solver is generally a problem! <

Please educate me. When is a 6th degree polynomial appropriate?

I can understand using a 2nd degree polynomial to model a single-bulge
relationship, and occasionally there may be an S-shaped relationship that
needs a 3rd degree polynomial.

But what kind of real-world situation needs a model with a 6th degree
polynomial?

(In most data analysis applications, I would be very concerned with
over-fitting and the numerical precision difficulties associated with
raising values to the 6th power.)

I really am curious.

- Mike Middletonhttp://www.DecisionToolworks.com
Decision Analysis Add-ins for Excel

"monir" <mo...@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message

news:1FA96B65-C86A-403B-883F-4FB186369978@xxxxxxxxxxxxxxxx

Dana;

Thank you for taking the time.

1) Your thoughts are valid. Yes, a combination of a 6th deg poly and
Solver
is generally a problem! However, as I indicated in my OP, for any value
of
the changing variable P, say P1, E has a single extremum (max) within the
range of interest of J and P.
Thus, Solver should not really get confused by seeing multiple min and
max!
There's only one local max point.

2) It seems to me that Excel / Solver performs better when the target is
set
to a value rather than to a max or min! And that's precisely why I had to
reformulate the problem such that d(E)/d(J) is set to zero instead of
(originally) setting E to max. It works!

3) It's evident from your reply that you're familiar with the subject
matter, and possibly have tried other optimizers. If so, would you be
kind
enough to share some of your experience and thoughts on such other
optimizers
regarding: reliability, limitations, options, availability of VBA code,
etc.

Thank you.
Monir

"Dana DeLouis" wrote:

1) I've reformulated the optimization problem by deriving the
analytical
expression for the 1st derivative d(E)/d(J) as a function of J and P.

(fn1, fn2, fn4 are complicated, lengthy, 6th deg polys, ~ 1 page long
each

Just a thought. Without knowing what you have, solving 6th Deg Polyn.
can
be hard, and have multiple solutions.(Derivative =0's)
Solver was probably seeing multiple local Min & Max points on your curve.
As one curve was increasing, probably the other curves were decreasing,
etc... Once Solver gets confused, it easily gives up. Once Solver locks
onto an incorrect local minimum, Solver doesn't have enought logic to get
out of it, and give proper warnings.
Make sure you model was not using functions like IF(), as these cause
problems for Solver.

Mentioning 6th Deg Polys with Solver should send a red flag! It may be
possible to write a vba routine using the Newton Method. That can
sometimes
help.
You may have a valid solution, but my experience would suggest caution
with
the answer given.

--
Dana DeLouis

"monir" <mo...@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote in message
news:B9133248-488A-471C-A110-168B0B4CFEEB@xxxxxxxxxxxxxxxx
Steve;

Thank you kindly for your thoughtful reply.

It's very clear that Excel / Solver (also developed by Frontline!) is
the
problem!!! No surprise.

1) I've reformulated the optimization problem by deriving the
analytical
expression for the 1st derivative d(E)/d(J) as a function of J and P.
The
problem now reads:
Given the relations:
..T = fn1(J, P)
..Q = fn2(J, P)
..E = fn3(T, Q, J)
..B = fn4(J, Q)
find the values of:
..J (bet 0.05 and 1.40)
..P (bet 0.50 and 1.50)
such that:
..target: d(E)/d(J) = 0.0
..constraint: B = a known B0
(fn1, fn2, fn4 are complicated, lengthy, 6th deg polys, ~ 1 page long
each.
That's why I didn't include them in my OP.)

2) Now the Solver solution (J, P, E) is perfect!!

3) I've never used or even heard of NLP IPOPT (Fortran)!! but will look
at
it. Thanks for the link.

4) One of the difficulties would be to convert IPOPT to VBA. It is
hard
to
know without examining the code. My programming experience is about
60/40
Fortran/VBA. But again, the feasibility of the task would depend on
the
complexity of IPOPT.
Based on your expertise in the field, do you know of or used a reliable
VBA
optimizer macro ??

Thank you once again for your help.
Monir

"SteveM" wrote:

On Jan 29, 2:21 pm, monir <mo...@xxxxxxxxxxxxxxxxxxxxxxxxx> wrote:
Steve;

Thank you for your prompt reply.

a) Let me first clarify item 2) of my OP.
The 2nd sentence should read:
"For any value of the changing variable P, say P1, in the range
P=0.50
to
1.50, there is a max E1 at J1, which DOES NOT necessarily satisfy
the
constraint B18 = 1."

b) Please review my suggestion item 3) of my OP in light of the
above
clarification.

c) I'm running Excel Solver from a macro.

d) FYI. Frontline Premium Solver is poorly developed and
technically
unreliable! I've extensively tested it earlier on, and reported its
numerous
and serious errors to Frontline.

Any thoughts ?? Thank you

"SteveM" wrote:
On Jan 29, 12:00 pm, monir <mo...@xxxxxxxxxxxxxxxxxxxxxxxxx>
wrote:
Hello;

I would very much appreciate your help in the following
relatively
simple
non-linear optimization problem.

1) By applying Solver:
....Set target cell: E18 to max....(E18 represents dependent
variable E)
....By changing cells: J18 ..........(J18 represents independent
variable J)
............................: P18...........(P18 represents
independent
variable P)
....Subject to constraints: B18=1
....................................: J range 0.05 to 1.40
....................................: P range 0.50 to 1.50
it produces a feasible solution (J, P, E), which is not exactly
the
"correct" one (differs by about 10% of what it should be)

2) The difficulty is directly associated with the above
formulation
of the
problem.
For any value of the changing variable P, say P1, in the range
P=0.50 to
1.50, there is a max E1 at J1.
Solver in 1) above appears to seek the solution for max E based
on
the
changing combinations of J and P and settles for the combination
that
produces max E in comparison with other combinations and subject
to
the
constraints.
This is clearly not what I had in mind!!

3) The correct formulation of the problem should be, I think,
something like:
..Solver1: for each tried value of P, say, P1 in the range 0.50
to
1.50
..Set target cell: E18 to max.....(E18 represents dependent
variable E)
..By changing cells: J18 ...........(J18 represents independent
variable J)
..Subject to constraints: J range 0.05 to 1.40
.............(solution: J1, max E1 at each P1)
....Solver2:
....Set target cell: B18 = 1
....By changing cells: J18 ..........(J18 now represents the new
variable J1)
....Subject to constraints: J1 range 0.05 to 1.40 (same range of
J
is fine)
.............(solution: J2, P2, max E2)

4) How would you intelligently combine Solver1 and Solver2 as a
2-level
Solver ?? either by running Solver manually or by a macro. And,
is
it
possible to do so relying entirely on the Solver internal trial
solutions
without establishing the relation between P1s and E1s (which is
not
easy to
do) ??

Thank you kindly.

monir,

Well first of all, what are you trying to maximize? If your 1 is
formulated properly then yes, it is doing what it is supposed to
be
doing. So if not E then what?

It's better to describe your problem in equation (not Excel) form
because that is much clearer to another modeler. I.e., it would
be
good to know what the functional relationship of P's and J's are
to
E. BTW, if P and J are changing cells then they are decision
variables. It's not clear what they are either. Are they single
cells or a vector of cells. What are the non-linear constraint
functions that are functions of them? If you could post that it
would
be helpful.

Your 2 implies E is only a function of J unless there is some sort
of
plus/minus symmetry between J and P. But then you could make a P
a
function of J and replace all the P's in your formulation. But
assuming there is not then P is not in the basis for any value of
J.
Otherwise you'd a different Emax. So if P is not in the objective
function and always non basic. I don't see how including it can
contribute to the formulation.

You proposed parametric

...

read more >>

I gotta step in here. Regardless of what the function is (excepting
that it's smooth and continuous in the feasible region, the max/min
value of the objective function may be meaningless by itself. I.e.
setting first derivatives to 0. Because the feasible region is
defined by the bounded poly-tope comprised of both the objective
function and the constraint set.

Say you have a simple parabola, y = x2 that you want to minimize.
Setting the first derivative = 0, gives an inflection point. Simple
enough. But now add the constraint y >= 3 which is represented by a
line that passes right through the parabola. So now the feasible
region has been reduced taking the 0 point of the objective function
out of play. So the derivative by itself provides you with no value.

The convexity of higher order systems of non-linear inequalities is
generally impossible to establish by inspection. It can be determined
by examining if the Hessian of the Lagrangian equation meets KKT
.

Relevant Pages

  • Re: precision in solver
    ... Precision in Solver isn't so much about numerical calculation. ... you may have a constraint that xx <= 6. ... If is hard to quantify the precision of Solver's solution also in part to the options you see in Solver's options. ... Changing the Solver option for "Derivatives" and "Search" can also have an impact on the final solution. ...
    (microsoft.public.excel.misc)
  • Re: Solver Constraints
    ... Solver will usually start off by not having tight restrictions on the ... "Rick" wrote in message ... > There's a similar problem with the integer constraint. ... >> "Dana DeLouis" wrote: ...
    (microsoft.public.excel.misc)
  • Re: Secant method vs Newtons method
    ... I am presenting on my web site the NIH method, a natural generalization to higher order of the bracketed Secant Method for root finding, where the Secant Method occurs naturally as the zero order case. ... My main example is the computation of the roots of all the Legendre polynomials up to a certain given order which is a nice example which suits my purposes. ... I have chosen to call them Normalized Hermite Polynomails, but am open to suggstions and maybe Two-Point Normalized Hermite [Interpolation Polynomials] would be better. ... The two root finding methods which I am presenting use these same polynomials to interpolate the inverse of a given function (after transforming the derivatives according to the rules). ...
    (sci.math.num-analysis)
  • Re: Possible Calculus of Variations Problem
    ... subject to the constraint that a<g<b for all x? ... of variations has nothing to say about this problem. ... calculus of variations has nothing to do with ... Dini derivatives. ...
    (sci.math)
  • Re: A Two-Level SOLVER ??
    ... and possibly have tried other optimizers. ... Solver was probably seeing multiple local Min & Max points on your curve. ... ..constraint: ... function of J and replace all the P's in your formulation. ...
    (microsoft.public.excel.programming)
Loading