The Results of the T est Runs

4.3 The TL T2 Problem

4.3.2 The Results of the T est Runs

Eet of the Regularization

Herewerunthetestswithafullmappingparametervetororrespondingto

afull matrix

A

^,^ie. ^the^number ^of ^parameters ^is

n p = 7

^. ^The ^toleranes ^for

themainproblemandthesubproblemsaresetto

ε ₁ = 10 ⁻ ¹⁴

^and

ε ₂ = 10 ⁻ ⁴

Theoptions used inthe Marquardtalgorithm are opts

=

^[1e-8 ^1e-4 ^1e-4

200 1e-12℄. Thetest runsprodue the following iteration sequenes.

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.3.2: Withregularization

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.3.3: Without

regulariza-tion

Thedesiredauray

ε ₂

^of ^the^Parameter ^Extration ^problems ^has^a^rather

large value ompared to

ε ₁

^, ^and orrespondingly the options used in the stoppingriteria for marquardtareofthe same size. Thisishosenbeause

itgivesthebestresults. Ifwe altertheoptions to

ε ₂ = 10 ⁻ ¹⁴

^and^the

opts-vetorto[1e-8 1e-14 1e-14 200 1e-12℄thealgorithmatuallyonverges

slower asshowninthe gures4.3.4 and 4.3.5.

This seems strange, sine one would expet a higher auray of the

map-pingparameters, resulting ina better surrogatemodel andtherebya faster

onvergene. But this isnot the ase. It isnot obvious why this behaviour

0 5 10 15 10 ⁻¹⁵

10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure4.3.4: Withregularization

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.3.5: Without

regulariza-tion

gradientmathto thedesiredauray,and themarquardt-algorithm keeps

iterating,atually produing aworse setof mapping parameters.

In the ase of no regularization there is a possibility of having an

overde-termined problemfromthe

6

^th ^iteration. ^At ^this^point ^of ^the^iteration,^the

iterateisalreadyverylosetotheoptimizer,anditapparentlyisnoproblem

thattheregularization term is notadded.

Eet of the Normalization Fators

Thenormalizationfatorshave animportant inueneonboth theiteration

sequene and on the surrogate model in the optimizer. The next gures

show the performane of the algorithm with three ases of normalization

orrespondingto setion 4.1.1:

•

Normalization of allresidual elements

•

^Onlynormalization of thegradient residual

•

^No normalization

First weonsider theaseof no regularization.

Weseethatthe onvergeneisalittle slowerinase2,where

a ₁ , . . . , a _k = 1

but

d 6 = 1

^. ^With^no normalization thesolution is not as good ompared to theestimateof

x ^∗

In the gures 4.3.9-4.3.11 we see the approximation errors

E _s

^(light ^grid)

and

E l

^(dark^grid) orresponding to the threeases of saling.

Here there is a big dierene in the surrogate models: Case 1 with

om-0 5 10 15 10 ⁻¹⁵

10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.3.6: With

nor-malization

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.3.7: Only

nor-malization of gradient

residual

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.3.8: Without

normalization

0 50

100 150 40 60 80 100 120 140

0 1 2 3 4 5 6

x ₂ Approximation error for fine model Taylor appr. and surrogate model

with expansion point x = [ 74.23 , 79.27 ]

x 1

Norm of approximation error

Figure 4.3.9: With

nor-malization

0 50

100 150 40 60 80 100 120 140

0 1 2 3 4 5 6

x ₂ Approximation error for fine model Taylor appr. and surrogate model

with expansion point x = [ 74.23 , 79.27 ]

x 1

Norm of approximation error

Figure4.3.10: Only

nor-malization of gradient

residual

0 50

100 150 40 60 80 100 120 140

0 1 2 3 4 5 6

x ₂ Approximation error for fine model Taylor appr. and surrogate model

with expansion point x = [ 74.26 , 79.24 ]

x 1

Norm of approximation error

Figure 4.3.11: Without

normalization

use only the normalization fator

d

^for ^the ^gradient ^residual, ^the ^model ^is

well-behaved and provides a muh better approximation to the ne model.

Thethirdasewithnosalingoftheresidual vetoratall,providesabetter

resultthan the rstase, but the seond aseis stillto prefer.

Foromparisonweviewtheiterationsequenesintheaseoftheregularized

residual.

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* || ₂ F(x ^(k) ) − F(x ^* )

Figure4.3.12: With

nor-malization

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* || ₂ F(x ^(k) ) − F(x ^* )

Figure4.3.13: Only

nor-malization of gradient

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* || ₂ F(x ^(k) ) − F(x ^* )

Figure 4.3.14: Without

normalization

0 50

100 150 40 60 80 100 120 140

0 1 2 3 4 5 6

x ₂ Approximation error for fine model Taylor appr. and surrogate model

with expansion point x = [ 74.23 , 79.27 ]

x ₁

Norm of approximation error

Figure4.3.15: With

nor-malization

0 50

100 150 40 60 80 100 120 140

0 1 2 3 4 5 6

x ₂ Approximation error for fine model Taylor appr. and surrogate model

with expansion point x = [ 74.23 , 79.27 ]

x ₁

Norm of approximation error

Figure4.3.16: Only

nor-malization of gradient

residual

0 50

100 150 40 60 80 100 120 140

0 1 2 3 4 5 6

x ₂ Approximation error for fine model Taylor appr. and surrogate model

with expansion point x = [ 74.21 , 79.29 ]

x ₁

Norm of approximation error

Figure 4.3.17: Without

normalization

Asseenfromgures4.3.12-4.3.14thenormalizationfatorsareofnorelevant

importane of the performane in the ase of regularization. The solution

is again less aurate in the third ase without any normalization. Also

thesurrogate modelapproximationerrorsingures4.3.15-4.3.17 arealmost

unaetedbythesaling.

We instead onsider the ase where we put

a ₁ , . . . , a _k

^equal ^to

1

^, ^but ^still

havethenormalization fator

d 6 = 1

^,^we^get ^the ^resultsⁱⁿ^gure ^4.3.18:

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

0 50

100 150 40 60 80 100 120 140

0 1 2 3 4 5 6

x ₂

Approximation error for fine model Taylor appr. and surrogate model with expansion point x = [ 74.23 , 79.27 ]

x ₁

Norm of approximation error

Figure 4.3.18: Withnormalization only ofgradient residual

We see that the partly normalization is resulting in a signiantly better

surrogate approximationerror, whih isnowbetter that theapproximation

error fromusing alinearTaylor model.

We onlude that for this partiular problem, thesaling of theresidual

el-ementsis important for the qualityof thesurrogateapproximation, butnot

Eet of the Weighting Fators

Herewetestsomeeetsoftheweightingfatorsbylookingattheasewith

noregularization oftheresidual. Theweightingfatorshavenoeetonthe

iteration sequene, asitisseenfrom thegures 4.3.19 and4.3.20.

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.3.19: With normalization

andwithout weighting

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.3.20: Withnormalization

and with weighting

Thenumberofunknownparametersis

n p = 7

^,^whih^means,^that^the

weight-ingfators will possibly inuene the results fromiteration

6

^and ^onwards.

Butthesurrogatemodelapproximationerrororrespondingtotheweighted

aseissimilarto gure4.3.9,and servesasapoorapproximationto thene

model.

Atlastweonsidertheweightingstrategyombinedwithonlynormalization

of the gradient residual. The results fromthis senario areshown in gure

4.3.21.

Now the surrogate is again well-behaved, whih is aused by putting the

normalization fators

a ₁ , . . . , a _k = 1

We onlude,that the weighting fatorsin thistest problem arepratially

without inuene on the results - both regarding performane of the Spae

Mappingalgorithmandregardingthequalityofthesurrogatemodel

approx-imationintheoptimizer.

Referring to setion 3.1 we an only expet the results to be within the

aurayofthemaximal errorsfrom notusingtheexat gradients. The

tol-eraneused inthese testresults is

ε ₁ = 10 ⁻ ¹⁴

^,^whih ^is ^probably^too ^strit.

We therefore onlude, that the problem is not suited for investigating the

0 5 10 15 10 ⁻¹⁵

10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

0 50

100 150 40 60 80 100 120 140

0 1 2 3 4 5 6

x ₂

Approximation error for fine model Taylor appr. and surrogate model with expansion point x = [ 74.23 , 79.27 ]

x ₁

Norm of approximation error

Figure4.3.21: Withnormalizationonlyofgradientresidualandwith

weight-ing

Eet of the Number of Mapping Parameters

We here bringthe resultsof the testruns withthediagonal inputmapping

parametermatrix

A

^. ^In^this ^ase^we^have

n p = 5

^elementsⁱⁿ^the^parameter

vetor

p

^. ^There ^is^a possibilityof having anoverdetermined systemin iter-ation

4

^. ^The^test ^runs ^are^made ^with^the ^tolerane ^parameters

ε ₁ = 10 ⁻ ¹⁴

ε ₁ = 10 ⁻ ⁴

^and ^opts

=

^[1e-8 ^1e-4 ^1e-4 ²⁰⁰ ^1e-12℄. ^Figures ^4.3.22 ^and

4.3.23 show the results with a diagonal input mapping matrix in the

regu-larizedand the unregularized ase.

0 5 10 15 20 25

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* || ₂ F(x ^(k) ) − F(x ^* )

Figure 4.3.22: Withregularization

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* || ₂ F(x ^(k) ) − F(x ^* )

Figure 4.3.23: Without

regulariza-tion

We seethatthe onvergene is slowerompared to gures4.3.12 resp. 4.3.6

bothwithandwithouttheregularizationtermadded,whenonlyonsidering

theredued parameter vetor. The orresponding approximation errorsfor

gures4.3.24 and 4.3.25.

0 50

100 150 40 60 80 100 120 140

0 1 2 3 4 5 6

x ₂

Approximation error for fine model Taylor appr. and surrogate model with expansion point x = [ 74.23 , 79.27 ]

x ₁

Norm of approximation error

Figure4.3.24: Withregularization

0 50 100 150

40 80 60 120 100 140 0

2 4 6 8 10

x ₂

Approximation error for fine model Taylor appr. and surrogate model with expansion point x = [ 74.23 , 79.27 ]

x ₁

Norm of approximation error

Figure 4.3.25: Without

regulariza-tion

Againthesurrogatemodelapproximationerrorislargefortheunregularized

aseompared to theregularized. In gure4.3.25 thesurrogate

approxima-tionisnotasgoodastheapproximationwithalinearTaylormodelinmost

afthe design parameter region.

Alsoin thisase the tolerane options areof great importane. We run the

algorithm with smaller toleranes:

ε ₁ = ε ₂ = 10 ⁻ ¹⁴

^and ^the

marquardt-options [1e-8 1e-14 1e-14 200 1e-12℄. As the results in gures 4.3.26

and 4.3.27 show, the onvergene is now as fast as withthe full parameter

vetor. Thisappliesforboththeasewithregularization andthease

with-out regularization.

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure4.3.26: Withregularization

0 5 10 15

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.3.27: Without

regulariza-tion

The approximation errors

E s

^and

E l

orresponding to the regularized and

unregularizedtests,areseeningures4.3.28and4.3.29. Theapproximation

error for the surrogate model is not better, when using a smaller tolerane

value.

0 50

100 150 40 60 80 100 120 140

0 1 2 3 4 5 6

x ₂

Approximation error for fine model Taylor appr. and surrogate model with expansion point x = [ 74.23 , 79.27 ]

x ₁

Norm of approximation error

Figure 4.3.28: Withregularization

0 50

100 150 40 60

80 100 120

140 0

2 4 6 8 10

x ₂

Approximation error for fine model Taylor appr. and surrogate model with expansion point x = [ 74.23 , 79.27 ]

x 1

Norm of approximation error

Figure 4.3.29: Without

regulariza-tion

We onlude that the optional tolerane values are of great importane in

this problem. A smaller tolerane here results in faster onvergene, but

thesurrogateapproximationerrorsarenoteetedpositively bythesmaller

tolerane. In order to ensure good surrogate approximations over a large

regionofthe designparameterspae, wemustputthenormalizationfators

a ₁ , . . . , a k

^equal ^to

1

Optimal Mapping Parameters

Finallywe lookat thevalues ofthemappingparameters intheoptimal

sur-rogate model. We have the initial mapping parameters

A _i = I

b _i = 0

^and

α i = 1

^for^all^response^funtions

i = 1, . . . , 11

ⁱⁿ^iteration

0

^. ^It^isinteresting to seehowdierent the optimal mapping parameters arefrom thesevalues.

We onsidertwo test senarios:

•

^Withregularization and withnormalization (gure4.3.15)

•

^Withoutregularization and withnormalization (gure 4.3.9)

In the rst ase a representative matrix

A

ⁱⁿ ^the ^optimal ^surrogate ^model

is:

A =

1.09 0.10 0.05 1.07

The elements of

b _i

^are ^of ^order ^of ^magnitude

10 ⁻ ³

^, ^and ^all ^values ^of ^the

output mapping parameters

α

^v^ary ⁱⁿ ^the ^interval

[0.75 , 1.45]

^. ^But ^for

responsefuntion number

2

^we ^have:

A ₂ =

− 0.06 − 1.27

− 1.11 0.34

b ₂ =

− 0.024 0.018

whih is not lose to the identity matrix. We onlude that most of the

matries

A

^are^not ^very^dierent^from ^the^initial ^identity^matrix, ^and^most

ofthe elementsin

b

^are^lose ^to

0

^. ^The^surrogate^model^with^these^mapping

parameters iswell-behaved aswe have seeningure4.3.15.

But in the ase of no regularization we nd a muh more varying piture.

Someofthe

A i

^'s^are^lose^to^the^identity^matrix^some^are^not,^though^all

A i

-elements have absolute values between

0

^and

3.5

^. ^Again ^response ^funtion

number

2

^is ^speial:

A ₂ =

− 0.37 − 0.31 2.12 1.70

b ₂ =

− 93.91

− 65.52

α ₂ = − 12.85

Some

b

^-vetors^have^elements^lose^to

0

^,^but^as^the^result^above ^shows^there

are also examples of extremely dierent

b

^-values. ^The

α i

^'s ^vary ^between

− 12.85

^and

1.42

^. ^The^surrogate ^modelapproximation errorshown ingure 4.3.9 hasextreme variation, whih ouldbe aonsequene of thevariations

ofthe mapping parameters.

This behaviour of the mapping parameters is probably onneted to the

missingregularizationterm. Intheregularizationaseweforethemapping

parameterstobelosetothepreviousset, andinthisway,weannotendup

withresultsveryfar from theinitial values. We onlude,thatifwe donot

regularize wrt.the previous parameter set, we ouldend up withasolution

veryfar from the previous.

Finallywe onsiderthe mapping parameters orrespondingto gure 4.3.10.

Herethe approximationerrorforthe surrogateisnotaslarge asbefore. The

mapping parameters forresponsefuntion

2

^are^now:

A ₂ =

4.08 4.52

− 1.06 2.73

b ₂ =

− 231.01

− 72.83

α ₂ = − 0.38

Generallythemappingparametersinthisasestillvarymuhfromresponse

to response, but apparently thesurrogate approximation isbetter.

Diret Optimization

For omparing the Spae Mapping algorithm with a lassial optimization

modelbythetwoalgorithmsdiretanddiretdfromtheSMISframework

implementedbyFrank Pedersen.

0 10 20 30 40 50 60

10 ⁻²⁰ 10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

0 5 10 15 20 25 30 35 40

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.3.30: Performane of diret optimization of the ne model (diret

left,diretdright)

The diret-algorithm uses a Broyden updated approximation of the rst

orderpartialderivatives, whiletheotherusestheJaobian diretlyfromthe

ne modelfuntion.

Itisobvious,thatthe SpaeMappingalgorithm ismuhmore eientthan

thelassialoptimization algorithms.

Summary of the Results

By running the dierent versions of the Spae Mapping algorithm on this

test problem,wehave seen, thattheresults arevery dierent. Itis diult

to onlude, why the results look as they do, and impossible to generalize

the behaviour to other problem types. But for this partiular problem the

results showthe following:

•

^Theregularization seemsto have a positive eet onboth the onver-genespeed and the optimalsurrogate aproximation.

•

^The normalization fators have only little eet on the onvergene speed, but a big inuene on the quality of thesurrogate

approxima-tion, whenwe don't useregularization.

•

^The^weighting^fators^do ^not ^seem^to ^have ^a^notiable ^eet ^on^either

theonvergene or the surrogateapproximation.

•

^The ^redution ^of ^the ^mapping ^parameters ^still ^provides ^good

onver-generesults,although the tolerane options have an eet.

•

^The^optimal ^mapping ^parameters ^are^inuened ^by^theregularization

term.

•

^The ^tolerane ^options ^have ^an ^eet ^on^the^iteration ^sequenes.

In document IMM YGBY2004ESAESRETR.54 ei (Sider 74-85)

4.3 The TL T2 Problem

4.3.2 The Results of the T est Runs

A

n p = 7

ε 1 = 10 − 14

ε 2 = 10 − 4

=

0 5 10 15

10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

0 5 10 15

10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

ε 2

ε 1

ε 2 = 10 − 14

0 5 10 15 10 −15

10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

0 5 10 15

10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

6

•

•

•

a 1 , . . . , a k = 1

d 6 = 1

x ∗

E s

E l

om-0 5 10 15 10 −15

10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

0 5 10 15

10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

0 5 10 15

10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

0 50

100

150 40 60 80 100 120 140

0 1 2 3 4 5 6

x 2 Approximation error for fine model Taylor appr. and surrogate model

with expansion point x = [ 74.23 , 79.27 ]

x 1

Norm of approximation error

0 50

100

150 40 60 80 100 120 140

0 1 2 3 4 5 6

x 2 Approximation error for fine model Taylor appr. and surrogate model

with expansion point x = [ 74.23 , 79.27 ]

x 1

Norm of approximation error

0 50

100

ε ₁ = 10 ⁻ ¹⁴

ε ₂ = 10 ⁻ ⁴

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

ε ₂

ε ₁

ε ₂ = 10 ⁻ ¹⁴

0 5 10 15 10 ⁻¹⁵

10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

a ₁ , . . . , a _k = 1

x ^∗

E _s

om-0 5 10 15 10 ⁻¹⁵

10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

x ₂ Approximation error for fine model Taylor appr. and surrogate model

x ₂ Approximation error for fine model Taylor appr. and surrogate model

x ₂ Approximation error for fine model Taylor appr. and surrogate model

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* || ₂ F(x ^(k) ) − F(x ^* )

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* || ₂ F(x ^(k) ) − F(x ^* )

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* || ₂ F(x ^(k) ) − F(x ^* )

x ₂ Approximation error for fine model Taylor appr. and surrogate model

x ₁

x ₂ Approximation error for fine model Taylor appr. and surrogate model

x ₁

x ₂ Approximation error for fine model Taylor appr. and surrogate model

x ₁

a ₁ , . . . , a _k

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

x ₂

x ₁

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )