The Results of the T est Runs

4.4 The TL T7 Problem

4.4.2 The Results of the T est Runs

Eet of the Regularization

Thealgorithm does not onvergeto theoptimizerinthe asewhere we use

theregularized residual inthe Parameter Extration. When solvingthe

un-regularizedParameterExtrationproblemwegettheperformaneresultsof

the algorithm depited in gure 4.4.2. The result is with normalization of

the residualelements anda full sizeparameter vetor.

The optimizer is found in 10 iterations within the auray

10 ⁻ ¹⁶

^of ^the

objetive funtion. Thenumberofunknown mappingparameters ineahof

the Parameter Extration problems is

n p = 57

^. ^In ^the ^ase ^of ^no

regular-ization this means, that until iteration

51

^we ^will ^have ^less^equations ^than

unknowns. Thisis obviouslynot an obstale, sine we ndthe optimizerin

12

iterations.

It is not known, why the test senario with theregularized Parameter

Ex-0 5 10 15 10 ⁻²⁰

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure4.4.2: Withoutregularization

Eet of the Normalization Fators

Ingure4.4.3we seethe iterationsequenefor the aseofnonormalization

of the residual vetor, orresponding to

a ₁ , . . . , a _k = 1

^and

d = 1

^. ^The^test

isomputed withfull parametervetor.

Theonvergene is alittle faster than theresultfrom gure4.4.2.

0 5 10 15

10 ⁻²⁰ 10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure4.4.3: Withoutnormalization

Eet of the Weighting Fators

With no redution of the number of unknown mapping parameters in the

Parameter Extration we have

n p = 57

^, ^and ^hereby ^the ^weighting ^fators

may inuene the results from iteration number

51

^. ^Beause^the ^optimizer

isalreadyfoundatthispoint,weannotusethistestsenariotoinvestigate

Instead we look at the senario withredution of the parameter vetor. In

thisasewehave

n p = 15

^,^and^some^of^the^weighting^fators^will^be^dierent

from

1

^from ^iteration

9

^and ^onwards, ^if ^we ^use ^the ^strategy ^from ^setion

3.6. Figure 4.4.5 on the right shows the iteration sequene with the useof

weightingstrategy. Foromparison theresults withoutweighting areshown

ingure4.4.4 onthe left.

0 5 10 15

10 ⁻²⁰ 10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.4.4: Withoutweighting

0 5 10 15

10 ⁻²⁰ 10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure4.4.5: Withweighting

The gures are almost idential, and the weighting fators are pratially

withoutinuene inthis problem.

Eet of the Number of MappingParameters

Wenowshowtheresultsfromusingareduedparametervetor,

orrespond-ingtoadiagonal matrixforthe inputmappingmatrix

A

^. ^Figure^4.4.6^is^for

the regularized ase, andgure 4.4.7for theunregularized ase.

Thealgorithmdoesnot onvergeintherstasewithregularization. Inthe

seondase theoptimizeris foundin

12

^iteration ^steps ^with^an ^auray^of

10 ⁻ ¹⁵

^on ^the ^ne ^model ^objetive ^funtion. ^It ^is ^not ^known, ^why ^the ^test

runwith regularizationdoesnot onverge to theoptimizer.

Thereispratiallynodierenebetweentheiterationswiththefull

parame-tervetor(gure4.4.2)andthereduedparametervetor(gure4.4.7). The

algorithm works well when solving the unregularized Parameter Extration

problem,and ithasno eet whetherthe problemsareunderdetermined or

0 10 20 30 40 50 60 70 80 10 ⁻²⁰

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* || ₂ F(x ^(k) ) − F(x ^* )

Figure4.4.6: Withregularization

0 5 10 15

10 ⁻²⁰ 10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure 4.4.7: Without

regulariza-tion

Optimal Mapping Parameters

This problem is of larger dimension than the other test problems, and we

havenot investigated the optimal mapping parameters.

Diret optimization

Belowwe seetheperformaneof thediretd-algorithm.

0 20 40 60 80 100 120

10 ⁻²⁰ 10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

Performance

Iteration

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

Figure4.4.8: Performaneof diretoptimization ofthene model

The lassial optimization method onverges in

127

iterations, and we on-ludethatthe SpaeMappingtehnique isveryuseful inthisproblem,sine

itreduesthe numberofnemodelevaluationswithapproximately afator

10

Summaryof the Results

•

^When ^the ^Parameter ^Extration ^problem ^is regularized we have no onvergene to theoptimizer, both inthe aseof afull and a redued

parameter vetor.

•

^The normalization fatorshave littleeet on theonvergene speed.

•

^The ^weighting ^fators ^have ^pratially ^no ^inuene ^on ^the ^results ⁱⁿ

the aseofa redued parametervetor.

•

^The ^redution^of ^the^mapping ^parameters ^has^no ^eet ^on ^the

perfor-mane inthis test problem.

•

^It ^is^not^possible^to^get^as^good^onvergene ^results^as^the^results^from

[1 ℄ and [2 ℄, probably beause the problem is not idential to the one

solvedinthis report.

Chapter 5

Future Work

5.1 Improvements of the SMIS Implementation

Anumberofhangesoftheimplementation ouldbemadeinorderto make

theSMISframeworkmoreexibleanduser-friendly. Alsotheaurayofthe

omputationsouldbeimprovedforsomeproblems. Theurrentframework

onsistsof many diretories and les linked together ina ertain struture.

The suggested hanges will therefore inuene many of the inluded les,

whihwill eet theduration of thework involved.

Intheurrentimplementation thegradientsofthesurrogatemodelwrt.the

parameter vetor are alulated by forward dierene approximations. In

ordertominimizethetrunationerrors,itwouldbeanadvantageto exploit

theexat gradients, ifthey areavailable.

TheMatlab-lediffjaobianusedtoalulatetheforward dierene

ap-proximation. In the urrent implementation the parameter

η

^dening ^the

relative step length is xed at a ertain value, and an only be altered by

modifyingthe Matlab-le diretly.

Theuser friendliness ouldbeimproved byinluding more optional

param-eters to the problem setup-le. The setup-le should dene the following

optionalparameters:

∆

^Initial^trust ^region ^radius ^for ^Algorithm^1.

max _f1

^Maximal ^number^of ^funtionevaluations inAlgorithm1.

max _f2

^Maximal ^number^of ^funtionevaluations inAlgorithm2.

max _f3

^Maximal ^number^of ^funtionevaluations inAlgorithm3.

η x

^Step^length ⁱⁿdiffjaobianwhenalulatingJaobian wrt.

x

η p

^Step^length ⁱⁿdiffjaobianwhenalulatingJaobian wrt.

p

ε _F

^Usedⁱⁿ^the ^stopping ^riterion ^for ^the^ne ^model^objetive ^funtion.

ε K

^Usedⁱⁿ^the ^stopping ^riterion ^for ^the^gradient ^mathing.

ε hx

^Usedⁱⁿ^the ^stopping ^riterion ^for ^the^step ^length^for

x

^-iterates.

ε _hp

^Usedⁱⁿ^the ^stopping ^riterion ^for ^the^step ^length^for

p

^-iterates.

opts Options formarquardt ([1e-8 1e-4 1e-4 200 1e-12℄).

diagA Parameter dening thenumberof mapping parameters

(0: fullmatrix

A

^,^1: ^diagonal ^matrix

A

⁾

Someof the above options arealready used in theSMIS framework, ie.

∆

max _f1

max _f2

^,^and^diagA,^and^also^two ^tolerane^parameters orresponding to

ε F

ε K

ε hx

^and

ε hp

We also list some possible hanges inorder to inrease the auray of the

omputations:

•

^Introduing^the^step ^length^parameter

η

^as^an^input ^parameter^to ^the

funtion diffjaobian.

•

^Making ^it^possible ^to ^exploit^exat ^gradients, ^if^they ^are^available.

In document IMM YGBY2004ESAESRETR.54 ei (Sider 86-92)

4.4 The TL T7 Problem

4.4.2 The Results of the T est Runs

10 − 16

n p = 57

51

12

Ex-0 5 10 15 10 −20

10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

a 1 , . . . , a k = 1

d = 1

0 5 10 15

10 −20 10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

n p = 57

51

n p = 15

1

9

0 5 10 15

10 −20 10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

0 5 10 15

10 −20 10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

A

12

10 − 15

0 10 20 30 40 50 60 70 80 10 −20

10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * || 2 F(x (k) ) − F(x * )

0 5 10 15

10 −20 10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

0 20 40 60 80 100 120

10 −20 10 −15 10 −10 10 −5 10 0 10 5

Performance

Iteration

||x (k) −x * ||

2 F(x (k) ) − F(x * )

127

10

•

•

•

•

•

η

∆

max f1

max f2

max f3

η x

x

η p

p

ε F

ε K

ε hx

x

ε hp

p

10 ⁻ ¹⁶

Ex-0 5 10 15 10 ⁻²⁰

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

a ₁ , . . . , a _k = 1

10 ⁻²⁰ 10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

10 ⁻²⁰ 10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

10 ⁻²⁰ 10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

10 ⁻ ¹⁵

0 10 20 30 40 50 60 70 80 10 ⁻²⁰

10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* || ₂ F(x ^(k) ) − F(x ^* )

10 ⁻²⁰ 10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

10 ⁻²⁰ 10 ⁻¹⁵ 10 ⁻¹⁰ 10 ⁻⁵ 10 ⁰ 10 ⁵

||x ^(k) −x ^* ||

2 F(x ^(k) ) − F(x ^* )

max _f1

max _f2

max _f3

ε _F

ε _hp

max _f1

max _f2