The Penalty F ator

P ^′ _x (x, p) = A =

A ₁₁ 0 0 A ₂₂

, P ^′ _p (x, p) = H(x) =

x ₁ 0 1 0 0 x ₂ 0 1

(3.4.9)

where

n p = 2n

^{. The new redued matrix}

H

^{is the result of} eliminating the seond and the third olumn of the old

H

^{, sine we have eliminated the}

seondandthirdelement of

p

^{. Itis}straightforward toprodue thematries

A

^and

H

orresponding to an arbitrary element elimination of the full pa-rametervetor

p ∈ R ^(n(n+1))

^.

When using the diagonal matrix approah we redue the number of input

parameters by a fator

n+1

2

^{. Aordinglywe an expet a redution of the}

alulationtime, whihan beuseful inproblems oflarge dimension.

A dierent approah ould be to alter the number of parameters for every

mainiteration,withtheaimof thepossibilityofrepeatedlyhavingaunique

solution to the Parameter Extration. The length of the residual vetor is

inreased by

1

^{for eah main iteration, starting at}

n

^for

k = 0

^{, and we an}

augment the

p

^{-vetor with one extra element. The order of the elements}

an be hosen in many ways, a suggestion would be to begin with

p = [α]

and augment with the diagonal elements of

A

^{one by one followed by the}

b

^{-elements. The initial value for the mapping parameters would still be}

A ⁽⁰⁾ _i = I

^,

b ⁽⁰⁾ _i = 0

^,

α ⁽⁰⁾ _i = 1

^{,and this initial valueis valid, until the}

param-eteris ontained in

p

^.

lem, if an exat math is possible in the ases, where we have either more

parameters than residual elements or the same number of both. Usually

it is possible to satisfy all equations exatly, but one an nd examples of

problems,where itisnot. Weassumethattheproblemsonsideredhereare

onsistent. In ase of a full rank onsistent problem the weighting of the

residual elements has no eet on thesolution. When theproblem is rank

deient the solution spae for the unweighted problem is the same as for

the weighted problem, andthe twoproblems have idential minimum norm

solutions.

Inthe aseof an non-onsistent problem, theweighting fatorsan havean

eet.

Underdetermined System of Equations

Iftherearemoreparameters

n p

^{than residualelements}

n r

^{,theproblemdoes}

not have a unique solution. The Jaobian of

r

^{is denoted}

J _r

^{and has the}

rank

q

^{, where}

q ≤ n r

^{. There will then be an}

(n p − q)

^{-innity of solutions}

to the Parameter Extration. For eah of the innitely many solutions we

arethen sure,that all

n r

^{equations aresatised,ie.}

r(p ^∗ ) = 0

^{. Algorithm3}

provides the solutiondened bythe Marquardtparameters

µ

^.

Themultipliationwiththeweigthingfatorsnowresultsintheresidual

Wr

andthe Jaobian

WJ r

^,where

r

^and

J r

^{refer toequation (3.3.9) ,andwhere}

the diagonal matrix

W

^{isgiven by(3.5.1) :}

W =





















w ₁ 0

.

.

.

w _k σ

.

.

.

0 σ





















(3.5.1)

Inaseof a onsistent problem thesolutionto thepenalizedproblem isthe

same as the unpenalized solution. In this ase the penalty fators have no

eet onthesolution.

Overdetermined System of Equations

Provided that

J r

^{hasfullrankintheasewhere}

n r = n p

^{,thesolutiontothe}

Parameter Extration isunique. All theequations aresatisedexatly,and

sine

r(p ^∗ ) = 0

^,thenalso

W · r(p ^∗ ) = 0

^.

Iftherearemoreequationsthanunknownparameters(

n r > n p

^),wendthe

leastsquaressolutionbyAlgorithm3. Weannotbesuretosatisfyall

equa-tions, and the weight of eah equation determines, whih solution is found,

sine this solution minimizes

1/2 · r

^T

r

^{. A largepenaltyfator will resultin}

a solution thatis guaranteed to satisfythegradient residual, provided that

thesaling of theresidual elements isnot bad. If theproblem onthe other

hand isonsistent,the penaltyfator hasno inuene.

3.6 The Weighting Fators

Sine the weighting fators funtion in the same way as the penalty fator

disusedintheprevious setion,theanalysisregardingthesystemsof

equa-tions isalso valid inthepresent setion.

In the new Spae Mapping formulation we wish the mapping parameters

to minimize the residual funtion (1.3.12) , onsisting of the funtion value

residualandthegradientresidual. Weassume,thatwehavemade

k + 1

iter-ations inthe mainalgorithm. Therstiterateis

x ⁽⁰⁾

^{. Theresidualdepends}

on

k

^ofthe

k + 1

^{iterates,aswellasthegradient wrt.thedesignparameters}

inthebestiterate, sinethefuntionvaluesoftheneandsurrogatemodels

in the best iterate already math. The length of the residual vetor is

de-noted

n r

^{anddependson the numberof main iterationsand thenumber of}

designparameters (

n r = k + n

^{). Withuseofthe}regularizationterm

n r

^also

dependsonthenumberofmapping parameters,inthisase

n r = k + n + n p

^.

We introdue weighting fatorsforthefuntion valueresidualrows denoted

w i

^for

i = 1, . . . , k

^{. Thegradientresidualisweightedwiththepenaltyfator}

σ

^{,whihis disussedinsetion 3.5.}

Linear Weight Funtion

IntheimplementationoftheSMISframeworkbyFrankPedersen,the

weight-ingfatorsaregiven bya linearfuntion depited ingure3.6.1.

where

ε res

^{is a given threshold value (the gure shows}

ε res = 0.25

^{). The}

weighting funtion dereases linearly from

1

^to

0

^{as the distane to the}

best iteration point grows, giving the expression for the weighting fators

w ₁ , . . . , w k

^:

0 0.2 0.4 0.6 0.8 1 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

dX

w

Linear weight function w(dX)

Threshold value ε _r = 0.25

Figure3.6.1: Linear weight funtion

w i =



 

 

1

^for

dX _i < ε _res 2 − ^dX ε res ⁱ

for

dX i < 2ε res

0

^for

dX _i ≥ 2ε _res

Gauss Distributed Weight Funtion

Another approah is to let the weighting fators depend on thenumber of

residualrows

n _r

^{. Thenumber ofunknown parameters is}

n _p

^{,henewe need}

atleast

n p − n + 1

^{iterationsto ensure,thattheproblemis not} underdeter-mined.

We hoosea weight funtion asa Gaussurve.

0 0.2 0.4 0.6 0.8 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

dX

w

Gauss weight function w(dX)

Threshold value ε _r = 0.1

Figure3.6.2: Gaussdistributed weight funtion

w i =

( 1

^if

n r < n p

exp( − γ · dX _i ² )

^if

n r ≥ n p

(3.6.1)

where the fator

γ

^{is determined by}

γ = − ^ln(ε _{dX ¯} ^{res 2 )}

^{, with}

dX ¯

orresponding to the

(n p − n)

^{th best iterate. This gives the weighting fator}

ε res

^{for this}

partiular residualelement. Inthisways theweighting fatorsfor the

resid-ual elements orresponding to the best iterates are kept above or equal to

the threshold value

ε _res

^{, and all other points are onsidered low priority.}

Theweighting funtion for

ε r = 0.10

^{isseen ingure3.6.2. Ifthere arenot}

enough rows in the residual to math the number of unknown parameters,

allresidual rows areweighted equallywithafator

1

^.

IftheParameterExtrationproblemisoverdetermined,theweightingfators

makesure thatthe bestpointsaregiven the highestpriority. Asmentioned

before the weighting fators do not inuene the solution, if the system is

onsistent. The threshold valueis usedto deide,how important arole the

'unneessary'iterates should playinndingthe optimal parameter set.

If theParameter Extration problem is underdetermined, it is usually

pos-sibleto satisfyall equations exatly. Inthis asethe weighting fatorshave

no eet on the solution, sine if

r(p ^∗ ) = 0

^{then also}

W · r(p ^∗ ) = 0

^{. The}

strategy for determining the weighting fators inthease

n _r < n _p

^is

there-forenot important,aslong asthe

w

^{'sarenot equal to zero.}

In document IMM YGBY2004ESAESRETR.54 ei (Sider 49-53)