(QGYLGHUHXGE\GHUIDNXOWHWHWIRUVNHOOLJHXGGDQQHOVHVIRUOELVDPDUEHMGHPHGYLUNVRPKHGHURJ RIIHQWOLJHLQVWLWXWLRQHU'LVVHRPIDWWHUEODQGWDQGHWIRUVJVYLV:RUN%DVHG/HDUQLQJGHUEHQ\WWHU GHQSGDJRJLVNHWLOJDQJIUDGHQSUREOHPRULHQWHUHGHRJSURMHNWRUJDQLVHUHGHXQGHUYLVQLQJVPRGHO SnNRQNUHWHSUREOHPVWLOOLQJHULHQYLUNVRPKHG,GHHQHUVnOHGHVDWYLUNVRPKHGHQEnGHInUOVW NRQNUHWHSUREOHPHURJXGGDQQHWVLQHPHGDUEHMGHUH
$QDO\VHURJYXUGHULQJHU
'HWHUHYDOXHULQJVJUXSSHQVRSIDWWHOVHDWIDNXOWHWHWKDUXGYLNOHWHQXGGDQQHOVHVPRGHOGHU JUXQGOJJHQGHHUYHOHJQHWRJIRUPnOVWMHQOLJVHWLIRUKROGWLOIDNXOWHWHWVSURILORJVRPJLYHUJRG PXOLJKHGIRUDWXQGHUVWWWHHWRPYHUGHQVRULHQWHUHWIRNXV8GGDQQHOVHVPRGHOOHQHUHQGYLGHUH NHQGHWHJQHWYHGHQKMVWXGLHHIIHNWLYLWHWDWGLPLWWHQGHUQHHUWLOIUHGVHPHGGHQRJDWGHKXUWLJW NRPPHULDUEHMGH
3nLQVWLWXWLRQVEHVJHWEOHYGHUSnWYUVDIDOOHJUXSSHUVWXGHUHQGHXQGHUYLVHUHIRUVNHUHRJ HUKYHUYVOLYXGWU\NWVWRUWLOIUHGVKHGPHGRJRSEDNQLQJWLOXGGDQQHOVHVPRGHOOHQ'HWEOHYLPLGOHUWLG IUHPKYHWDWGHUEODQGWEnGHVWXGHUHQGHRJVWXGLHOHGHUHHUHQRSPUNVRPKHGRPNULQJIRUKROG YHGXGGDQQHOVHVPRGHOOHQGHUPHGIRUGHONXQQHQGUHV%nGHEODQGWVWXGLHOHGHUQHRJEODQGWGH VWXGHUHQGHEOHYGHUJLYHWXGWU\NIRUDWVWXGLHPRGHOOHQPHGWLVHPHVWUHPHGSURMHNWRSJDYHULNNH QGYHQGLJYLVHURSWLPDORJGHUEOHYHIWHUO\VWPLQGUHRJIUUHSURMHNWHUVDPWLQGLYLGXHOWDUEHMGHL QRJOHSURMHNWHU'HWEOHYIUHPKYHWSnLQVWLWXWLRQVEHVJHWDWGHUHNVSHULPHQWHUHVPHGUHVXOWDWHW SURGXNWHWDISURMHNWDUEHMGHWVnOHGHVDWGHUSnQRJOHVWXGLHULHQNHOWHVHPHVWUHNDQDIOHYHUHVHQ DUWLNHOHOOHUHQSRVWHULVWHGHWIRUHQSURMHNWUDSSRUW
(QGYLGHUHIUHPKYHGHQRJOHXQGHUYLVHUHSnEHVJHWDWSURMHNWHQKHGVNXUVHUQHLNNHLWLOVWUNNHOLJ JUDGSUYHVYHGSURMHNWHNVDPLQHUQHRJDWGHW\GHUPHUHRIWHHUJUXQGOJJHQGHIDJHOHPHQWHUGHU EHKDQGOHVLGLVVHNXUVHUGHUGHUIRUOLJHVnJRGWNXQQHKDYHKDIWVWDWXVDIHWVWXGLHHQKHGVNXUVXV PHGHNVDPHQ6DPWLGLJHUGHUIRUVNHOSnRPVWXGLHHQKHGVNXUVHUQHEHGPPHVHIWHUVNDODHQ HOOHUPHGEHVWnHWLNNHEHVWnHW
(YDOXHULQJVJUXSSHQDQEHIDOHUDWIDNXOWHWHWEUXJHUHYDOXHULQJHQVRPHQDQOHGQLQJWLOHQ V\VWHPDWLVNJHQQHPJDQJDIXGGDQQHOVHVPRGHOOHQPHGKHQEOLNSnDWXGYLNOHGHQ\GHUOLJHUH
,ERJHQ7KH$DOERUJ(NVSHULPHQWIUDIUHPKYHVGHWDWXGGDQQHOVHVPRGHOOHQKDUEHW\GQLQJ IRUGLPLWWHQGHUQHVIDJOLJHSURILO%RJHQIUHPKYHULHQVDPPHQOLJQLQJPHG
FLYLOLQJHQLUGLPLWWHQGHUIUD'DQPDUNV7HNQLVNH8QLYHUVLWHWIDNXOWHWHWVFLYLOLQJHQLUGLPLWWHQGHUVRP VWUNHLQGHQIRUSUREOHPOVQLQJNRPPXQLNDWLRQVDPDUEHMGHRJJHQHUHOWHNQLVNYLGHQPHQVRP PLQGUHVWUNHLQGHQIRUSDUDWYLGHQRJIDJOLJPHWRGH'HQQHYXUGHULQJDIGLPLWWHQGHUQHVIDJOLJH SURILOEOHYRJVnEHNUIWHWSnEHVJHW
(YDOXHULQJVJUXSSHQGHOHUYXUGHULQJHQDIDWXGGDQQHOVHVPRGHOOHQJHQQHPHWIRNXVSnNRQNUHWH LQJHQLUPVVLJHSUREOHPVWLOOLQJHURJJHQQHPSURMHNWRUJDQLVHULQJJLYHUGLPLWWHQGHUPHGVWUNH NRPSHWHQFHUVRPSUREOHPOVHUHRJHQG\EWJnHQGHNRPSHWHQFHSnXGYDOJWHRPUnGHU6DPWLGLJ
QVNHUHYDOXHULQJVJUXSSHQDWIUHPKYHYLJWLJKHGHQDIDWIDNXOWHWHWVLNUHUWLOVWUNNHOLJEUHGGHRJ G\EGHLGHQIDJOLJHIXQGHULQJKHUXQGHUIDJOLJPHWRGHLGHWGHUHIWHUHYDOXHULQJVJUXSSHQV RSIDWWHOVHHUHQLERHQGHULVLNRIRUDWIDNXOWHWHWVSULRULWHULQJNDQSnYLUNHGLVVHIRUKROG
3UDNVLVUHODWLRQHQLGHVWXGHUHQGHVSURMHNWHUJHQQHPLQYROYHULQJDIYLUNVRPKHGHUYXUGHUHVVRPHQ VXFFHV'HWHUHYDOXHULQJVJUXSSHQVYXUGHULQJDWVXFFHVHQHUNRPPHWJHQQHPHQDNWLYLQGVDWV EnGHSnIDNXOWHWVQLYHDXRJSnGHFHQWUDOWQLYHDX'HVXGHQHUGHWIRUPRGHQWOLJHQXQGHUVWWWHQGH IDNWRUDWXQLYHUVLWHWHWSnJUXQGDIVLQSODFHULQJLYLGXGVWUNQLQJXGJUGHQPXOLJKHG
YLUNVRPKHGHULRPUnGHWKDUIRUDWLQGGUDJHVWXGHUHQGHLGHUHVDUEHMGH
$WGHWHUO\NNHGHVIRUIDNXOWHWHWDWJHRSWDJHWWLOLQJHQLUXGGDQQHOVHUQHLHQSHULRGHKYRUGHW KDUYUHWYDQVNHOLJWDWUHNUXWWHUHSnRPUnGHWJHQHUHOWILQGHUHYDOXHULQJVJUXSSHQSRVLWLYW(Q PXOLJIRUNODULQJSnGHWWHHUXQLYHUVLWHWHWVRJIDNXOWHWHWVVDWVQLQJSnSURPRYHULQJDI
XGGDQQHOVHUQHRJIDNXOWHWHWVHYQHWLODWIRUQ\XGGDQQHOVHUQHVnGHIUHPVWnUVRPUHOHYDQWHIRUGH VWXGLHVJHQGH8GGDQQHOVHVPRGHOOHQVPRGXOLVHUHGHIRUPJLYHUPXOLJKHGIRUDWWLOSDVVH
XGGDQQHOVHUQHWLOGHVWXGLHVJHQGHVQVNHUPHGUHODWLYWODYHRPNRVWQLQJHU6DPWLGLJXQGHUVWWWHV WLOSDVQLQJHQDIXGGDQQHOVHUQHYHGDWGHWHUGHVWXGHUHQGHGHUYOJHUSURMHNWHURJKHULJHQQHP SUHVVHUSnIRUDWDNWXHOOHHPQHUWDJHVRSLSURMHNWHUQH$IEHVJHWILNHYDOXHULQJVJUXSSHQGHW LQGWU\NDWGHVWXGHUHQGHRJGHUHVSURMHNWDUEHMGHVSLOOHUHQYVHQWOLJUROOHIRUIRUQ\HOVHQDI IDNXOWHWHWVIDJOLJHSURILORJGHUPHGHUPHGWLODWVWWHGDJVRUGHQHQ(YDOXHULQJVJUXSSHQILQGHUDW IDNXOWHWHWLYLGXGVWUNQLQJWDJHUGHVWXGHUHQGHDOYRUOLJWRJDNWLYWVJHUDWLQWHJUHUHGHPL IDNXOWHWHWVPLOM
(YDOXHULQJVJUXSSHQEHPUNHUDWGHUHUVNHWHQEHW\GHOLJHNVSDQVLRQLVWXGHQWHUEHVWDQGHQXGHQ DWGHQIDVWHXQGHUYLVHUVWDEHUIXOGWPHG)UDPGHWPHGXQGHUYLVHUQHILNHYDOXHULQJVJUXSSHQGHW LQGWU\NDWGHWWHNXQKDUYUHWPXOLJWJHQQHPHQHNVWUDXQGHUYLVQLQJVLQGVDWVIUD
XQGHUYLVHUJUXSSHQ(QVnGDQHNVWUDRUGLQULQGVDWVPnIRUYHQWHVDWYUHYDQVNHOLJDWRSUHWKROGH SnODQJWVLJWIRUGLGHUOJJHVSUHVSnXQGHUYLVHUQHVHQJDJHPHQWRJIRUVNQLQJVWLG
3nGHQEDJJUXQGHUGHWHYDOXHULQJVJUXSSHQVDQEHIDOLQJDWIDNXOWHWHWRYHUYHMHURPGHWHU IRUVYDUOLJWLIRUKROGWLOXGGDQQHOVHUQHVNYDOLWHWIRUWVDWDWJHRSWDJHWDIVWXGHUHQGHXGHQHQ WLOVYDUHQGHIRUJHOVHLXQGHUYLVHUVWDEHQ6DPWLGLJNDQGHQQHRYHUYHMHOVHYUHYDQVNHOLJSnHW WLGVSXQNWKYRUGHUIUDHUKYHUYVOLYHWVVLGHHIWHUVSUJHVIOHUHGLPLWWHQGHUPHGWHNQLVN
QDWXUYLGHQVNDEHOLJEDJJUXQGRJQnUPXOLJKHGHQIRUDWXGYLGHXQGHUYLVHUVWDEHQEODQGWDQGHW KQJHUVDPPHQPHGXGYLNOLQJHQLIRUVNQLQJVPLGOHUQH
)DNXOWHWHWKDUDUEHMGHWDNWLYWPHGLQWHUQDWLRQDOLVHULQJDIXGGDQQHOVHUQHEnGHKYDGJnUHWDEOHULQJ DIPXOLJKHGHUIRUDWHJQHVWXGHUHQGHNDQWDJHDIVWHGLIRUPDIHWVUOLJWWLOUHWWHODJWVHPHVWHUL IRUPDIHQJHOVNVSURJHGHXGGDQQHOVHVXGEXGRJLIRUPDILQWHUQDWLRQDOWVDPDUEHMGHLUHODWLRQWLO IDNXOWHWHWVSGDJRJLVNHPRGHO'HVWXGHUHQGHHYDOXHULQJVJUXSSHQPGWHSnEHVJHWKDYGHGRJ LNNHHUIDULQJHUPHGXGODQGVRSKROG,IRUKROGWLOIDNXOWHWHWVLQWHUQDWLRQDOHNRQWDNWIODGHYXUGHUHV GHWSRVLWLYWDWIDNXOWHWHWPHGYLUNHUL(&,8(XURSHDQ&RQVRUWLXPRI,QQRYDWLYH8QLYHUVLWLHV'HWHU LPLGOHUWLGLNNHNODUWIRUHYDOXHULQJVJUXSSHQKYLONHNRQNUHWHIRUGHOHVDPDUEHMGHWKDUKDIWVRP DGVNLOOHUVLJIUDDQGUHW\SHUDIVDPDUEHMGVDIWDOHU7DJHWGHDPELWLVHPnOVWQLQJHULEHWUDJWQLQJ NXQQHGHWKDYHYUHWIRUYHQWHWDWVDPDUEHMGHWKDYGHHQPHUHPDUNDQWUROOHLEHVNULYHOVHQDI IDNXOWHWHWVDNWLYLWHWHU2PYHQGWHUGHUIRUVWnHOVHIRUDWQ\HW\SHUDILQWHUQDWLRQDOWVDPDUEHMGHNDQ YUHYDQVNHOLJHDWHWDEOHUH
,QGHQIRUHIWHURJYLGHUHXGGDQQHOVHVRPUnGHWYLGQHULNNHPLQGVWZRUNEDVHGOHDUQLQJRJOLYVODQJ XGGDQQHOVHVXJHRPHQLQQRYDWLYWLOJDQJWLORPUnGHW8GIUDRPIDQJHWDIIDNXOWHWHWVDNWLYLWHWHU YXUGHUHUHYDOXHULQJVJUXSSHQDWHIWHURJYLGHUHXGGDQQHOVHHUHWRPUnGHVRPIDNXOWHWHWNXQQH VDWVHHQGQXPHUHSn/LYVODQJXGGDQQHOVHHUXGIUDGHOWDJHUDQWDOOHWRJGHOWDJHUYXUGHULQJHUQHDW GPPHHQVXFFHVPHQVGHWHQGQXHUIRUWLGOLJWDWVLJHQRJHWRPKYRUYLGWZRUNEDVHGOHDUQLQJYLO EOLYHHQVXFFHVEODQGWYLUNVRPKHGHUQH
)RUVNQLQJ
)DNXOWHWHWKDULVHOYHYDOXHULQJHQYDOJWDWUHGHJUHIRUVLQHIRUVNQLQJVPVVLJHUHVXOWDWHURJ IRUPLGOLQJHQDIGHPXQGHURYHUVNULIWHUQHIRUVNHUQHVGHOWDJHOVHLGHWJOREDOHIRUVNHUVDPIXQGV YLGHQVNDEHOVHIRUVNQLQJVIRUPLGOLQJRJRSI\OGHOVHDIYLUNVRPKHGHUQHVYLGHQVEHKRY6nOHGHVKDU IDNXOWHWHWYDOJWDWVRQGUHPHOOHPIRUVNQLQJRJIRUVNQLQJVIRUPLGOLQJ1HGHQIRUUHGHJUHVGHUIRU IDNXOWHWHWVIRUVNQLQJVPVVLJHUHVXOWDWHURJIRUPLGOLQJHQDIGHPXQGHUHW
D )DNXOWHWHWVIRUVNQLQJVPLOMHU
)DNXOWHWHWVIRUVNQLQJHURUJDQLVHUHWLLQVWLWXWWHURJHQUNNHFHQWUH2UJDQLVHULQJHQDI IRUVNQLQJHQKDUIRUDQGUHWVLJVLGHQRSUHWWHOVHQLPHGXGVSDOWQLQJHQDIQ\HLQVWLWXWWHU IO\WQLQJDIIDJJUXSSHUPHOOHPLQVWLWXWWHURJRSUHWWHOVHQDIIRUVNHOOLJHFHQWUH,VHOYHYDOXHULQJHQ EHVNULYHVUHODWLRQHQPHOOHPLQVWLWXWWHURJFHQWUHVRPHQPDWUL[
,UHODWLRQWLOIRUVNQLQJHQIUHPKYHUIDNXOWHWHWLVHOYHYDOXHULQJHQRSUHWWHOVHQDIHQUNNHFHQWUH VRPLIRUVNHOOLJWRPIDQJKDURSE\JJHWNRPSHWHQFHUSnUHJLRQDOWQDWLRQDOWRJLQWHUQDWLRQDOW QLYHDX&HQWUHQHIUHPKYHVRJVnVRPHNVHPSOHUSnVDPDUEHMGHWPHGHUKYHUYVOLYHWRJL IRUELQGHOVHPHGIRUPLGOLQJDIIDNXOWHWHWVIRUVNQLQJ,VHOYHYDOXHULQJHQQYQHVRJSUVHQWHUHV IOJHQGHFHQWUH&HQWHUIRU6DQVH0RWRULVN,QWHUDNWLRQ&HQWHUIRU3HUVRQ.RPPXQLNDWLRQ&HQWHU IRU,QGXVWULHO3URGXNWLRQ&HQWHUIRU0HQQHVNHWV/\GRSIDWWHOVH&HQWHUIRU-RUGEUXJVWHNQRORJL
&HQWHUIRU0DWHULDOHWHNQRORJLRJ1RXKDX]
1HGHQIRUSUVHQWHUHVGHLQVWLWXWWHURJGHWFHQWHUVRPKDUGHOWDJHWLGHQGHFHQWUDOHGHODI VHOYHYDOXHULQJVSURFHVVHQVDPW&HQWHUIRU3HUVRQ.RPPXQLNDWLRQ
,QVWLWXWIRU'DWDORJL
,QVWLWXWWHWHURUJDQLVHUHWLILUHIRUVNQLQJVHQKHGHUPHGIOJHQGHIRUVNQLQJVRPUnGHU'DWDEDVHRJ SURJUDPPHULQJVWHNQRORJLEHVOXWQLQJVVWWWHV\VWHPHULQIRUPDWLRQVV\VWHPHURJGLVWULEXHUHGH V\VWHPHURJVHPDQWLN9HG,QVWLWXWIRU'DWDORJLHUGHUDQVDWSURIHVVRUHURJOHNWRUHU DGMXQNWHURJSKGVWXGHUHQGH
,QVWLWXWWHWJHQQHPIUHUIRUVNQLQJVHYDOXHULQJHUPHGLQGGUDJHOVHDIXGHIUDNRPPHQGHHNVSHUWHU ,GHQVHQHVWHIRUVNQLQJVHYDOXHULQJIUDMXQLYXUGHUHVLQVWLWXWWHWPHJHWSRVLWLYW
,QVWLWXWWHWHWDEOHUHGHLFHQWHUHW1RXKDX]XGIUDHWQVNHRPDWHWDEOHUHHWWWWHUH VDPDUEHMGHPHOOHPLWYLUNVRPKHGHULUHJLRQHQRJLQVWLWXWWHWVIRUVNQLQJVHQKHGHU1RXKDX]HUHW IRUXPIRUGLVNXVVLRQVDPDUEHMGHRJNRQWDNWPHOOHPIRUVNHUHVWXGHUHQGHRJLWPHGDUEHMGHUHL 1RUGM\OODQG
,QVWLWXWIRU9DQG-RUGRJ0LOMWHNQLN
,QVWLWXWWHWHURUJDQLVHUHWLWUHHQKHGHU/DERUDWRULHWIRU*HRWHNQLNRJ)XQGHULQJ/DERUDWRULHWIRU +\GUDXOLNRJ+DYQHE\JQLQJVDPW6WUXNWXUHO'\QDPLN9HG,QVWLWXWIRU9DQG-RUGRJ0LOMWHNQLNHU GHUDQVDWSURIHVVRUHURJOHNWRUHUDGMXQNWHURJSKGVWXGHUHQGH
,QVWLWXWWHWKDUXQGHUJnHWIOHUHRUJDQLVDWRULVNHIRUDQGULQJHU9HG$$8ªVRSUHWWHOVHEOHYLQVWLWXWWHW HWDEOHUHWPHGIRNXVSnYDQGE\JQLQJJHRWHNQLNRJWHNQLVNK\JLHMQH6LGHQHUGHUEOHYHWXGYLNOHW HWPLOMLQGHQIRUELRWHNQLNGHUQXKDUGDQQHWVLWHJHWLQVWLWXW)RURPWUHQWHWnUVLGHQYDOJWH DIGHOLQJHQIRUPLOMWHNQLNDWRYHUJnWLO,QVWLWXWIRU%LRWHNQLNXGIUDHQYXUGHULQJDIHWVWUUHIDJOLJW IOOHVVNDE,QVWLWXWWHWHUWLOJHQJOGEOHYHWXGYLGHWPHGRPUnGHWVWUXNWXUHOG\QDPLN,
VHOYHYDOXHULQJHQIUHPKYHULQVWLWXWWHWYNVWGHOWDJHOVHYHGLQWHUQDWLRQDOHNRQIHUHQFHURJHQVWRU VDPDUEHMGVIODGHVRPHNVHPSOHUSnUHVXOWDWHU(QGYLGHUHIUHPKYHVLQVWLWXWWHWV
ODERUDWRULHIDFLOLWHWHUVRPHWVXFFHVIXOGWVDWVQLQJVRPUnGH
,QVWLWXWIRU(QHUJLWHNQLN
,QVWLWXWWHWHURUJDQLVHUHWLIOJHQGHVHNWLRQHU(OHNWULVNHIRUV\QLQJVV\VWHPHUWHUPLVNH HQHUJLV\VWHPHUHIIHNWHOHNWURQLVNHV\VWHPHUIOXLGPHNDQLNRJIRUEUQGLQJVWHNQLNHOHNWULVNH PDVNLQHURJK\GUDXOLVNHRJPHNDWURQLVNHV\VWHPHU9HG,QVWLWXWIRU(QHUJLWHNQLNHUGHUDQVDW SURIHVVRUHURJOHNWRUHUDGMXQNWHURJSKGVWXGHUHQGH
,QVWLWXWWHWKDULJHQQHPGHVHQHVWHWLnUKDIWHQPnOVWQLQJRPDWXGE\JJHVLQIRUVNQLQJVPVVLJH DNWLYLWHW,QVWLWXWWHWIUHPIUHUGHWVRPHQVXFFHVDWGHWHUO\NNHGHVDWJHGHNYDQWLWDWLYHUHVXOWDWHU LIRUKROGWLOLQGLNDWRUHUVRPI[DQWDODUWLNOHUSDWHQWVDJHUSKGSURMHNWHURJHNVWHUQWILQDQVLHUHGH IRUVNQLQJVDNWLYLWHWHU'HVXGHQIUHPKYHVWUHVWUDWHJLVNHVDWVQLQJHUPHGHNVWHUQHSDUWHU
• 'DQIRVV3URIHVVRU3URJUDPPHWGHUKDUWLOIRUPnODWVW\UNHHIIHNWHOHNWURQLNRPUnGHW3URMHNWHW JHQQHPIUHVPHGVWWWHIUD'DQIRVV'ULYHVVWDWHQRJ$$8
• ,VDPDUEHMGHPHG6DXHU'DQIRVV$6HUGHUWDJHWHWLQLWLDWLYLQGHQIRU(OHNWUR0HNDQLVN6\VWHP 'HVLJQGHUVNDOXGYLNOHRJLQGNUHHQQ\XGGDQQHOVHLQGHQIRUPHFKDWURQLFRPUnGHWVDPWVW\UNH IRUVNQLQJHQLQGHQIRUVDPPHRPUnGH
• (OHNWULVN'HVLJQRJ6W\ULQJHUHWSURJUDPGHUHUHWDEOHUHWLVWUDWHJLVNDOOLDQFHPHG5LVYHGUUHQGH HOHNWULVNGHVLJQRJVW\ULQJDIYLQGPOOHU
&HQWHUIRU6DQVH0RWRULVN,QWHUDNWLRQ
&HQWHUHWEOHYRSUHWWHWLVRPHWJUXQGIRUVNQLQJVFHQWHU(QVWRUGHODIFHQWHUHWVEHYLOOLQJWLO IRUVNQLQJRJIRUVNHUVNROHILQDQVLHUHVDI'DQPDUNV*UXQGIRUVNQLQJVIRQG)LQDQVLHULQJHQOEHU LQGWLO&HQWHUIRU6DQVH0RWRULVN,QWHUDNWLRQKDUWLONQ\WWHWSURIHVVRUHURJOHNWRUHU DGMXQNWRJSKGVWLSHQGLDWHU
)RUPnOHWPHGFHQWHUHWHUDWHWDEOHUHRJEHGULYHIRUVNQLQJRJGULYHHQIRUVNHUVNROHDIKM LQWHUQDWLRQDONYDOLWHW0HUHNRQNUHWVNDOFHQWHUHWXQGHUVJHEDVDOHRJNOLQLVNHDVSHNWHUDI PHQQHVNHUVVDQVHPRWRULVNHLQWHUDNWLRQRJXGYLNOHPHWRGHUWLODWJHQHWDEOHUHGHODJWHVDQVH PRWRULVNHIXQNWLRQHU&HQWHUHWVNRQWDNWWLOGHWUHJLRQDOHHUKYHUYVOLYEHVWnUDIXIRUPHOOH
SURMHNWVDPDUEHMGHURJGHOWDJHOVHLQHWYUNVDNWLYLWHWHU'HVXGHQLQGJnUFHQWHUHW
VDPDUEHMGVDIWDOHUPHGYLUNVRPKHGHURPnUHWVRPRIWHVWRPRSUHWWHOVHDISKGIRUOERJ DIWDOHURPUHNYLUHUHWDUEHMGHRPnUHW
&HQWHUIRU3HUVRQ.RPPXQLNDWLRQ&3.
&HQWHUIRU3HUVRQ.RPPXQLNDWLRQHURUJDQLVDWRULVNWLONQ\WWHW,QVWLWXWIRU(OHNWURQLVNH6\VWHPHU
$IGHOLQJHQIRU.RPPXQLNDWLRQVWHNQRORJL&HQWHUHWVIRUVNQLQJVRPUnGHRPIDWWHUEUHGWGHILQHUHW WHRULWHNQLNVDPWGHVLJQRJNRQVWUXNWLRQDIXGVW\UYHGRYHUIUVHORJEHKDQGOLQJDIVLJQDOHURJ LQIRUPDWLRQLQGHQIRUWUnGOVNRPPXQLNDWLRQ&HQWHUIRU3HUVRQ.RPPXQLNDWLRQKDUWLONQ\WWHW SURIHVVRUHURJOHNWRUHUDGMXQNWHURJSKGVWLSHQGLDWHU
,VHOYHYDOXHULQJHQDQIUHVGHWDW&HQWHUIRU3HUVRQ.RPPXQLNDWLRQKDUHQVWUN
HUKYHUYVWLONQ\WQLQJRJHQGYLGHUHKDUVSLOOHWHQFHQWUDOUROOHIRUSODFHULQJHQDIIRUVNQLQJVRJ XGYLNOLQJVDIGHOLQJHULQGHQIRUWHOHNRPPXQLNDWLRQL1RUGM\OODQG'HWIUHPKYHVHQGYLGHUHDW FHQWHUHWVDPDUEHMGHUPHGHWVWRUWDQWDOYLUNVRPKHGHURJDWIDNXOWHWHWXGGDQQHUHWVWRUWDQWDO NDQGLGDWHULQGHQIRURPUnGHW
&HQWHUHWKDUQHGVDWHQLQWHUQDWLRQDOUnGJLYQLQJVNRPLWH,QWHUQDWLRQDO$GYLVRU\%RDUGGHUIOJHU FHQWHUHWVDNWLYLWHWHU,UnGJLYQLQJVNRPLWHHQVXGWDOHOVHIUDYXUGHUHVDNWLYLWHWHUQHYHGFHQWHUHW SRVLWLYWRJEODQGWDQGHWIUHPKYHVGHQRPIDWWHQGHWLGVVNULIWSXEOLFHULQJRJGHOWDJHOVHYHG NRQIHUHQFHUVDPWVWDQGDUGLVHULQJVDNWLYLWHWHURJIRUVNQLQJVSURMHNWHUVDPPHQPHGHUKYHUYVOLYHW
&HQWHUHWVDNWLYLWHWHUILQDQVLHUHVDIIDNXOWHWHWRJ6WDWHQV7HNQLVN9LGHQVNDEHOLJH)RUVNQLQJVUnG3n EHVJHWRJLELODJVPDWHULDOHWJUHVGHURSPUNVRPSnDWUDPPHEHYLOOLQJHQIUDIRUVNQLQJVUnGHW VQDUWXGOEHURJDWFHQWHUHWGHUIRUKDUEHKRYIRUDWILQGHDOWHUQDWLYHILQDQVLHULQJVNLOGHUIRUDW RSUHWKROGHVLQHDNWLYLWHWHU
E )RUVNQLQJVNYDOLWHW
,VHOYHYDOXHULQJHQDQIUHVGHWDWNYDOLWHWHQDIIDNXOWHWHWVIRUVNQLQJGHOVYXUGHUHVYHGDW PHGDUEHMGHUQHLNRQNXUUHQFHPHGDQGUHIRUVNHUHDQVJHURPRJInUWLOGHOWUHVVRXUFHUIUD IRUVNQLQJVUnGRJSULYDWHIRQGHGHOVYXUGHUHVYHGJHQQHPIUHOVHDIIRUVNQLQJVHYDOXHULQJHUDI LQVWLWXWWHUQHVIRUVNQLQJKYHUWIHPWHnUVHRJVnDIVQLWRPNYDOLWHWVVLNULQJ
(QGYLGHUHYXUGHUHUIDNXOWHWHWPHGXGJDQJVSXQNWLIRUVNQLQJVEHUHWQLQJHQRPIDQJHWRJNYDOLWHWHQ DIDOOHSURIHVVRUHUVRJOHNWRUHUVIRUVNQLQJLKHQKROGWLOIOJHQGHWUHNDWHJRULHUVUOLJW
IRUVNQLQJVDNWLYHQRUPDOWIRUVNQLQJVDNWLYHRJVUOLJWXQGHUYLVQLQJVDNWLYH)DNXOWHWHWGHILQHUHU IRUVNQLQJVRPHQVRFLDODNWLYLWHWLPRGVWQLQJWLOIDJOLJXGYLNOLQJKYLONHWPHGIUHUDWIRUVNQLQJHQV
UHVXOWDWHUSRVLWLYHVRPQHJDWLYHVNDOSXEOLFHUHV'HWWHEHW\GHUDWHQYLJWLJLQGLNDWRUIRU YXUGHULQJHQDINYDOLWHWHQRJRPIDQJHWDIGHHQNHOWHIRUVNHUHVIRUVNQLQJHUDQWDOOHWDI
SXEOLNDWLRQHULLQWHUQDWLRQDOWDQHUNHQGWHIDJWLGVVNULIWHULQGOJSnLQWHUQDWLRQDOHIDJNRQJUHVVHU DIKDQGOLQJHURJOLJQHQGH(IWHUDWIDNXOWHWHWHUEHJ\QGWDWWLOGHOHLQVWLWXWWHUQHIRUVNQLQJVnUVYUN HIWHUHQYXUGHULQJDINYDOLWHWHQRJRPIDQJHWDIHQNHOWIRUVNHUHVDUEHMGHHUDQWDOOHWDISXEOLFHUHGH YLGHQVNDEHOLJHDUWLNOHURJNRQIHUHQFHLQGOJVWHJHW,VDPPHSHULRGHKDUGHUYUHWHWIDOGLLNNH SXEOLFHUHGHZRUNLQJSDSHUVPY
7DEHO
9LGHQVNDEHOLJHDUWLNOHURJZRUNLQJSDSHUVPY
QGULQJ
9LGHQVNDEHOLJHDUWLNOHURJNRQIHUHQFHLQGOJ
:RUNLQJSDSHUVPY
$QWDOYLGHQVNDEHOLJHPHGDUEHMGHUHJHQQHPVQLW
.LOGH6HOYHYDOXHULQJHQVGHORJIDNXOWHWVNRQWRUHW
9HGIDNXOWHWHWHUGHULSHULRGHQEOHYHWWLOGHOWGRNWRUJUDGHU
7DEHO
7LOGHOWHGRNWRUJUDGHU
7LOGHOWHGRNWRUJUDGHU
.LOGH6HOYHYDOXHULQJHQVGHO F )RUVNHUXGGDQQHOVH
,RSUHWWHGHIDNXOWHWHWHQIRUVNHUVNROHVRPOHGHVDIHQEHVW\UHOVHSnILUHSHUVRQHU
)RUVNHUVNROHQVSULPUHPnOHUDWVNDEHGHEHGVWPXOLJHUDPPHUIRUIRUVNHUXGGDQQHOVHQRJVLNUH DWIRUVNHUXGGDQQHOVHQHUSnLQWHUQDWLRQDOWQLYHDX'HUXGE\GHVWLSKGSURJUDPPHUXQGHU )RUVNHUVNROHQ
$IVHOYHYDOXHULQJHQIUHPJnUGHWDWHQVWRUGHODIIDNXOWHWHWVSKGVWXGHUHQGHHUILQDQVLHUHWDI HNVWHUQHEHYLOOLQJHU2YHUILQDQVORYHQWLOGHOHVIDNXOWHWHWSKGVWLSHQGLHUKYLONHWJLYHUJUXQGODJ IRUDWDQVWWHVWLSHQGLDWHURPnUHW6RPIOJHDIHNVWHUQHEHYLOOLQJHURSVOnUIDNXOWHWHWKYHUWnU
JDQJHVnPDQJHVWLSHQGLHU6RPGHWIUHPJnUDIQHGHQVWnHQGHWDEHOHURSWDJHWDISKG VWXGHUHQGHVWHJHWLSHULRGHQVHOYRPGHUHUEHW\GHOLJHXGVYLQJPHOOHPGHIRUVNHOOLJHnU, SHULRGHQKDUGHUHQGYLGHUHYUHWHQVWLJQLQJSnLDQWDOOHWDIWLOGHOWHSKG JUDGHU
7DEHO
7LOGHOWHSKGJUDGHURJRSWDJQHSKGVWXGHUHQGH
3KGJUDGHU
2SWDJQHSKGVWXGHUHQGH
.LOGH6HOYHYDOXHULQJHQVGHO)RUVNHUXGGDQQHOVHVUnGHWVVHNUHWDULDW)DNXOWHWVNRQWRUHW 1RWHªªLQGLNHUHULQJHQRSO\VQLQJHU
,)RUVNHUVNROHQVELGUDJWLOVHOYHYDOXHULQJHQIUHPKYHVEODQGWDQGHWIOJHQGHUHVXOWDWHUHQ JHQQHPVQLWOLJJHQQHPIUHOVHVWLGSnnUDQWDOOHWDIXGEXGWHSKGNXUVHUHUVWDGLJWVWLJHQGH RJDIDOOHSKGVWXGHUHQGHHUSnOQJHUHYDUHQGHPHUHHQGWUHPnQHGHUXGODQGVRSKROG 'HVXGHQIUHPKYHVGHWLVHOYHYDOXHULQJHQDW)RUVNHUVNROHQKDUVLNUHWDWIRUVNHUXGGDQQHOVHQWDJHV DOYRUOLJWRJVnUHVVRXUFHPVVLJWYHGIDNXOWHWHWRJDWXGGDQQHOVHQNYDOLWHWVVLNUHV
,IOJHGHQUHJLVWHUEDVHUHGHXQGHUVJHOVHILQGHUPHUHHQGKDOYGHOHQDIIDNXOWHWHWVSKGHUH DQVWWHOVHLGHWSULYDWHHUKYHUYVOLY
G 6DPDUEHMGHPHGYLUNVRPKHGHURJRUJDQLVDWLRQHU
$IVHOYHYDOXHULQJHQIUHPJnUGHWDWIDNXOWHWHWQVNHUDWWLOVN\QGHWLOVDPDUEHMGHPHGDOOHUHOHYDQWH SDUWHUXGHQIRUXQLYHUVLWHWHWVRPKDUHQLQWHUHVVHLIDNXOWHWHWVYLUNVRPKHG)DNXOWHWHWQVNHU HQGYLGHUHDWPHGYLUNHWLOHQIRUPDOLVHULQJDIVDPDUEHMGHWPHGXQLYHUVLWHWHUPYQnUGHUHUWDOHRP HWVDPDUEHMGHRYHUOQJHUHWLG
,RSUHWWHGHXQLYHUVLWHWHWHWQHWYUNVFHQWHUPHGKHQEOLNSnDWV\VWHPDWLVHUHRJVW\UNH XQLYHUVLWHWHWVIRUPLGOLQJVUHODWLRQHUPHGGHWRPJLYHQGHVDPIXQG1HWYUNVFHQWHUHWVNDOIXQJHUH VRPHQV\QOLJRJXIRUPHOLQGJDQJVSRUWDOWLOVDPDUEHMGVUHODWLRQHUPHGXQLYHUVLWHWHW6LGHQ HWDEOHULQJHQHUGHUVNHWHQEHW\GHOLJYNVWLDQWDOOHWDIIRUPDOLVHUHGHQHWYUNRJ
QHWYUNVPHGOHPPHUMIQHGHQVWnHQGHWDEHO
7DEHO
1HWYUNVDNWLYLWHWHUKHOHXQLYHUVLWHWHW
$QWDOQHWYUNVPHGOHPPHU
.LOGH6HOYHYDOXHULQJHQVGHO
)RUDWNQ\WWHXQLYHUVLWHWHWRJHUKYHUYVOLYHWWWWHUHVDPPHQHUXQLYHUVLWHWHWLVDPDUEHMGHPHG 1RUGM\OODQGV$PWLJDQJPHGDWXGYLNOHGDWDEDVHQ9LGHQVEDVH1RUGM\OODQG'DWDEDVHQVNDO NRRUGLQHUHRJIRUPLGOHQXYUHQGHIRUVNQLQJVSURMHNWHURJIRUVNQLQJVUHVXOWDWHUSn$DOERUJ 8QLYHUVLWHWWLOJDYQIRUEnGH$DOERUJ8QLYHUVLWHWRJHUKYHUYVOLYHWL1RUGM\OODQG
$QWDOOHWDIVDPDUEHMGVDIWDOHUPHOOHPIDNXOWHWHWVIRUVNHUHIRUVNQLQJVPLOMHURJSULYDWH YLUNVRPKHGHUHUVWHJHWPHGLSHULRGHQRJGHQJHQQHPVQLWOLJHLQGWJWSU VDPDUEHMGVDIWDOHHUVWHJHWFLUNDNUMIQHGHQVWnHQGHWDEHO
7DEHO
6DPDUEHMGVDIWDOHUPHGSULYDWHYLUNVRPKHGHU
$QWDOVDPDUEHMGVDIWDOHU
*HQQHPVQLWOLJLQGWJWSUDIWDOHNU
6DPOHWLQGWJWNU
.LOGH6HOYHYDOXHULQJHQVGHO
129,LQQRYDWLRQ$6QYQHVVRPHQYLJWLJNDWDO\VDWRUIRUUHVXOWDWHULUHODWLRQWLO
IRUVNQLQJVIRUPLGOLQJLIRUPDIVSLQRIIYLUNVRPKHGHU129,GHUEOHYLQGYLHWLHUHQEODQGLQJ DIIRUVNHUSDUNLQQRYDWLRQVPLOMRJYHQWXUHLQYHVWRU.RQNUHWQYQHVWHOHNRPPXQLNDWLRQVRJ DNXVWLNRPUnGHWVRPHNVHPSOHUKYRUIDNXOWHWHWKDUYUHWHQPHGYLUNHQGHIDNWRULXGYLNOLQJHQDI NRPSHWHQFHRJNRQNXUUHQFHHYQHEODQGWIOHUHUHJLRQDOHRJQDWLRQDOHYLUNVRPKHGHU
$QDO\VHURJYXUGHULQJHU
(YDOXHULQJVJUXSSHQILQGHUDWGHQIDJOLJHRUJDQLVHULQJHUYLJWLJIRUDWIRUVWnIDNXOWHWHWVVXFFHVHUSn IRUVNQLQJVRPUnGHW)DNXOWHWHWVIRUVNQLQJVRUJDQLVHULQJRJGHUPHGGHQIRUPHOOHRSGHOLQJDI
IDJRPUnGHUQHHUSUJHWDIEHW\GHOLJG\QDPLN'HWHURSIDWWHOVHQIUDVnYHOVHOYHYDOXHULQJVRP EHVJDWIDNXOWHWHWVIDJOLJHRUJDQLVHULQJPHGIRUGHONDQIRUVWnVXGIUDHWSURMHNWSHUVSHNWLYGHU LNNHHUXOLJGHQPnGHXQGHUYLVQLQJHQRUJDQLVHUHVSn'HWNRPPHUWLOXGWU\NYHGDWRUJDQLVDWLRQHQ IRUDQGUHVJHQQHPNQRSVN\GQLQJKYRUDIGHOLQJHUI[RSUHWWHVXQGHULQVWLWXWWHUQHIRUHYHQWXHOW VHQHUHDWXGJUHW\QJGHSXQNWHWLHWFHQWHUHOOHURPYHQGWKYRUFHQWUHEOLYHULQGOHMUHWLHWLQVWLWXW 'HQQHG\QDPLNKDULNNHPLQGVWYUHWYLJWLJLHQVLWXDWLRQKYRUIRUVNQLQJVIHOWHUQHRIWHKDUOLJJHW SnJUQVHQDIWUDGLWLRQHOOHIRUVNQLQJVIHOWHU
)DNXOWHWHWVEHVNULYHOVHDIIRUVNQLQJVRUJDQLVHULQJHQWDJHUDIVWLHQPDWUL[IRUVWnHOVHGHUJLYHU EHJUQVHWPXOLJKHGIRUDWIRUVWnGHQG\QDPLNGHUHULRUJDQLVDWLRQHQRYHUWLG'HUIRUYLO HYDOXHULQJVJUXSSHQDQEHIDOHIDNXOWHWHWDWDUEHMGHDNWLYWSnDWRSQnHQJHWIRUVWnHOVHDIGHW VUHJQHYHGIRUVNQLQJVPLOMHWSnIDNXOWHWHWKHUXQGHUKYLONHQPnGHGHWWHKDUPHGYLUNHWWLOGH VXFFHVHUIDNXOWHWHWKDUNXQQHIUHPYLVHSnRPUnGHW
$QEHIDOLQJHQVNDOGHVXGHQVHVLO\VHWDIDWEnGHQnUGHWJOGHUIRUVNQLQJRJQnUGHWJOGHU XGGDQQHOVHKDUIDNXOWHWHWHQWLOJDQJGHUDIYLJHUIUDGHQWUDGLWLRQHOOHXQLYHUVLWUHWLOJDQJ3n XGGDQQHOVHVRPUnGHWHUGHUHWKMWEHYLGVWKHGVQLYHDXRPNULQJKYRULVW\UNHUQHEHVWnU'HQQH EHYLGVWKHGJHQILQGHUHYDOXHULQJVJUXSSHQLNNHSnIRUVNQLQJVRPUnGHW
)DNXOWHWHWVRUJDQLVHULQJDIIRUVNQLQJHQLLQVWLWXWWHUDIGHOLQJHURJFHQWUHV\QHVLNNHDWIOJHHQNODU ORJLN+YHUNHQDIVHOYHYDOXHULQJHQHOOHUDIEHVJHWEOHYGHWNODUWKYDGGHUNRQVWLWXHUHUKHQKROGVYLV HWLQVWLWXWHQDIGHOLQJRJHWFHQWHURJGHUV\QHVDWYUHIRUVNHOOLJHRSIDWWHOVHUSnIDNXOWHWHWDI GHOVKYRUGDQRJKYRUQnUGHURSUHWWHVKHQKROGVYLVFHQWUHRJLQVWLWXWWHUGHOVKYLONHQUHODWLRQGHU VNDOYUHPHOOHPFHQWUHRJLQVWLWXWWHU(YDOXHULQJVJUXSSHQKDUIRUVWnHOVHIRUDW
FHQWHUNRQVWUXNWLRQHUHUYHOHJQHGHWLODWVNDEHHNVWHUQHNVSRQHULQJDIIDNXOWHWHWVIRUVNHOOLJH DNWLYLWHWHU$OOLJHYHOILQGHUHYDOXHULQJVJUXSSHQDWIDNXOWHWHWEUNODUJUHSUPLVVHUQHIRU RUJDQLVHULQJHQDIIRUVNQLQJPYLLQVWLWXWWHUDIGHOLQJHURJFHQWUH
,VHOYHYDOXHULQJHQHUIDNXOWHWHWVRSJUHOVHDIUHVXOWDWHULYLGXGVWUNQLQJRULHQWHUHWPRG
NYDQWLWDWLYHYNVWSDUDPHWUH8GRYHUDWGHUHUWDOHRPVWLJQLQJHUHUGHWLNNHNODUWKYRUGDQGLVVH XGYLNOLQJHUVNDOHYDOXHUHVRJKYDGGHVLJHURPNYDOLWHWHQDIIDNXOWHWHWVIRUVNQLQJ6RPELODJWLO IDNXOWHWHWVVHOYHYDOXHULQJHUGHUYHGODJWHNVHPSOHUSnHNVWHUQHHYDOXHULQJHUDIKHOH
IRUVNQLQJVPLOMHU(YDOXHULQJHUQHGRNXPHQWHUHUDWGHUIRUHJnUIRUVNQLQJSnKMWQLYHDX'HWHU HYDOXHULQJVJUXSSHQVRSIDWWHOVHDWGHWYLOOHKDYHYUHWJLYHQGHIRUHYDOXHULQJHQKYLVIDNXOWHWHWYDU NRPPHWPHGHQV\VWHPDWLVNNYDOLWDWLYYXUGHULQJDIIRUVNQLQJVPLOMHUQHVRPVXSSOHPHQWWLOGHQ NYDQWLWDWLYHEHO\VQLQJ,NNHPLQGVWVHWLO\VHWDIDWIDNXOWHWHWHNVSOLFLWVNULYHULVHOYHYDOXHULQJHQDW GHWRSHUHUHUPHGHQLQGGHOLQJDIIRUVNQLQJHQLILUHNODVVHU:RUOG&ODVV,QWHUQDWLRQDO&ODVV 1DWLRQDO&ODVVRJ5HJLRQDO&ODVV(YDOXHULQJVJUXSSHQEHPUNHUDWGHWIRUHNRPPHUNXQVWLJWDW
EHWHJQHGHQPLQGVWYHOO\NNHGHIRUVNQLQJVRPUHJLRQDOSnHWXQLYHUVLWHWGHUQHWRSKDUKDIWVXFFHV PHGDWVNDEHG\QDPLNPHOOHPGHWUHJLRQDOHRJGHWLQWHUQDWLRQDOHQLYHDX
,IRUOQJHOVHKHUDIHUGHWHYDOXHULQJVJUXSSHQVRSIDWWHOVHDWIDNXOWHWHWVNYDQWLWDWLYWRULHQWHUHGH PnGHDWRSJUHVLQHUHVXOWDWHUSnVWLOOHUIDNXOWHWHWRYHUIRUHWGLOHPPD3nGHQHQHVLGHJUGHQ GHWYDQVNHOLJWDWJLYHHQI\OGHVWJUHQGHEHO\VQLQJDIIDNXOWHWHWVUHVXOWDWHUGHUNXQLEHJUQVHW RPIDQJODGHUVLJEHVNULYHDIHQNRQYHQWLRQHONYDQWLWDWLYWRULHQWHUHWGRNXPHQWDWLRQ3nGHQDQGHQ VLGHHUGHQNRQYHQWLRQHOOHWLOJDQJHWYVHQWOLJELGUDJWLODWNXQQHGRNXPHQWHUHVLQHUHVXOWDWHUL IRUKROGWLORPYHUGHQHQ,GHQIRUELQGHOVHVNDOGHWEHPUNHVVRPSRVLWLYWDWIDNXOWHWHWKDUHW DGPLQLVWUDWLYWV\VWHPGHUNDQJLYHRSO\VQLQJHURPUHVXOWDWRSQnHOVH
(YDOXHULQJVJUXSSHQYLODQEHIDOHIDNXOWHWHWQMHDWRYHUYHMHKYRUGDQGHUNDQRSQnVEDODQFHPHOOHP GHWRSROHULGLOHPPDHWXGHQDWULVLNHUHDWVWWHGHVUOLJHNDUDNWHULVWLNDGHUHUIDNXOWHWHWV VW\UNHSRVLWLRQRYHUVW\U
'HWYXUGHUHVLXGJDQJVSXQNWHWSRVLWLYWDWIDNXOWHWHWJHQQHPGHVLGVWHnUKDUJHWDQWDOOHWDI DUWLNOHUSXEOLFHUHWLWLGVVNULIWHUPHGSHHUUHYLHZOLJHVRPGHWYXUGHUHVSRVLWLYWDWGHWHUO\NNHGHV IDNXOWHWHWDWWLOWUNNHVWLJHQGHHNVWHUQHIRUVNQLQJVPLGOHU(YDOXHULQJVJUXSSHQILQGHUGHW DIJUHQGHDWGHUHUHQRSPUNVRPKHGRPNULQJDWGHQPnGHIDNXOWHWHWRSJUVLQHUHVXOWDWHUSn SnIDNXOWHWVQLYHDXKDUNRQVHNYHQVHUIRUKYRUGDQIRUVNHUQHDJHUHU$IEHVJHWIUHPJLNGHWDW SXEOLFHUHGHDUWLNOHUHUHQSDUDPHWHUVRPGHULQWHUQWSnIDNXOWHWHWOJJHVPHJHWYJWSnRJVRP KDUHQYLVDGIUGVUHJXOHUHQGHHJHQVNDE6RPGHWEOHYVDJWSnHWPGHXQGHULQVWLWXWLRQVEHVJHW VnInUPDQDUWLNOHUQnUPDQWOOHUDUWLNOHU
'HWHUHYDOXHULQJVJUXSSHQVRSIDWWHOVHDWIRNXVHULQJHQSnHNVWHUQSXEOLFHULQJKDUYUHWPHGWLODW VW\UNHRSPUNVRPKHGHQRPNULQJIRUVNQLQJSnIDNXOWHWHW6DPWLGLJHUGHUWYLYORPKYRUYLGWGHQQH VWUDWHJLHUKHQVLJWVPVVLJSnODQJWVLJWIRUGLHWXSULRULWHUHWIRNXVSnHNVWHUQSXEOLFHULQJNDQ PHGIUHHQULVLNRIRUIDJOLJXGW\QGLQJSnEHNRVWQLQJDIGHWVUHJQHYHGIDNXOWHWHWVIRUVNQLQJI[
GHQSURMHNWRULHQWHUHGHRUJDQLVHULQJVIRUPRJRULHQWHULQJHQPRGHUKYHUYVOLYHW
)RUVNHUVNROHQRSIDWWHVVRPHWSRVLWLYWLQLWLDWLYGHUNDQPHGYLUNHWLODWVWWHIRNXVSn
IRUVNHUXGGDQQHOVHQRJVLNUHDWGHQWDJHVDOYRUOLJWSnLQVWLWXWWHUQH6DPWLGLJHUGHWLQGWU\NNHWIUD LQVWLWXWLRQVEHVJHWRJIUD)RUVNHUVNROHQVELGUDJWLOVHOYHYDOXHULQJHQDW)RUVNHUVNROHQHQGQXLNNH KDUIXQGHWVLQHQGHOLJHIRUPSnDOOHRPUnGHU%ODQGWDQGHWIUHPVWnUIDNXOWHWHWVXGI\OGHOVHDIGH IRUPHOOHUDPPHUIRUIRUVNHUXGGDQQHOVHQPHGHQGHOLJVWXGLHSODQRJKDOYnUVHYDOXHULQJHUVRPPHUH EXUHDXNUDWLVNHHQGKYDGGHUHUKHQVLJWVPVVLJW6DPOHWKDUHYDOXHULQJVJUXSSHQLNNHInHWLQGWU\N DIDW)RUVNHUVNROHQKDUUHDOLVHUHWGHQWLOWQNWHSRVLWLRQLIRUKROGWLOIRUVNHUXGGDQQHOVHQSn
IDNXOWHWHW
'HWHUHYDOXHULQJVJUXSSHQVRSIDWWHOVHDWIDNXOWHWHWKDUWDJHWYLGHQVRYHUIUVHOWLOHUKYHUYVOLYHW DOYRUOLJW1HWYUNVFHQWHUHWRJGHPDQJHVDPDUEHMGVSURMHNWHUHUHNVHPSOHUSnGHWWH)DNXOWHWHW KDUHWDEOHUHWHQEUHGNRQWDNWIODGHWLOYLUNVRPKHGHURJDQGUHRUJDQLVDWLRQHU'HWHURSIDWWHOVHQDW GHUHUEHW\GHOLJHIRUVNHOOHSnGHIRUVNHOOLJHRPUnGHU,QGHQIRUWHOHNRPPXQLNDWLRQVRPUnGHWHUGHW I[HYDOXHULQJVJUXSSHQVRSIDWWHOVHDWGHUHUVNDEWQHWYUNGHUHUYRNVHWIUHPRYHUHQOQJHUH SHULRGH3nDQGUHRPUnGHUHUGHUIUVWIRUQ\OLJWDJHWLQLWLDWLYHUGHUVNDOVW\UNHVDPDUEHMGH'HW JOGHUI[1RXKDX]GHUHUHWFHQWHUHWDEOHUHWPHGGHWIRUPnODWVW\UNHVDPDUEHMGHWPHOOHP ,QVWLWXWIRU'DWDORJLRJHUKYHUYVOLYHW
'HWHUHYDOXHULQJVJUXSSHQVRSIDWWHOVHDWGHQ\HRPUnGHUGHUHUKMHPWDJHWEODQGWDQGHW
VWDPFHOOHIRUVNQLQJVPLOMHWRJHQGHODIGHWELRWHNQLVNHPLOMLNNHKDUGHQVDPPHJUDGDINRQWDNW WLOGHWORNDOHRJUHJLRQDOHHUKYHUYVOLY
)DNXOWHWHWRSIRUGUHVWLODWXQGHUVWWWHDWHWVnGDQWNRQWDNWQHWHWDEOHUHVGDHUKYHUYVNRQWDNWHQHUHW YUGLIXOGWNHQGHWHJQIRUIDNXOWHWHWVRPGHWIRUPRGHQWOLJYLONUYHHQDNWLYLQGVDWVDWRSUHWKROGHL WDNWPHGDWIDNXOWHWHWYRNVHURJWLOIUHVIOHUHRPUnGHU6DPWLGLJNDQHUKYHUYVNRQWDNWHQEOLYH FHQWUDOQnUGHQIDJOLJHLQGVDWVVNDOVLNUHHQEUHG\JWLJXGGDQQHOVHVDNWLYLWHW
9LGHQVRYHUIUVHOIUDYLUNVRPKHGHUWLOXQLYHUVLWHWHWVSLOOHUHQEHVNHGHQUROOHLIDNXOWHWHWV
VHOYHYDOXHULQJ3nLQVWLWXWLRQVEHVJHWJDYIDNXOWHWHWXGWU\NIRUDWGHUHURSPUNVRPKHGRPNULQJ GHWWH'HWEOHYGRJVDPWLGLJSnSHJHWDIHQGHODIGHYLUNVRPKHGHUIDNXOWHWHWVDPDUEHMGHUPHG LNNHKDUHJHQIRUVNQLQJVSURGXNWLRQ'HWHURSIDWWHOVHQDWIDNXOWHWHWPHGIRUGHONXQQHLQGWQNH YLGHQVRYHUIUVHOIUDYLUNVRPKHGHUWLOIDNXOWHWHWPHUHDNWLYWLVLQRPYHUGHQVEHVNULYHOVH
,IRUKROGWLOGHWLQWHUQDWLRQDOHSHUVSHNWLYILQGHUHYDOXHULQJVJUXSSHQDWHWYLJWLJWDVSHNWYHG LQWHUQDWLRQDOLVHULQJHUDWGHUEHGULYHVIRUVNQLQJSnHWQLYHDXVnLQWHUQDWLRQDOHNROOHJDHUEOLYHU LQWHUHVVHUHWLVDPDUEHMGH
6WUXNWXUHURJSURFHVVHU
,GHWWHNDSLWHOEHKDQGOHVGHVWUXNWXUHURJSURFHVVHUGHUXGJUJUXQGODJHWIRUGHUHVXOWDWHU IDNXOWHWHWKDURSQnHW.DSLWOHWRPKDQGOHUVnDWVLJHGHWDSSDUDWIDNXOWHWHWDQYHQGHUIRUDWRSQn VLQHUHVXOWDWHU.DSLWOHWHURSGHOWLILUHXQGHUDIVQLW'HWIUVWHIRNXVHUHUSnKYRUGDQIDNXOWHWHW IRUPXOHUHUPnOVWQLQJHUIRUVLQYLUNVRPKHGPnOVWQLQJHURJVWUDWHJL'HWDQGHWEHKDQGOHUGHQ RUJDQLVDWLRQVVWUXNWXUGHUHUSnIDNXOWHWHW'HWWUHGMHRPKDQGOHUGHQOHGHOVHVPRGHOGHUHUYDOJWSn IDNXOWHWHW7LOVLGVWEHKDQGOHVGHNYDOLWHWVVLNULQJVPHNDQLVPHUGHUHUHWDEOHUHWWLODWEHO\VHRJ XGYLNOHNYDOLWHWHQDIIDNXOWHWHWVXGGDQQHOVHRJIRUVNQLQJ
'HILUHDIVQLWHURSGHOWVnOHGHVDWIDNXOWHWVIUHPVWLOOLQJIUVWJHQJLYHVHIWHUIXOJWDI HYDOXHULQJVJUXSSHQVDQDO\VHURJYXUGHULQJHU