'HVLJQ
(YDOXHULQJDIGHVLJQXGGDQQHOVHUQHXQGHU.XOWXUPLQLVWHULHW
'$10$5.6 (9$/8(5,1*6,167,787
'HVLJQ
'DQPDUNV (YDOXHULQJVLQVWLWXW 7U\NWKRV9HVWHU.RSL
(IWHUWU\NPHG NLOGHDQJLYHOVHHUWLOODGW
%HVWLOOHVKRV
6WDWHQV,QIRUPDWLRQ 3XEOLNDWLRQVDIGHOLQJHQ 3RVWERNV .EHQKDYQ.
7 )
(VS#VLGN +ZZZVLGN
.ULQNOPRPV
,6%1
,QGKROG
)RURUG
,QGOHGQLQJ
)RUPnO
(YDOXHULQJVJUXSSHQRJSURMHNWOHGHOVH
3UVHQWDWLRQDIXGGDQQHOVHVRPUnGHW
'DQPDUNV'HVLJQVNROH
'HVLJQVNROHQL.ROGLQJ
$UNLWHNWVNROHQL$DUKXV
.XQVWDNDGHPLHWV$UNLWHNWVNROH
*ODV .HUDPLNVNROHQSn%RUQKROP
'RNXPHQWDWLRQVPDWHULDOHW
5DSSRUWHQVRSE\JQLQJ
5HVXPp
$QDO\VHRJDQEHIDOLQJHU
(YDOXHULQJVRJVHOYHYDOXHULQJVSURFHVVHQ
'HVLJQEHJUHEHW
)DJOLJSURILOYLVLRQHURJSULRULWHULQJHU
'DQPDUNV'HVLJQVNROH
'HVLJQVNROHQL.ROGLQJ
$UNLWHNWVNROHQL$DUKXV
.XQVWDNDGHPLHWV$UNLWHNWVNROH
*ODV .HUDPLNVNROHQSn%RUQKROP
*HQHUHOOH DQEHIDOLQJHU YHGUUHQGH IDJOLJ SURILO YLVLRQHU RJ
SULRULWHULQJHU
$UEHMGVGHOLQJRJVDPDUEHMGHPHOOHPGHVLJQXGGDQQHOVHUQH
)RUVNQLQJRJIRUVNQLQJVSRWHQWLDOHW
)RUVNQLQJVEHJUHEHW
'DQPDUNV'HVLJQVNROH
'HVLJQVNROHQL.ROGLQJ
$UNLWHNWVNROHQL$DUKXV
.XQVWDNDGHPLHWV$UNLWHNWVNROH
*ODV .HUDPLNVNROHQSn%RUQKROP
*HQHUHOOH YXUGHULQJHU RJ DQEHIDOLQJHU YHGU IRUVNQLQJ Sn GHVLJQ
XGGDQQHOVHUQH
7LWXODWXU
0nOVWQLQJHUIDJOLJWLQGKROGRJVWUXNWXU
'DQPDUNV'HVLJQVNROH
0nOVWQLQJ
)DJOLJWLQGKROG
8GGDQQHOVHQVVWUXNWXU
'HVLJQVNROHQL.ROGLQJ
0nOVWQLQJHU
)DJOLJWLQGKROG
6WUXNWXU
$UNLWHNWVNROHQL$DUKXV
0nOVWQLQJ
)DJOLJWLQGKROG
6WUXNWXU
.XQVWDNDGHPLHWV$UNLWHNWVNROH
0nOVWQLQJHU
)DJOLJWLQGKROG
6WUXNWXU
*ODV .HUDPLNVNROHQSn%RUQKROP
0nOVWQLQJHU
)DJOLJWLQGKROG
6WUXNWXU
*HQHUHOOHYXUGHULQJHURJDQEHIDOLQJHUYHGUPnOVWQLQJHUIDJOLJW
LQGKROGRJVWUXNWXU
6WXGLHIRUPHUSUDNWLNRJEHGPPHOVHVIRUKROG
'DQPDUNV'HVLJQVNROH
'HVLJQVNROHQL.ROGLQJ
$UNLWHNWVNROHQL$DUKXV
.XQVWDNDGHPLHWV$UNLWHNWVNROH
*ODV .HUDPLNVNROHQSn%RUQKROP
*HQHUHOOH YXUGHULQJHU RJ DQEHIDOLQJHU RP XQGHUYLVQLQJ VWXGLHIRUPHUSUDNWLNRJEHGPPHOVHVIRUPHU
'HVWXGHUHQGH
2SWDJHOVH
6WXGLHPLOM
6WXGLHYHMOHGQLQJ
6WXGLHWLGRJIUDIDOG
/HGHOVHRJRUJDQLVDWRULVNHIRUKROG
'DQPDUNV'HVLJQVNROH
'HVLJQVNROHQL.ROGLQJ
$UNLWHNWVNROHQL$DUKXV
.XQVWDNDGHPLHWV$UNLWHNWVNROH
*ODV .HUDPLNVNROHQSn%RUQKROP
NRQRPLVNHIRUKROG
)DFLOLWHWHURJE\JQLQJHU
8QGHUYLVHUJUXSSHQ
,QWHUQHYDOXHULQJ
'DQPDUNV'HVLJQVNROH
'HVLJQVNROHQL.ROGLQJ
$UNLWHNWVNROHQL$DUKXV
.XQVWDNDGHPLHWV$UNLWHNWVNROH
*ODV .HUDPLNVNROHQSn%RUQKROP
$UEHMGVPDUNHGHWRJHUKYHUYVOLYHW
1DWLRQDOWRJLQWHUQDWLRQDOWVDPDUEHMGH 6DPDUEHMGHPHOOHPXGGDQQHOVHVLQVWLWXWLRQHUL'DQPDUN
0HULWIRUVWXGHUHQGH
,QWHUQDWLRQDOWVDPDUEHMGH
$SSHQGLNV$
$SSHQGLNV%
$SSHQGLNV&
$SSHQGLNV'
$SSHQGLNV(
$SSHQGLNV)
)RURUG
, GHQQH UDSSRUW IUHPOJJHV HYDOXHULQJHQ DI GHVLJQXGGDQQHOVHUQH XQGHU .XOWXUPLQLVWHULHW .XOWXUPLQLVWHULHWHURSGUDJVJLYHURJHYDOXHULQJHQHULYUNVDWDI'DQPDUNV(YDOXHULQJVLQVWLWXWGHU HU HQ VHOYVWQGLJ LQVWLWXWLRQ XQGHU 8QGHUYLVQLQJVPLQLVWHULHW 'DQPDUNV (YDOXHULQJVLQVWLWXW ELGUDJHU JHQQHP HYDOXHULQJ WLO XGYLNOLQJ RJ V\QOLJKHG L GHW GDQVNH XGGDQQHOVHVV\VWHP ¥ IUD IRONHVNROHRYHUXQJGRPVXGGDQQHOVHUWLOYLGHUHJnHQGHXGGDQQHOVHU
(YDOXHULQJHQ DI GHVLJQXGGDQQHOVHUQH XQGHU .XOWXUPLQLVWHULHW HU JHQQHPIUW L HW VDPDUEHMGH PHOOHP 'DQPDUNV (YDOXHULQJVLQVWLWXW HQ IDJOLJ HYDOXHULQJVJUXSSH RJ IOJHJUXSSHU Sn GH LQYROYHUHGH XGGDQQHOVHVLQVWLWXWLRQHU 8QGHUYLVHUH VWXGHUHQGH RJ DGPLQLVWUDWLYW SHUVRQDOH Sn XGGDQQHOVHUQH KDU GHOWDJHW L KHQKROGVYLV XGDUEHMGHOVHQ DI VHOYHYDOXHULQJVUDSSRUWHUQH RJ YHG LQVWLWXWLRQVEHVJHQH
5DSSRUWHQV DQDO\VHU RJ DQEHIDOLQJHU YLO LQGJn L .XOWXUPLQLVWHULHWV YLGHUH DUEHMGH RJ VDPWLGLJ GDQQH JUXQGODJ IRU XGGDQQHOVHVLQVWLWXWLRQHUQHV IRUWVDWWH DUEHMGH PHG XGYLNOLQJ RJ VLNULQJ DI NYDOLWHWHQSnXGGDQQHOVHUQH
,QGOHGQLQJ
(YDOXHULQJHQ DI GHVLJQXGGDQQHOVHUQH XQGHU .XOWXUPLQLVWHULHW HU LJDQJVDW IRUnUHW HIWHU DQPRGQLQJ DI .XOWXUPLQLVWHULHW (YDOXHULQJHQ RPIDWWHU GHVLJQXGGDQQHOVHUQH Sn 'DQPDUNV 'HVLJQVNROH 'HVLJQVNROHQ L .ROGLQJ *ODV .HUDPLNVNROHQ Sn %RUQKROP .XQVWDNDGHPLHWV
$UNLWHNWVNROHRJ$UNLWHNWVNROHQL$DUKXV
)RUPnO
, KHQKROG WLO NRPPLVVRULHW HU IRUPnOHW PHG HYDOXHULQJHQ ¨PHG XGJDQJVSXQNW L XGGDQQHOVHUQHV PnOVWQLQJHUDWEHO\VHXGGDQQHOVHVNYDOLWHWHQJHQQHPHQDQDO\VHGHOVDIYVHQWOLJHDVSHNWHUYHG XGGDQQHOVHUQHSnGHEHUUWHLQVWLWXWLRQHUGHOVDIGHYLONnUVRPXGGDQQHOVHUQHIXQJHUHUXQGHU¨
(YDOXHULQJHQVNDOLQGGUDJH
• 2UJDQLVDWRULVNHVWXGLHVWUXNWXUHOOHYLONnU
• $GPLQLVWUDWLYHNRQRPLVNHYLONnU
• 8QGHUYLVHUJUXSSHQVVDPPHQVWQLQJKHUXQGHUDQVWWHOVHVIRUKROG
• 6WXGLHRSE\JQLQJKHUXQGHURSWDJHOVHVNULWHULHUDIVWLJQLQJVPXOLJKHGHURJPXOLJKHGIRUPHULWSnGH
YULJHGHVLJQXGGDQQHOVHU
• 8GGDQQHOVHVIRUOEKHUXQGHUXGGDQQHOVHQVOQJGH
• )DJOLJWLQGKROGKHUXQGHUGHKnQGYUNVPVVLJHRJWHRUHWLVNHDVSHNWHU
• 3UDNWLNRUGQLQJHU
• 6WXGLHIRUPHU
• 3GDJRJLVNXGYLNOLQJRJHIWHUXGGDQQHOVH
• (YDOXHULQJVSUYHRJHNVDPHQVIRUKROG
• ,QWHUQDWLRQDOLVHULQJVDVSHNWHWLXGGDQQHOVHQKHUXQGHUVSUJVPnOHWRPWLWXODWXU
• 'LPLWWHQGHUQHVDQVWWHOVHVIRUKROG
• (NVWHUQHUHODWLRQHUPHGQDWLRQDOHRJLQWHUQDWLRQDOHXGGDQQHOVHVLQVWLWXWLRQHURJHUKYHUYVOLYHW
• .YDOLWHWVVLNULQJDIXQGHUYLVQLQJHQ
+HUXGRYHU VNDO HYDOXHULQJHQ EHO\VH RJ LQGHKROGH IRUVODJ WLO HQ DUEHMGVGHOLQJ PHOOHP LQVWLWXWLRQHUQH IRU Vn YLGW DQJnU XGGDQQHOVHVXGEXGHW (QGYLGHUH VNDO HYDOXHULQJHQ DIGNNH L KYLONHW RPIDQJ VNROHUQH KDU HW IRUVNQLQJVSRWHQWLDOH IRUVWnHW VRP GH YLONnU RJ UDPPHU VRP HNVLVWHUHUIRUIRUVNQLQJKHUXQGHUEODOUHUJUXSSHQVVDPPHQVWQLQJRJXGGDQQHOVHLIRUKROGWLO UHOHYDQWH IRUVNQLQJVRPUnGHU VDPW L KYLONHQ JUDG XQGHUYLVQLQJHQ LQGGUDJHU UHVXOWDWHUQH DI IRUVNQLQJ RJ NXQVWQHULVN XGYLNOLQJVDUEHMGH <GHUOLJHUH XGGDQQHOVHVDVSHNWHU NDQ LQGGUDJHV L GHW RPIDQJGHSnGUDJHUVLJVUOLJRSPUNVRPKHG
(YDOXHULQJVJUXSSHQRJSURMHNWOHGHOVH
, VDPUnG PHG RSGUDJVJLYHU KDU 'DQPDUNV (YDOXHULQJVLQVWLWXW QHGVDW HQ HYDOXHULQJVJUXSSH PHG GHW IDJOLJH DQVYDU IRU HYDOXHULQJHQ KHUXQGHU GH DQEHIDOLQJHU RJ NRQNOXVLRQHU VRP HU JLYHW L UDSSRUWHQPHGXGJDQJVSXQNWLGRNXPHQWDWLRQVPDWHULDOHW
(YDOXHULQJVJUXSSHQVVDPPHQVWQLQJHUVRPIOJHU
• 3URUHNWRU%HQWH.ULVWHQVHQ+DQGHOVKMVNROHQL.EHQKDYQIRUPDQG
• 'LUHNWU3HWHU%\VWHG%\VWHG$6
• 3URJUDPDQVYDUOLJ+nNDQ(GHKROW0DOP+JVNROD
• 3URIHVVRU3HU2ODI)MHOG$UNLWHNWKMVNROHQL2VOR
• .XQVWKnQGYUNHU8UVXOD0XQFK3HWHUVHQ
'DQPDUNV (YDOXHULQJVLQVWLWXW KDU KDIW GHW SUDNWLVNH RJ PHWRGLVNH DQVYDU IRU HYDOXHULQJHQ KHUXQGHULQGVDPOLQJRJEHDUEHMGQLQJDIGRNXPHQWDWLRQVPDWHULDOHNRQWDNWWLOHYDOXHULQJVJUXSSHQ VDPWWLOLQVWLWXWLRQHUPHGIOHUH(YDOXHULQJVUDSSRUWHQHUVNUHYHWDIHYDOXHULQJVNRQVXOHQW0LFKDHO
$QGHUVHQRJHYDOXHULQJVPHGDUEHMGHU0DULH/RXLVH:LGGLQJ
3UVHQWDWLRQDIXGGDQQHOVHVRPUnGHW
, GHW IOJHQGH YLO GHU EOLYH UHGHJMRUW IRU GH IHP XGGDQQHOVHVLQVWLWXWLRQHUV LQVWLWXWLRQHOOH UDPPHU KYRUHIWHUHQUHGHJUHOVHIRUKYHUHQNHOWXGGDQQHOVHVLQVWLWXWLRQVVWUXNWXURJRSWDJHOVHVSURFHGXUHU YLOIOJH
6DPWOLJHIHPXGGDQQHOVHVLQVWLWXWLRQHUHULYDULHUHQGHRPIDQJUHJXOHUHWYHG/RYQUDI DSULORPYLGHUHJnHQGHXGGDQQHOVHVLQVWLWXWLRQHUXQGHU.XOWXUPLQLVWHULHW6RPGHQHQHVWHDI LQVWLWXWLRQHUQH HU 'HVLJQVNROHQ L .ROGLQJ IRUWVDW HQ VHOYHMHQGH LQVWLWXWLRQ KYRUIRU 'HVLJQVNROHQ L .ROGLQJ NXQ GHOYLVW HU XQGHUODJW RYHQVWnHQGH EHVWHPPHOVHU $I ORYHQ IUHPJnU GH ILUH VWDWVLQVWLWXWLRQHUVRSJDYHUVRPIOJHU
¨ .XQVWDNDGHPLHWV $UNLWHNWVNROH RJ $UNLWHNWVNROHQ L $DUKXV KDU VRP KMHUH XGGDQQHOVHVLQVWLWXWLRQHU WLO RSJDYH Sn NXQVWQHULVN RJ YLGHQVNDEHOLJW JUXQGODJ DW JLYH XGGDQQHOVH L DUNLWHNWXU LQGWLO GHW KMHVWH QLYHDX VDPW DW XGYH NXQVWQHULVN XGYLNOLQJVYLUNVRPKHG RJ Sn YLGHQVNDEHOLJW JUXQGODJ DW GULYH IRUVNQLQJ LQGHQ IRU DUNLWHNWXUHQ
D 'DQPDUNV 'HVLJQVNROH KDU WLO RSJDYH DW JLYH XGGDQQHOVH LQGHQIRU NXQVWKnQGYUN GHVLJQRJEHVOJWHGHIDJ
E*ODV .HUDPLNVNROHQSn%RUQKROPKDUWLORSJDYHDWJLYHXGGDQQHOVHLQGHQIRUJODV RJ NHUDPLN 6NROHQ NDQ KHUXGRYHU JLYH XGGDQQHOVH L DQGUH IDJ LQGHQ IRU NXQVWKnQGYUN¨
)RU 'HVLJQVNROHQ .ROGLQJ JOGHU GHW DW GHQ HU RPIDWWHW DI ORYHQV NDSLWHO YHGUUHQGH XGGDQQHOVHRJIRUVNQLQJNDSLWHOYHGUUHQGHNRQRPLVNHIRUKROGVDPWNDSLWHODNXQIRU VHOYHMHQGH LQVWLWXWLRQHU HIWHU KYLONHQ .XOWXUPLQLVWHULHW EOD VNDO JRGNHQGH LQVWLWXWLRQHQV YHGWJWHU VDPW L YULJW NDQ IDVWVWWH QUPHUH UHJOHU YHGUUHQGH YLONnU IRU WLOVNXG DQWDO VWXGHUHQGHNRQWURORJWLOV\QPHGLQVWLWXWLRQHQPY
6RP QHGHQVWnHQGH WDEHO YLVHU HU GHU VWRU IRUVNHO Sn DQWDOOHW DI DQVJHUH RJ Sn KYRU PDQJH DI GLVVHGHUEOLYHURSWDJHWSnGHUHVSHNWLYHXGGDQQHOVHVLQVWLWXWLRQHU
6HQHVWQGUHWYHG/RYQUDIPDUWV
7DEHO
$QWDODQVJHUHRJRSWDJQHVDPWRSWDJHOVHVNYRWLHQWL
$QWDODQVJHUH $QWDORSWDJQH 2SWDJQHLDI DQVJHUH
'DQPDUNV'HVLJQVNROH
'HVLJQVNROHQL.ROGLQJ
*ODV .HUDPLNVNROHQSn%RUQKROP .XQVWDNDGHPLHWV$UNLWHNWVNROH
$UNLWHNWVNROHQL$DUKXV
7DOOHQH IRU .XQVWDNDGHPLHWV $UNLWHNWVNROH RJ $UNLWHNWVNROHQ L $DUKXV HU GHW VDPOHGH DQWDO DQVJHUH RJ RSWDJQHKHULEODQGWGHVLJQDIGHOLQJHUQH
, GHW IOJHQGH YLO GH HQNHOWH XGGDQQHOVHVLQVWLWXWLRQHUV VWUXNWXU RJ RSWDJHOVHVSURFHGXUHU EOLYH EHVNUHYHW 2YHURUGQHW NDQ PDQ LQGGHOH XGGDQQHOVHVLQVWLWXWLRQHUQH L WR RPUnGHU 'HVLJQVNROHUQH PHG'DQPDUNV'HVLJQVNROHRJ'HVLJQVNROHQL.ROGLQJRJDUNLWHNWVNROHUQHPHG.XQVWDNDGHPLHWV
$UNLWHNWVNROHRJ$UNLWHNWVNROHQL$DUKXV+HUXGRYHUHUGHU*ODV .HUDPLNVNROHQSn%RUQKROPGHU LNNHXPLGGHOEDUWNDQSODFHUHVLHWDIGLVVHRPUnGHUJUXQGHWVNROHQVVUOLJHVWUXNWXURJIRUPnO
'DQPDUNV'HVLJQVNROH
8GGDQQHOVHQSn'DQPDUNV'HVLJQVNROHHUIHPnULJPHGPXOLJKHGIRUDWWDJHDIJDQJHIWHUånU
8GGDQQHOVHQ HU RSE\JJHW VnOHGHV DW DOOH VWXGLHVWDUWHUH EHJ\QGHU Sn HW IOOHV EDVLVnU GHU KDU WLO IRUPnODW¨VNDEHIOOHVIXQGDPHQWIRUXGGDQQHOVHQVRPGHVLJQHUXGYLNOHGHVWXGHUHQGHVNUHDWLYH RJ NXQVWQHULVNH HYQHU EHDUEHMGH KROGQLQJHU RJ VNDEH IRUVWnHOVH IRU GHUHV UROOH RJ DQVYDU VRP XGYHQGHGHVLJQHUH¨(IWHUEDVLVnUHWEHJ\QGHUGHVWXGHUHQGHLQVWLWXWVWXGLHWGHUKDUHWRPIDQJDI WR nU 'H VWXGHUHQGH KDU PXOLJKHG IRU DW DIVWLJH HIWHU LQVWLWXWVWXGLHW PHG HW XGGDQQHOVHVEHYLV VnIUHPWGHJRGNHQGHVWLORSU\NQLQJWLOVWXGLHWWUHGMHGHOPHQPXOLJKHGHQEHQ\WWHVNXQDIJDQVNH InVWXGHUHQGHFDWRKYHUWnU8GGDQQHOVHVEHYLVHWJLYHUGRJLNNHPHULWRYHUIUVHOWLOIRUWVWWHOVH Sn DQGUH VWXGLHU ERUWVHW IUD 'HVLJQVNROHQ L .ROGLQJ 6WXGLHWV WUHGMH GHO HU VSHFLDOH RJ DIJDQJVVWXGLHGHUKDUHWRPIDQJDIHWKDOYWWLOWRnU
+YLVGHQVWXGHUHQGHLQGVWLOOHUVLJWLODIJDQJHIWHUJRGNHQGWDIVOXWQLQJDIDQGHQGHOJnUGHU\GHUOLJHUHHWKDOYWnU LQGHQHQGHOLJDIJDQJ
6WXGLHKnQGERJHQ
2SWDJHOVHQSn'DQPDUNV'HVLJQVNROHWDJHUXGJDQJVSXQNWLWLOPHOGHOVHQWLORSWDJHOVHVSUYHQKYRU DQVJHUQH YHGOJJHU HQ PDSSH WLO IRUYXUGHULQJ LQGHKROGHQGH UHOHYDQWH DUEHMGHU 'DQPDUNV 'HVLJQVNROHXGYOJHUSnEDJJUXQGDIPDSSHUQHHWYLVWDQWDODQVJHUHXGWLORSWDJHOVHVSUYHSn VNROHQ GHU YDUHU L ILUH GDJH RJ EHVWnU DI OVQLQJ DI RSJDYHU VDPW HQ SHUVRQOLJ VDPWDOH 3n EDJJUXQGDIPDSSHQRSWDJHOVHVSUYHQRJVDPWDOHQXGYOJHVGHWHQGHOLJHDQWDORSWDJQH
'HVLJQVNROHQL.ROGLQJ
8GGDQQHOVHQSn'HVLJQVNROHQL.ROGLQJHUIHPnULJRJEHVWnUDIHQHWnULJJUXQGXGGDQQHOVHPHG WYUJnHQGHEDVLVNXUVHULIRUPDIWHRULRJIDJOLJLQWURGXNWLRQ+HUHIWHUNRPPHUGHQWRnULJHDQGHQ GHO DI XGGDQQHOVHQ KYRU GH VWXGHUHQGH EHJ\QGHU Sn GHQ IDJOLJH GHO LQGHKROGHQGH WHNQLVNH RJ NXQVWQHULVNH VWXGLHU VnYHO VSHFLILNW IDJOLJW VRP WYUIDJOLJW 'H VWXGHUHQGH KDU PXOLJKHG IRU DIVWLJQLQJ HIWHU JHQQHPIUVHO DI DQGHQGHOHQ KYRU 'HVLJQVNROHQ L .ROGLQJ XGVWHGHU HW OLQLHEHYLV .QDSW KYHU IHPWH VWXGHUHQGH KDU L EHQ\WWHW VLJ DI GHQQH PXOLJKHG 1RJOH DI GLVVH VWXGHUHQGH YOJHU DW IUGLJJUHVWXGLHWHQWHQSnHQDQGHQVNROHLXGODQGHWHOOHUSn'DQPDUNV 'HVLJQVNROH 3n GHQ WRnULJH WUHGMHGHO VSHFLDOLVHUHU GH VWXGHUHQGH VLJ LQGHQIRU GHUHV IDJOLJH RPUnGHKYRUGHOLJHOHGHVKDUPXOLJKHGIRUSUDNWLNHOOHUXGHQODQGVRSKROG+HUXGRYHUDUEHMGHUGH VWXGHUHQGH Sn GHUHV LQGVWLOOLQJV RJ DIJDQJVSURMHNWHU 'H VWXGHUHQGH KDU PXOLJKHG IRU DW WDJH DIJDQJLOEHWDIWUHGMHGHOHQVWRnU
9HGDQVJQLQJRPRSWDJHOVHSn'HVLJQVNROHQL.ROGLQJDQEHIDOHUVNROHQDWDQVJHUHQLPLQGVWWR nUKDUDUEHMGHWPHGWHJQLQJIRUPRJIDUYHHOOHUKDUOLJQHQGHNUHDWLYHUIDULQJ9HGWLOPHOGLQJHQWLO RSWDJHOVHVSUYHQ WLOVHQGHV DQVJHUQH HW DQWDO KMHPPHRSJDYHU GHU VNDO OVHV RJ YHGOJJHV WLOPHOGLQJHQ 'LVVH KMHPPHRSJDYHU XGJU JUXQGODJHW IRU XGYOJHOVHQ DI DQVJHUH WLO RSWDJHOVHVSUYHQ6HOYHRSWDJHOVHVSUYHQYDUHULWUHGDJHKYRUDQVJHUHQOVHURSJDYHUSnVNROHQ RJ LQGNDOGHV WLO SHUVRQOLJ VDPWDOH 6NROHQ XGYOJHU Sn EDJJUXQG DI IRUYXUGHULQJHQ RSWDJHOVHVSUYHQRJVDPWDOHQKYLONHDQVJHUHGHURSWDJHV
$UNLWHNWVNROHQL$DUKXV
8GGDQQHOVHQSnDUNLWHNWVNROHQHUIHPnULJRJHURSGHOWLHQIUVWHRJDQGHOGHOKYRUIUVWHGHOHQ RPIDWWHUGHWRIUVWHVWXGLHnURJDQGHQGHOHQGHWUHVLGVWHVWXGLHnU)UVWHGHOHQHUIOOHVIRUDOOH VWXGHUHQGH RJ VNDO EOD JLYH GH VWXGHUHQGH HQ DOPHQ DUNLWHNWIDJOLJ IRUVWnHOVH RJ YLGHQ 9HG RYHUJDQJHQ WLO VWXGLHWV DQGHQ GHO VJHU GH VWXGHUHQGH RP RSWDJHOVH Sn GHQ QVNHGH DIGHOLQJ LQGHQ IRU VNDODRPUnGHUQH SODQOJQLQJ E\JQLQJVNXQVW HOOHU GHVLJQ 2SWDJHOVHQ Sn DIGHOLQJHQ DIKQJHUDIRPGHQVWXGHUHQGHHIWHUDIVOXWQLQJHQSnIUVWHGHOHQHUEOHYHWJRGNHQGWDIVNROHQ 3n DQGHQ GHOHQ DI VWXGLHW VSHFLDOLVHUHU GHQ VWXGHUHQGH VLJVnOHGHVLQGHQIRUGHQYDOJWHDIGHOLQJV
IHOW'HUHULQJHQDIVWLJQLQJVPXOLJKHGHUXQGHUVWXGLHIRUOEHWGHUDIVOXWWHVPHGHWDIJDQJVSURMHNWL VWXGLHnU
, OLJKHG PHG DQGUH YLGHUHJnHQGH XGGDQQHOVHU RSWDJHU $UNLWHNWVNROHQ L $DUKXV GH VWXGHUHQGH JHQQHP NYRWH RJNYRWH9LDNYRWHRSWDJHVDIGHVWXGHUHQGHSnEDJJUXQGDIGHUHV DGJDQJVJLYHQGHJ\PQDVLDOHHNVDPHQPHQVDIGHVWXGHUHQGHRSWDJHVJHQQHPNYRWHSn EDJJUXQG DI LQGOHYHUHGH KMHPPHRSJDYHU XGDUEHMGHW L IOOHVVNDE DI DUNLWHNWVNROHUQH VDPW HQ RSWDJHOVHVSUYH
'HWRDUNLWHNWVNROHUVDPRUGQHUGHUHVRSWDJDIVWXGHUHQGHYLDNYRWHRJXGDUEHMGHULIOOHVVNDE HQ LQIRUPDWLRQVEURFKXUH EODQGW DQGHW LQGHKROGHQGH IRUPXOHULQJHQ DI KMHPPHRSJDYHUQH
$QVJHUQHXGYOJHVSnEDJJUXQGDIKMHPPHRSJDYHUQHWLORSWDJHOVHVSUYHQGHUDIKROGHVRYHUWUH GDJHKYRUHIWHUGHWHQGHOLJHDQWDODQVJHUHXGYOJHV
$UNLWHNWVNROHUQHVDPDUEHMGHUOLJHOHGHVRPDIKROGHOVHQDIRSWDJHOVHVSUYHQGHUDIKROGHVVDPWLGLJ VnOHGHVDWDQVJHUHYLDNYRWHNXQNDQVJHpQDIDUNLWHNWVNROHUQH
.XQVWDNDGHPLHWV$UNLWHNWVNROH
8GGDQQHOVHQSn.XQVWDNDGHPLHWV$UNLWHNWVNROHHUIHPnULJRJRSGHOWLHQWRnULJIUVWHGHORJHQ WUHnULJ DQGHQ GHO 8GGDQQHOVHQ WDJHU DIVW L HQ DIGHOLQJVEDVHUHW XGGDQQHOVHVPRGHO 'HQ VWXGHUHQGH YOJHU IUD VWXGLHWV VWDUW Sn KYLONHQ DI GH HOOHYH VWXGLHDIGHOLQJHU KHUXQGHU GHVLJQ KXQKDQQVNHUDWVWXGHUH)RUPnOHWPHGIUVWHGHOHQHUDWJLYHGHVWXGHUHQGHHQJUXQGOJJHQGH WLOHJQHOVH DI DUNLWHNWIDJHWV IRUXGVWQLQJHU RJ RSOULQJ L IDJHWV PHWRGHU 9HG RYHUJDQJHQ WLO DQGHQGHOHQ Sn WUHGMH VWXGLHnU WLOUHWWHOJJHV HW IRUOE GHW VnNDOGWH PLGWYHMVVWXGLH KYRU GHQ VWXGHUHQGHNDQIRUG\EHVLJLVUOLJHHPQHU'HQVWXGHUHQGHRSIRUGUHVWLODWWDJHGHWWHVWXGLHnUSn HQDQGHQVWXGLHDIGHOLQJHQGGHQYDOJWH'HWRVLGVWHVWXGLHnUKDUKRYHGYJWSnVSHFLDOLVHULQJRJ DIJDQJ RJ GHQ VWXGHUHQGH NDQ YOJH DW VSHFLDOLVHUH VLJ Sn GHQ SnEHJ\QGWH VSHFLDOLVHULQJ VWXGLHDIGHOLQJHOOHUIRUWVWWHSnHQDQGHQ'HUHULNNHPXOLJKHGIRUDIVWLJQLQJLOEHWDIVWXGLHWV IHPnU
, OLJKHG PHG DQGUH YLGHUHJnHQGH XGGDQQHOVHU RSWDJHU .XQVWDNDGHPLHWV $UNLWHNWVNROH YLD GHQ NRRUGLQHUHGH WLOPHOGLQJ DI GH VWXGHUHQGH RSWDJHV JHQQHP NYRWH RJ YLD NYRWH 6WXGHUHQGHGHUVJHUJHQQHPNYRWHRSWDJHVSnEDJJUXQGDIGHUHVDGJDQJVJLYHQGHJ\PQDVLDOH HNVDPHQ PHQV VWXGHUHQGH GHU VJHU RSWDJHOVH JHQQHP NYRWH RSWDJHV Sn EDJJUXQG DI OVWH KMHPPHRSJDYHUVDPWHQRSWDJHOVHVSUYH
*ODV .HUDPLNVNROHQSn%RUQKROP
8GGDQQHOVHQ Sn *ODV .HUDPLNVNROHQ Sn %RUQKROP HU WUHnULJ RJ GH VWXGHUHQGH YOJHU YHG RSWDJHOVHHQDIGHWRIDJOLQLHU3nVHPHVWHUXGDUEHMGHUGHVWXGHUHQGHHWPLGWYHMVSURMHNWVnOHGHV DW VNROHQ NDQ YXUGHUH RP GH VWXGHUHQGH KDU WLOVWUNNHOLJIDJOLJIRUVWnHOVHIRUGHUHVIDJRPUnGH 6WXGLHWDIVOXWWHVPHGHWDIJDQJVSURMHNW
)RUXGVWQLQJ IRU RSWDJHOVH Sn VNROHQ HU HQ EDJJUXQG L HQ IDJOLJ RJHOOHU WHRUHWLVN UHOHYDQW RSOULQJ L PLQGVW WUH nU HIWHU HQGW IRONHVNROH 6HOYH DQVJQLQJHQ VNDO YHGOJJHV IULKnQGVWHJQLQJHURJHYHQWXHOWIRWRVDIHJQHYUNHUKYRUXGIUDVNROHQXGYOJHUGHDQVJHUHGHU NYDOLILFHUHU VLJ WLO RSWDJHOVHVSUYHQ WLOUHWWHODJW DI VNROHQ 2SWDJHOVHVSUYHQ IRUHJnU RYHU WR GDJH RJ EHVWnU VnYHO DI SUDNWLVNH RSJDYHU VRP DI HQ SHUVRQOLJ VDPWDOH $QVJQLQJHQ RJ RSWDJHOVHVSUYHQGDQQHUJUXQGODJHWIRUGHQHQGHOLJHXGYOJHOVHDIVWXGHUHQGH'HWHUDIJUHQGH IRUVNROHQDWPDQHULVWDQGWLODWUHNUXWWHUHEUHGWJHRJUDILVNIRUVnOHGHVDWVLNUHHWWLOVWUNNHOLJW VWXGHQWHUJUXQGODJ
'RNXPHQWDWLRQVPDWHULDOHW
'HU HU L IRUELQGHOVH PHG HYDOXHULQJHQ DI GHVLJQXGGDQQHOVHUQH EOHYHW XGDUEHMGHW HW RPIDWWHQGH GRNXPHQWDWLRQVPDWHULDOH'HWRPIDWWHUIOJHQGH
)RUXQGHUVJHOVH
'DQPDUNV (YDOXHULQJVLQVWLWXW JHQQHPIUWH L IRUnUHWHQIRUXQGHUVJHOVHEDVHUHWSnDOOHUHGH HNVLVWHUHQGH PDWHULDOH VnVRP EHNHQGWJUHOVHU VWXGLHRUGQLQJHU RJ YHMOHGQLQJHU VDPW DQGHW EDJJUXQGVPDWHULDOH /LJHOHGHV EOHY GHU DIKROGW HW LQIRUPDWLRQVPGH L MXQL Sn 'DQPDUNV (YDOXHULQJVLQVWLWXWIRUVDPWOLJHLQYROYHUHGHXGGDQQHOVHVLQVWLWXWLRQHU
6HOYHYDOXHULQJVUDSSRUWHU
+YHU XGGDQQHOVHVLQVWLWXWLRQ KDU L HIWHUnUHW XGDUEHMGHW HQ XGIUOLJ VHOYHYDOXHULQJVUDSSRUW VRP HU XGIRUPHW Sn EDJJUXQG DI HQ YHMOHGQLQJ IUD 'DQPDUNV (YDOXHULQJVLQVWLWXW RJ HYDOXHULQJVJUXSSHQ RJ VRP LQGHKROGHU NYDOLWDWLYH VnYHO VRP NYDQWLWDWLYH VSUJVPnO 6HOYHYDOXHULQJVUDSSRUWHUQHEHVNULYHUHQODQJUNNHIRUXGGDQQHOVHUQHUHOHYDQWHIRUKROGRJYLONnU VDPW VW\UNHU RJ VYDJKHGHU YHG GLVVH +HUXGRYHU KDU KYHU XGGDQQHOVHVLQVWLWXWLRQ YHGODJW XQGHUYLVHUQHV&9ªHUVRPVXSSOHPHQWWLOVHOYHYDOXHULQJVUDSSRUWHQ
$UEHMGVPDUNHGVXQGHUVJHOVH
'D HYDOXHULQJHQ LNNH EORW VNDO EHO\VH XGGDQQHOVHUQH PHQ RJVn DUEHMGVPDUNHGVIRUKROGHQH KDU 'DQPDUNV (YDOXHULQJVLQVWLWXW L GHFHPEHU JHQQHPIUW HQ DUEHMGVPDUNHGVXQGHUVJHOVH 8QGHUVJHOVHQ EDVHUHU VLJ Sn RSO\VQLQJHU IUD 'DQPDUNV 6WDWLVWLN YHGUUHQGH GLPLWWHQGHUV DUEHMGVPDUNHGVIRUKROG'LPLWWHQGHUGHUHUGLPLWWHUHWLHOOHUGDQQHUJUXQGODJHW IRUXQGHUVJHOVHQGHUEHO\VHUGHUHVWLONQ\WQLQJWLODUEHMGVPDUNHGHWL
,QVWLWXWLRQVEHVJ
(YDOXHULQJVJUXSSHQRJ'DQPDUNV(YDOXHULQJVLQVWLWXWKDULMDQXDUJHQQHPIUWEHVJDIHQ GDJV YDULJKHG Sn XGGDQQHOVHVLQVWLWXWLRQHUQH 3n DUNLWHNWVNROHUQH YDU GHU VHSDUDWH PGHU PHG OHGHOVH IOJHJUXSSH XQGHUYLVHUHRJVWXGHUHQGH3nGHVLJQVNROHUQHRJ*ODV .HUDPLNVNROHQSn
%RUQKROP YDU GHU VHSDUDWH PGHU PHG IOJHJUXSSHQ VWXGHUHQGH RJ OHGHOVH VDPW LQVWLWXWDIGHOLQJVPGHU PHG XQGHUYLVHUH YUNVWHGVSHUVRQDOH RJ VWXGHUHQGH 9HG PGHUQH KDU GHU YUHW PXOLJKHG IRU DW GUIWH HQ ODQJ UNNH VSUJVPnO VRP EODQGW DQGHW KDU YUHW IRUDQOHGLJHWDIVHOYHYDOXHULQJVUDSSRUWHUQHRJDIDUEHMGVPDUNHGVXQGHUVJHOVHQ
(Q XGG\EHQGH EHVNULYHOVH DI HYDOXHULQJVPHWRGHQ GRNXPHQWDWLRQVPDWHULDOHW RJ LQVWLWXWLRQVEHVJHQHILQGHVLDSSHQGLNVYHGUUHQGHHYDOXHULQJVPHWRGHRJGDWDJUXQGODJ
5DSSRUWHQVRSE\JQLQJ
'HWFHQWUDOHLUDSSRUWHQHUDQDO\VHRJDQEHIDOLQJHUVRPILQGHVLNDSLWHO.DSLWHOLQGHKROGHUHQ LQGOHGQLQJPHQVNDSLWHOLQGHKROGHUHWUHVXPp.DSLWHOHUHQHQJHOVNRYHUVWWHOVHDIGHOHDI NDSLWHOKHOHNDSLWHOVDPWHQSUVHQWDWLRQSnHQJHOVNDIHYDOXHULQJVRPUnGHWRJGHQDQYHQGWH HYDOXHULQJVPHWRGH 7LO VLGVW L UDSSRUWHQ HU GHU HQ UNNH DSSHQGLNV RPIDWWHQGH HYDOXHULQJHQV NRPPLVVRULXP HQ IRUWHJQHOVH RYHU GHOWDJHUQH L HYDOXHULQJHQ HQ SUVHQWDWLRQ DI GHQ DQYHQGWH HYDOXHULQJVPHWRGHPY
5HVXPp
$UEHMGVGHOLQJHQRJVDPDUEHMGHWGHVLJQXGGDQQHOVHUQHLPHOOHP
'HIHPYLGHUHJnHQGHGHVLJQXGGDQQHOVHUXQGHU.XOWXUPLQLVWHULHWHUPHJHWIRUVNHOOLJHRJLOOXVWUHUHU GHUPHG EUHGGHQ L GHVLJQIDJHW L 'DQPDUN +YRU PDQ Sn DUNLWHNWVNROHUQH IUVW RJ IUHPPHVW XGGDQQHU DUNLWHNWHU RJ VHU GHVLJQ VRP SURFHVVHU GHU IRUJnU L UHODWLRQ WLO UXP Vn EHWRQHU GHVLJQVNROHUQHPHGVLQHVROLGHUGGHULKnQGYUNHWGHQWHNQLVNHRJNXQVWQHULVNHKnQGWHULQJDI PDWHULDOHUQHVRPGHWDIJUHQGHXGJDQJVSXQNWIRUIRUPJLYQLQJRJSURGXNWXGYLNOLQJ
6RP XGJDQJVSXQNW ILQGHU HYDOXHULQJVJUXSSHQ GHW YUGLIXOGW DW GHU Sn GDQVN JUXQG ILQGHV GHVLJQXGGDQQHOVHUGHUVSQGHUIUDDUNLWHNWVNROHUQHVWUDGLWLRQVEXQGQHDUNLWHNWIDJOLJHRJJHQHUHOW DQDO\WLVNH WLOJDQJ WLO GHVLJQEHJUHEHW WLO GHVLJQVNROHUQHV KnQGYUNV RJ PDWHULDOHRULHQWHUHGH XGJDQJVSXQNW%HJJHWLOJDQJHKDUKYHUIRUVLJRJLNRPELQDWLRQYUHWPHGWLODWXQGHUVWWWHHQ OLJHIUHPRJIXQNWLRQDOLVWLVNGHVLJQWUDGLWLRQVRPHUEOHYHWHWVUNHQGHIRU'DQPDUN
6DPWLGLJ ILQGHU HYDOXHULQJVJUXSSHQ GHW GRJ WYLQJHQGH QGYHQGLJW DW GHU L GHQ IUHPWLGLJH XGYLNOLQJDIGHVLJQXGGDQQHOVHUQHL'DQPDUNEHYLGVWRJPnOUHWWHWVDWVHVSnDWVNDEHHQIRUQ\HWRJ LQWHJUHUHW XGYLNOLQJ 'HW NDQ VLNUH GDQVN GHVLJQ HQ SRVLWLRQ L HW VDPIXQG KYRU GHW YLUWXHOOHUXP SDUDGRNVDOW QRN HU HQ UHDOLWHW RJ KYRU GHVLJQHUHQV WUDGLWLRQHOOH UROOH HU XQGHU GUDPDWLVN IRUDQGULQJ
(YDOXHULQJVJUXSSHQ ILQGHU GHW KHQVLJWVPVVLJW DW GHU ILQGHV VnYHO HQ DUNLWHNWVNROH VRP HQ GHVLJQVNROHLEnGHVWRJ9HVWGDQPDUN'HWIRUHNRPPHUOLJHOHGHVIRUQXIWLJWDWGHUSnDOOHGLVVH ILUHVNROHUHUPXOLJKHGIRUDWVSHFLDOLVHUHVLJLUHWQLQJDIIUHPVWLOOLQJDIXQLNDLUHWQLQJDILQGXVWULHO GHVLJQHOOHUDWYOJHHQEHVWHPWNRPELQDWLRQKHUDI
'HWPnHIWHUHYDOXHULQJVJUXSSHQVPHQLQJRYHUYHMHVQUPHUHKYLONHIDJRJVSHFLDOHUGHUVNDOYUH DGJDQJ WLO Sn GH ILUH VNROHU OLJHVRP GHW EU RYHUYHMHV DW UHGHILQHUH GH IRUVNHOOLJH VSHFLDOHLQGGHOLQJHU VnVRP 0EHO HOOHU ,QGXVWULHO 'HVLJQ ,QIRUPDWLRQVWHNQRORJL EU YUH HQ YVHQWOLJ RJ LQWHJUHUHW GHO DI DOOH GHVLJQXGGDQQHOVHUQH PHQV IHNV VSHFLDOHW 9LVXHO .RPPXQLNDWLRQNXQQHYUHNRQFHQWUHUHWpWVWHGLVWGDQPDUNRJpWVWHGL9HVWGDQPDUN'HWWH YLOOH LQGHEUH VnYHO UHVVRXUFHPVVLJH IRUGHOH VRP HW VWUUH VWXGHQWHUJUXQGODJ Sn LQVWLWXW HOOHU
IDJRPUnGHQLYHDXOLJHVRPHQVWUUHIDJOLJW\QJGHYLOOHNXQQHWLOWUNNHLQWHUQDWLRQDOHJVWHOUHUH RJGHUYHGVW\UNHGHQLQWHUQDWLRQDOHGLPHQVLRQLXGGDQQHOVHQ
*ODV .HUDPLNVNROHQ Sn %RUQKROP IDOGHU XGHQIRU GHQQH DUEHMGVGHOLQJ LGHW GHU HU WDOH RP HQ DQGHQW\SHVNROHGHULVWHGHWIRUHQGHVLJQIDJOLJHOOHUDUNLWHNWIDJOLJWLOJDQJKDUNDUDNWHUDIDWYUH HQ YLGHUHJnHQGH KnQGYUNVPVVLJ VSHFLDOXGGDQQHOVH LQGHQIRU NHUDPLN RJ JODV (Q XGGDQQHOVH PHGHQVWUNWVSHFLDOLVHUHWSURILOYLOIRUPHQWOLJRJVnIUHPRYHUNXQQHWLOWUNNHVWXGHUHQGHIUDKHOH 6NDQGLQDYLHQ LVU KYLV PDQ L KMHUH JUDG IRUPnU DW VW\UNH GHQ WHRUHWLVNH IRUG\EHOVH RJ GHW UHIOHNWHUHGHSHUVRQOLJHXGWU\N
6nYHO *ODV .HUDPLNVNROHQ Sn %RUQKROP VRP 'HVLJQVNROHQ L .ROGLQJ KDU \WUHW QVNHW RP DW XGYLGHXGGDQQHOVHQPHGVP\NNHGHVLJQHOOHUGHOPHWDO'DGHUHUWDOHRPHWOLOOHIDJRPUnGHGHU GRJ KDU HQ HNVLVWHQVEHUHWWLJHOVH EU XGGDQQHOVHQ HIWHU HYDOXHULQJVJUXSSHQV YXUGHULQJ NXQ HWDEOHUHV HW VWHG L 'DQPDUN 6nIUHPW .XOWXUPLQLVWHULHW QVNHU HW VnGDQW VSHFLDOH Sn HQ DI GH HYDOXHUHGHVNROHUDQEHIDOHVGHWDWGHWWHSODFHUHVSn'HVLJQVNROHQL.ROGLQJ
(Q PHJHW XGYLNOHW DUEHMGVGHOLQJ VNROHUQH LPHOOHP KYRU GH VWXGHUHQGH UHODWLYW OHW NXQQH EOLYH RYHUIUWPHOOHPVNROHUQHYLOOHIRUXGVWWHHQLQGE\UGHVWLOQUPHOVHDIVWUXNWXUHQIHNVLIRUPDI IRUVNHOOLJHYDULDQWHUDIHQWUHSOXVWRPRGHOVRPNHQGHVSn8QGHUYLVQLQJVPLQLVWHULHWVRPUnGH(Q VnGDQ WLOQUPHW VWUXNWXU NDQ nEQH RS IRU DW PDQ HIWHU GHIUVWHWUHnUPHGHWEHYLVLIRUPDI IHNV HQ XGIUOLJ VWXGLHMRXUQDO GHU GRNXPHQWHUHU GHW SnJOGHQGH QLYHDX NDQ VSHFLDOLVHUH VLJ LQGHQIRU HQ EUHGHUH YLIWH DI XGGDQQHOVHVWLOEXG Sn ODQGHWV DUNLWHNW RJ GHVLJQVNROHU +HUWLO NRPPHU DW LQWHUQDWLRQDO XGYHNVOLQJ YLOOH EOLYH QHPPHUH PHG HQ VnGDQ VWUXNWXUQGULQJ GHU GRJ LNNHPnYUHWLOKLQGHUIRUDWGHUIRUWVDWNDQYUHIRUVNHOOLJHSURILOHUSnVNROHUQHIHNVLIRUKROGWLO YJWQLQJHQDIGHWKnQGYUNVPVVLJHRJGHWWHRUHWLVNH'HWVNDOGRJXQGHUVWUHJHVDWEHVJHQH Sn VNROHUQH LNNH Sn QRJHQ PnGH JDY DQOHGQLQJWLODWWURDWHQVnGDQPRGHOHUHWQVNHEODQGW DQVDWWHRJVWXGHUHQGHSnGHVLJQXGGDQQHOVHUQH
(YDOXHULQJVJUXSSHQ QVNHU DW XQGHUVWUHJH DW KHQVLJWHQ PHG IRUVODJHW RP HQ WLOQUPHOVH DI VWUXNWXUHQ VNROHUQH LPHOOHP DOHQH HU DW VW\UNH IOHNVLELOLWHWHQ RJ PHULWPXOLJKHGHUQH LQGHQIRU GHVLJQXGGDQQHOVHUQHPHQVJUXSSHQLNNHQVNHDWYUHIRUWDOHUIRULQGIUHOVHDIHQEDFKHORUJUDG SnGHVLJQRPUnGHW
(W PHGOHP DI HYDOXHULQJVJUXSSHQ HU LNNH HQLJ L DQEHIDOLQJHQ YHGUUHQGH HWDEOHULQJ DI HQ DIVWLJQLQJVPXOLJKHG HIWHU WUHGMH nU Sn DUNLWHNWXGGDQQHOVHUQH Sn JUXQG DI PXOLJH NRQVHNYHQVHU IRU DUNLWHNWXGGDQQHOVHUQHV KHOKHGVVWUXNWXU
6DPDUEHMGHW PHOOHP GHVLJQVNROHUQH RJ DUNLWHNWVNROHUQH HU LNNH RPIDQJVULJW KYLONHW HYDOXHULQJVJUXSSHQ VHU VRPHWSUREOHP'HWHUJUXSSHQVYXUGHULQJDWHWXGYLGHWVDPDUEHMGHYLO NXQQH JDYQH GHVLJQXGGDQQHOVHUQH Sn EnGH GHVLJQ RJ DUNLWHNWVNROHUQH (YDOXHULQJVJUXSSHQ DQEHIDOHUDWVNROHUQHLKMHUHJUDGSULRULWHUHUXGYLNOLQJHQDIEnGHDUEHMGVGHOLQJHQRJVDPDUEHMGHW L IRUKROG WLO VnYHO XQGHUYLVQLQJV RJ IRUVNQLQJVPLOMHUQH VRP GHQ YUNVWHGV RJ KnQGYUNVPVVLJHNDSDFLWHW
6DPDUEHMGHW NDQ PHG IRUGHO RPIDWWH DQGUH XGGDQQHOVHVLQVWLWXWLRQHU HQG GHVLJQXGGDQQHOVHUQH, GHQKHQVHHQGHHUHWDIGHPHVWORYHQGHHNVHPSOHUSnQDWLRQDOWVDPDUEHMGHGHWYLUWXHOOH&HQWHU IRU ,QWHJUHUHW 'HVLJQ VRP HU HW WYUIDJOLJW RJ WYULQVWLWXWLRQHOW IRUVNQLQJVEDVHUHW VDPDUEHMGH PHOOHP EOD $UNLWHNWVNROHQ L $DUKXV $DOERUJ 8QLYHUVLWHW RJ 'DQPDUNV 7HNQLVNH 8QLYHUVLWHW (W DQGHWJRGWHNVHPSHOSnWYUJnHQGHVDPDUEHMGHHUHWHWnULJWWYUIDJOLJWDIJDQJVNXUVXVYHGQDYQ ,QWHJUHUHW 'HVLJQ VRP LQYROYHUHU 'DQPDUNV 'HVLJQVNROH 'DQPDUNV 7HNQLVNH 8QLYHUVLWHW VDPW +DQGHOVKMVNROHQL.EHQKDYQ
*HQHUHOW VHW HU GHU LNNH VWUUH SUREOHPHU IRUEXQGHW PHG DW In PHULW PHOOHP GH WR DUNLWHNWXGGDQQHOVHUHOOHUPHOOHPGHWRGHVLJQVNROHU0HOOHP'HVLJQVNROHQL.ROGLQJRJ'DQPDUNV 'HVLJQVNROH HU GHUIRUPHOWVHWXEHJUQVHWPHULWRYHUIUVHOLGHWVNROHUQHJHQVLGLJWKDUDQHUNHQGW VWXGLHIRUOEHQH VRP NYLYDOHQWH 'HW HU GHULPRG ODQJW YDQVNHOLJHUH DW In PHULW PHOOHP GHVLJQXGGDQQHOVHUQHRJDUNLWHNWXGGDQQHOVHUQH0HULWPXOLJKHGHUQHPHOOHP*ODV .HUDPLNVNROHQ Sn %RUQKROP RJ GH YULJH VNROHU IXQJHUHU KHOOHU LNNH WLOIUHGVVWLOOHQGH 'HUIRU DQEHIDOHU HYDOXHULQJVJUXSSHQ DW GHU LJDQJVWWHV HW PnOUHWWHW DUEHMGH IRU Sn NRUW VLJW DW IRUEHGUH PXOLJKHGHUQH IRU DW In PHULW 3n OQJHUH VLJW YLO HQ JUXQGOJJHQGH IRUEHGULQJ DI PHULWPXOLJKHGHUQH PHOOHP DOOH VNROHUQH GRJ IRUXGVWWH VWUXNWXUHOOH RJ LQGKROGVPVVLJH
QGULQJHUSnGLVVH
)RUVNQLQJRJIRUVNQLQJVSRWHQWLDOH
,IOJH NRPPLVVRULHW VNDO HYDOXHULQJHQ DI GHVLJQXGGDQQHOVHUQH DIGNNH RPIDQJHW DI VNROHUQHV IRUVNQLQJVSRWHQWLDOHU (YDOXHULQJVJUXSSHQV JUXQGODJ KHUIRU HU IUVW RJ IUHPPHVW VHOYHYDOXHULQJVUDSSRUWHUQH PGHUQH L IRUELQGHOVH PHG EHVJHQH VDPW GH DI VNROHUQH LQGVHQGWH
&9ªHU IRU XQGHUYLVHUQH &9ªHUQH HU JHQHUHOW VHW UHW XHQVDUWHGH RJ DIGNNHU NXQLXWLOVWUNNHOLJ JUDG NYDOLWDWLYH IRUKROG 'HUIRU KDU (YDOXHULQJVJUXSSHQ YDOJW DW IRNXVHUH Sn OUHUQHV XGGDQQHOVHVPVVLJH EDJJUXQG VDPW VNROHUQHV HJQH RSO\VQLQJHU RP GHUHV IRUVNQLQJVDNWLYLWHW WLO EHO\VQLQJDIIRUVNQLQJVSRWHQWLDOHWSnGHVLJQVNROHUQH
6nOHGHV KDU NXQ HW PLQGUHWDO DI GH IDVWDQVDWWH XQGHUYLVHUH Sn GHVLJQVNROHUQH HQ NDQGLGDWJUDG PHQV GH YULJH XQGHUYLVHUH LNNH KDU XGGDQQHOVHU IUD IRUVNQLQJVEDVHUHGH XQGHUYLVQLQJVLQVWLWXWLRQHU 'HWWH YLVHU DOW DQGHW OLJH DW IRUVNQLQJVSRWHQWLDOHW Sn GHVLJQVNROHUQH
IRUPHOW VHW Sn EDJJUXQG DI XGGDQQHOVHVEDJJUXQG HU VSLQNHOW 'HW VNDO GRJ XQGHUVWUHJHV DW
&9ªHUQH RJVn GRNXPHQWHUHU DW OUHUQH Sn GHVLJQVNROHUQH KDU HQ EHW\GHOLJ GHVLJQIDJOLJ HUIDULQJVEDJJUXQGLIRUPDIXGVWLOOLQJHUSXEOLNDWLRQHURJPHJHWDQGHW'HQQHHUIDULQJVEDJJUXQG HU PHJHW YUGLIXOG LNNH NXQ L IRUKROG WLO XQGHUYLVQLQJHQV NYDOLWHW PHQ RJVn VRP HQ GHO DI XGJDQJVSXQNWHWIRUIRUVNQLQJ
)RU DUNLWHNWVNROHUQHV YHGNRPPHQGH KDU HYDOXHULQJVJUXSSHQ WDJHW DG QRWDP DW GLVVH HU IRUVNQLQJVLQVWLWXWLRQHURJGHUPHGKDUIRUVNQLQJVSRWHQWLDOH(QQUPHUHEHO\VQLQJDIIRUVNQLQJHQV NYDOLWHWIRUGUHUHQHJHQWOLJIRUVNQLQJVHYDOXHULQJRJGHWOLJJHUXGHQIRUGHQQHHYDOXHULQJVUDPPHU
0HJHWDIGHQWHRULVRPGHULGDJXQGHUYLVHVLSnGHVLJQXGGDQQHOVHUQHKDQGOHURPNXQVWKLVWRULH PDWHULDOHNXQGVNDE RJ OLJQHQGH VDPW IRU DUNLWHNWXGGDQQHOVHUQHV YHGNRPPHQGH DUNLWHNWXUWHRUL 'HVXGHQILQGHVGHUUHIOHNVLRQHURYHUSUDNWLVNHHUIDULQJHU0HQGHUV\QHVDWYUHHWVWRUWXGNNHW EHKRY IRU DW XGYLNOH HQ RPUnGHVSHFLILN WHRULGDQQHOVH VRP JnU PHUH L G\EGHQ IRU DW IRUVWn GHVLJQSURFHVVHQ RJ VRP NDQ IXQJHUH YHMOHGHQGH L IRUKROG WLO PHWRGHU RJ SUDNVLV 'HWWH LNNH PLQGVW L EHWUDJWQLQJ DI GHQ KXUWLJH IRUDQGULQJVSURFHV PHG KHQV\Q WLO GH VDPIXQGVPVVLJH IRUYHQWQLQJHUWLOGHVLJQHUQHVNRPSHWHQFHU
6RPIRUXGVWQLQJHUIRUDWGHVLJQVNROHUQHNDQEOLYHIRUVNQLQJVLQVWLWXWLRQHUPnQYQHVHQG\EHUH GHVLJQPHWRGLVN WLOJDQJ WLO VWXGLHW HW EUHGHUH WYUIDJOLJW VDPDUEHMGH HQ IRUVNQLQJVPVVLJ RSNYDOLILFHULQJDIGHWQXYUHQGHSHUVRQDOHVDPWHQUHNUXWWHULQJDIIRUVNHUH'HUHUEHKRYIRUDW XGYLNOHHWWYUIDJOLJWGHVLJQVSURJRJHWXGE\JJHWVDPDUEHMGHPHGDQGUHIRUVNQLQJVLQVWLWXWLRQHU 'HU ILQGHV GRJ DOOHUHGH L GDJ HW EHJUQVHW PHQ SHUVSHNWLYULJW VDPDUEHMGH PHG IOHUH XGGDQQHOVHVLQVWLWXWLRQHU
6nIUHPW GHVLJQXGGDQQHOVHUQH EOHY IRUVNQLQJVEDVHUHGH YLOOH GHW YUH QGYHQGLJW DW WLOVLNUH GHQ QGYHQGLJH EDODQFH PHOOHP LQGYHOVHQ DI WHNQLNNHU PDWHULDOHNHQGVNDE VDPW PHWRGLVN RJ WHRUHWLVNLQGVLJW'HWYLOGHVXGHQYUHSnNUYHWDWUHGXFHUHDQWDOOHWDIJVWHOUHUHWLOIRUGHOIRU IOHUHIDVWDQVDWWHXQGHUYLVHUHPHGIRUVNQLQJVIRUSOLJWHOVH
'HU YLO JUDGYLVW NXQQH VNH HQ VW\UNHOVH DI IRUVNQLQJVSRWHQWLDOHW Sn GHVLJQXGGDQQHOVHUQH L GHW RPIDQJGHQ\DQVDWWHKDUIRUVNQLQJVHUIDULQJIUDHQSKGXGGDQQHOVH2JVnGHUIRUHUGHWYLJWLJWDW SKGXGGDQQHOVHQ SULRULWHUHV KMW Sn DUNLWHNWVNROHUQH , PRGVWQLQJ WLO .XQVWDNDGHPLHWV
$UNLWHNWVNROHKDUPDQSn$UNLWHNWVNROHQL$DUKXVKDIWOHWYHGDWUHNUXWWHUHNYDOLILFHUHGHDQVJHUH WLO SKGVWLOOLQJHU (YDOXHULQJVJUXSSHQ DQEHIDOHU .XQVWDNDGHPLHWV $UNLWHNWVNROH DW SULRULWHUH LQGVDWVHQIRUDWWLOWUNNHSKGVWXGHUHQGHLQGHQIRUGHVLJQRPUnGHWKMHUH
%HJJHDUNLWHNWVNROHUKDUYUHWJHQVWDQGIRUIRUVNQLQJVHYDOXHULQJHULOEHWDI6HQRWHV
'HW DQEHIDOHV DW GHVLJQVNROHUQH XGYLNOHU HW IRUVNQLQJVSRWHQWLDOH L VDPDUEHMGH PHG DUNLWHNWVNROHUQHVnYHOVRPPHGDQGUHXGGDQQHOVHVLQVWLWXWLRQHUKHUXQGHUXGDUEHMGHUVWUDWHJLHUIRU UHNUXWWHULQJDIIRUVNHUH'HDQVWWHOVHVRJUHVVRXUFHPVVLJHIRUXGVWQLQJHUIRUHWDEOHULQJHQDI HW IRUVNQLQJVPLOM Pn DIWDOHV PHOOHP .XOWXUPLQLVWHULHW RJ GH HQNHOWH VNROHU (QGYLGHUH EU GHU DUEHMGHVKHQLPRGDWLQGIUHHQDQHUNHQGWJUDGHIWHUIHPnUVXGGDQQHOVHPHGWLWXODWXUHQ0$L 'HVLJQKYLONHWPnVNHSDUDOOHOWPHGDWVNROHUQHXGYLNOHUGHUHVIRUVNQLQJVSROLWLN
(YDOXHULQJVJUXSSHQDQEHIDOHUDWGHUHWDEOHUHVHWODQGVGNNHQGHIRUVNQLQJVXGYDOJPHGPLGOHUIUD .XOWXUPLQLVWHULHW GHU KDU WLO RSJDYH DW IRUGHOH IRUVNQLQJVPLGOHU HIWHU NRQNUHWH DQVJQLQJHU IUD GHVLJQXGGDQQHOVHUQH +HQVLJWHQ PHG HQ VnGDQ RUGQLQJ HU DW KYH GHW JHQHUHOOH QLYHDX IRU IRUVNQLQJ LQGHQIRU GHVLJQIDJHW L DOPLQGHOLJKHG RJ L VUGHOHVKHG DW IRUEHGUH JUXQGODJHW IRU DW JUHGHVLJQVNROHUQHWLOIRUVNQLQJVLQVWLWXWLRQHU(YDOXHULQJVJUXSSHQVHUVDPWLGLJIRUVNQLQJVXGYDOJHW VRP HW QDWXUOLJW RPGUHMQLQJVSXQNW IRU HW JHW VDPDUEHMGH PHOOHP GHVLJQXGGDQQHOVHUQH L 'DQPDUN$IKHQV\QWLOVDPDUEHMGHWPHOOHPDUNLWHNWVNROHUQHRJGHVLJQVNROHUQHEUPLGOHUQHWLOHW VnGDQWIRUVNQLQJVXGYDOJLNNHWLOYHMHEULQJHVYHGDWEHVNUHGHQQXYUHQGHIRUVNQLQJVEHYLOOLQJWLO DUNLWHNWVNROHUQH
0nOVWQLQJHUIDJOLJWLQGKROGRJVWUXNWXU
(YDOXHULQJVJUXSSHQ DQEHIDOHU DW VnYHO GHVLJQVNROHUQH VRP DUNLWHNWVNROHUQH XGDUEHMGHU NODUHUH SURILOHUIRUGHUHVXGGDQQHOVHURJIRUDUNLWHNWVNROHUQHRJVnIRUGHUHVIRUVNQLQJSnGHVLJQRPUnGHW (QGYLGHUH ILQGHU HYDOXHULQJVJUXSSHQ DW HQ VWUUH JUDG DI VWUDWHJLVN EHYLGVWKHG PHG YDOJWH SULRULWHULQJHU XGIRUPQLQJ DI KDQGOLQJVSODQHU RSVWLOOLQJ DI VXFFHVNULWHULHU VDPW IDVWODJWH SURFHGXUHU IRU RSIOJQLQJ YLOOH VW\UNH GH VWUDWHJLVNH RYHUYHMHOVHU VRP HU LQGHKROGW L VHOYHYDOXHULQJVUDSSRUWHUQH
'HVLJQVNROHUQH RJ .XQVWDNDGHPLHWV $UNLWHNWVNROH NDQ HIWHU HYDOXHULQJVJUXSSHQV YXUGHULQJ KDYHJDYQDIDWSULRULWHUHDUEHMGHWPHGGHUHVPnOVWQLQJHUPHGKHQEOLNSnDWJUHGLVVHNRQNUHWH RJRSHUDWLRQHOOHVDPWDWVLNUHVDPPHQKQJHQPHOOHPGLVVHRJVWXGLHSODQHUQHPY
'HW DQEHIDOHV DW GHU VNHU HQ PnOUHWWHW VW\UNHOVH DI GH WHRUHWLVNH HOHPHQWHU L XGGDQQHOVHUQH Sn GHVLJQVNROHUQH KHUXQGHU L NXQVW RJ GHVLJQKLVWRULH VDPW ILORVRIL RJ VDPIXQGVIDJ (YDOXHULQJVJUXSSHQRSIRUGUHUGHVXGHQGHVLJQVNROHUQHWLODWVW\UNHGHWWYUIDJOLJHVDPDUEHMGH
(QGYLGHUHDQEHIDOHVGHWDW*ODV .HUDPLNVNROHQSn%RUQKROP¥MYQIUEHNHQGWJUHOVHQV¥L KMHUHJUDGSULRULWHUHU¨HQWHRUHWLVNVNROLQJ¨DIGHVWXGHUHQGHPHGKHQEOLNSnUHIOHNVLRQHURYHU IRUPJLYQLQJ VWHWLN RJ VDPIXQGVPVVLJ SURGXNWDQYHQGHOVH VnYHO VRP DQDO\WLVNH PHWRGHU L GHQQH IRUELQGHOVH (YDOXHULQJVJUXSSHQ DQEHIDOHU DW GHW L KMHUH JUDG JUHV IULYLOOLJW DW GHOWDJH L NXUVHURJIRUHOVQLQJHUIRUDWVW\UNHGHVWXGHUHQGHVVHOYVWQGLJKHGRJPXOLJKHGIRUIRUG\EHOVH
'HW DQEHIDOHV DW GHU Sn GHVLJQ RJ DUNLWHNWVNROHUQH DUEHMGHV PHG HQ WLOQUPHOVH DI VWUXNWXUHQ VRP DOOHUHGH QYQW IRU KHULJHQQHP DW OHWWH PXOLJKHGHQ IRU DW VNLIWH PHOOHP XGGDQQHOVHU RJ VNROHUVnYHOL'DQPDUNVRPLQWHUQDWLRQDOW(QVnGDQVWUXNWXUOJJHURSWLOHQPHUHNODUVNHOQHQ PHOOHPHWPHUHDOPHQWGHVLJQHOOHUDUNLWHNWIDJOLJWJUXQGQLYHDXRJHWLIRUKROGWLOIHNVPDWHULDOHU WHNQLNNHUHOOHUJHQVWDQGHPHUHVSHFLDOLVHUHWQLYHDX
6WXGLHIRUPHURJSUDNWLN
(YDOXHULQJVJUXSSHQ ILQGHU GHW ULJWLJW DW DUNLWHNW RJ GHVLJQXGGDQQHOVHUQH SULRULWHUHU SURMHNWDUEHMGVIRUPHQLNNHPLQGVWDIKHQV\QWLOGHVWXGHUHQGHVPXOLJKHGHUIRUDWNXQQHDUEHMGHL WHDPV HIWHU HQGW XGGDQQHOVH 3URMHNWRULHQWHUHW JUXSSHDUEHMGH DQYHQGHV GRJ L HW QRJHW PHUH YDULHUHQGH RPIDQJ 'HW DQEHIDOHV DW PDQ Sn DOOH XGGDQQHOVHUQH ELEHKROGHU SULRULWHULQJHQ DI SURMHNWDUEHMGVIRUPHQVDPWVLNUHUDWSURMHNWRULHQWHUHWJUXSSHDUEHMGHLQGJnUEODQGWVWXGLHIRUPHUQH VRPHWIDVWHOHPHQW
(YDOXHULQJVJUXSSHQILQGHUDWGHVLJQXGGDQQHOVHUQHVNDUDNWHUJUGHWKHQVLJWVPVVLJWDWDQYHQGH PHUHNYDOLWDWLYHEHGPPHOVHVIRUPHUHQGGHWWUDGLWLRQHOOHNDUDNWHUV\VWHP'LVVHPnGRJVDPWLGLJ KDYHHQYVHQWOLJLQIRUPDWLRQVYUGLIRUHYHQWXHOOHDIWDJHUH'HUHUIUDVNROHUQHJRGHHUIDULQJHU PHG EODQGW DQGHW VWXGLHMRXUQDOHU RJ PDQ NXQQH RYHUYHMH IHNV ZHEEDVHUHGH RJ NRQWLQXHUOLJW RSE\JJHGH ILOHU PHG GH VWXGHUHQGHV DUEHMGHU (YDOXHULQJVJUXSSHQ DQEHIDOHU DW GHVLJQXGGDQQHOVHUQH EnGH DI KHQV\Q WLO GH VWXGHUHQGH RJ DIWDJHUQH RYHUYHMHU EHGPPHOVHVIRUPHUQH KHUXQGHU VWXGLHMRXUQDOHU PHG UHJHOPVVLJH VNULIWOLJH YXUGHULQJHU DI GH VWXGHUHQGHVDUEHMGHU
'HWDQEHIDOHVDWSUDNWLNNHQJHQHUHOWSULRULWHUHVKMHUHRJIRUPDOLVHUHVPHGKHQEOLNSnDWEOLYHHW UHHOW WLOEXG WLO DOOH VWXGHUHQGH 6NROHUQH EU RSE\JJH YHGOLJHKROGH RJ LQIRUPHUH GH VWXGHUHQGH RP HW NRQWDNWQHW WLO UHOHYDQWH SUDNWLNVWHGHU 3UDNWLNNHQ EU KQJH WWWHUH VDPPHQ PHG XQGHUYLVQLQJHQ RJ EU HIWHUIOJHQGH HYDOXHUHV 3UDNWLNSODGVHU EU EHVJHV RJ JRGNHQGHV DI VNROHQRJHUIDULQJHUIUDSUDNWLNEUVDPOHVRJIRUPLGOHVWLOVWXGHUHQGHRJXQGHUYLVHUH
2SWDJHOVHVWXGLHWLGRJIUDIDOG
(YDOXHULQJVJUXSSHQILQGHUGHWKHQVLJWVPVVLJWDWGHUDQYHQGHVRSWDJHOVHVSUYHUYHGRSWDJHOVHWLO GHVLJQVNROHUQHPHQILQGHUDWNULWHULHUQHNDQJUHVPHUHNODUHIRUSRWHQWLHOOHDQVJHUQHRJPHUH PnOUHWWHGHLIRUKROGWLOXGGDQQHOVHUQHVNUDY3nDUNLWHNWXGGDQQHOVHUQHNDQRSWDJHOVHVSUYHUVLNUH DW GHU IRUWVDW OJJHV YJW Sn DQGUH IRUKROG HQG KMH NDUDNWHUHU YHG RSWDJHOVHQ (YDOXHULQJVJUXSSHQ KDU GHVXGHQ QRWHUHW VLJ DW GHU L GHQ GDJOLJH XQGHUYLVQLQJ HIWHU QRJOH In nU
-IQRWHV
WLOV\QHODGHQGHLNNHNDQLGHQWLILFHUHVIRUVNHOSnVWXGLHHJQHWKHGHQDIKQJLJDIRPPDQHURSWDJHW YLDRSWDJHOVHVSUYHHOOHUHM
6DPOHW VHW HU IUDIDOGHW Sn GHVLJQXGGDQQHOVHUQH Sn HW HIWHU HYDOXHULQJVJUXSSHQV YXUGHULQJ ULPHOLJW ODYW QLYHDX RJ VWXGLHWLGHQ HU Sn GHVLJQVNROHUQH WW Sn GH QRUPHUHGHIHPnU'HULPRG KDUDUNLWHNWXGGDQQHOVHUQHHQVWXGLHWLGGHURIWHVWRYHUVWLJHUGHQRUPHUHGHIHPnU
2UJDQLVDWRULVNHIRUKROG
*HQHUHOWVHWV\QHVGHIRUPHOOHRUJDQLVDWRULVNHVWUXNWXUHUDWYUHKHQVLJWVPVVLJHPHQGHUV\QHV GRJ DW YUH EHKRY IRU DW VNROHUQH IRUEHGUHU NRPPXQLNDWLRQHQ PHOOHP VWXGHUHQGH DQVDWWH RJ OHGHOVHLIRUKROGWLOGHPHUHRYHURUGQHGHSUREOHPVWLOOLQJHUYHGUUHQGHXGGDQQHOVHUQHVLQGKROGRJ IRUP 'HQ RYHURUGQHGH VWUXNWXU Sn .XQVWDNDGHPLHWV $UNLWHNWVNROH PHG pQ GHVLJQDIGHOLQJ RJ WL DUNLWHNWXUDIGHOLQJHUEUGRJHIWHUHYDOXHULQJVJUXSSHQVYXUGHULQJWDJHVRSWLOIRUQ\HWRYHUYHMHOVH
3n*ODV .HUDPLNVNROHQSn%RUQKROPHUGHUEHKRYIRUDWPDQLVWHGHWIRUDWEUXJHNUIWHUSnHQ Q\ VWUXNWXU IRNXVHUHU Sn KYDG GHW HU GHU JU DW PDQ LNNH NDQ In PGHUQH L VWXGLHUnGHW RJ L DIGHOLQJHUQH WLO DW IXQJHUH HIWHU KHQVLJWHQ RJ KYRUIRU GH VWXGHUHQGH LNNH HU WLOIUHGVH PHG NRPPXQLNDWLRQHQ SnVNROHQ'HUPnGHVXGHQQHGVWWHVHWVDPDUEHMGVXGYDOJRJXGDUEHMGHVHQ SHUVRQDOHSROLWLN
8QGHUYLVQLQJRJNYDOLWHWVVLNULQJ
'HW DQEHIDOHV DW VNROHUQH JHQHUHOW PHG XQGWDJHOVH DI *ODV .HUDPLNVNROHQ Sn %RUQKROP SULRULWHUHUDQVWWHOVHDIIOHUHIDVWHXQGHUYLVHUHKMHUHVnOHGHVDWHQVWUUHGHODIXQGHUYLVQLQJHQSn DOOHIDJRPUnGHUYDUHWDJHVDIIDVWDQVDWWHXQGHUYLVHUHRJVnOHGHVDWEUXJHQDIJVWHOUHUHLKMHUH JUDGKDUNDUDNWHUDIDWYUHHWYUGLIXOGWVXSSOHPHQW
8QGHUYLVHUQHPDQJOHUJHQHUHOWVHWIRUPHOOHSGDJRJLVNHNYDOLILNDWLRQHURJHUGHUIRUDIKQJLJDI SGDJRJLVN HIWHUXGGDQQHOVH 'HW DQEHIDOHV GHUIRU DW DOOH XQGHUYLVHUH Sn GHVLJQXGGDQQHOVHUQH VLNUHVOEHQGHSGDJRJLVNHIWHUXGGDQQHOVHVDPWDWPDQYHGIUHPWLGLJUHNUXWWHULQJSULRULWHUHUDW DQVWWH XQGHUYLVHUH GHU XGRYHU HW KMW IDJOLJW QLYHDX RJVn KDU GRNXPHQWHUHGH SGDJRJLVNH NYDOLILNDWLRQHU
'HWDQEHIDOHVDWNYDOLWHWVVLNULQJVPHNDQLVPHUQHLKMHUHJUDGIRUPDOLVHUHVSnGHVLJQXGGDQQHOVHUQH 'HWHUGRJHYDOXHULQJVJUXSSHQVLQGWU\NDWGHQLQWHUQHNYDOLWHWVVLNULQJSn$UNLWHNWVNROHQL$DUKXV IXQJHUHUWLOIUHGVVWLOOHQGHRJXGJUHWIXQNWLRQHOWUHGVNDEIRUVWXGLHQYQHW
%HVNIWLJHOVHVIRUKROG
6HW L O\VHW DI GHQVWRUHIRNXVGHUJHQQHPIOHUHnUKDUYUHWSnGHVLJQRPUnGHWRJGHQVWDWVOLJH VDWVQLQJ Sn RPUnGHW ILQGHU HYDOXHULQJVJUXSSHQ DUEHMGVOVKHGVWDOOHQH IRU Q\XGGDQQHGH NDQGLGDWHUDODUPHUHQGHKMH'HWDQEHIDOHVGHUIRU
$WHUKYHUYVOLYHWJHQQHPIRUVNHOOLJHIRUPHUIRUVDPDUEHMGHLKMHUHJUDGEOLYHUJMRUWRSPUNVRP SnGHNYDOLILNDWLRQHUVRPGLPLWWHQGHUQHKDU
$W VNROHUQH L IRUELQGHOVH PHG XQGHUYLVQLQJ KHUXQGHU RJVn L HUKYHUYVUHWWHGH IDJ RJ HUKYHUYVYHMOHGQLQJ JU PHUH XG DI DW IRUEHUHGH GH VWXGHUHQGH Sn DUEHMGVPDUNHGHW RJ HUKYHUYVOLYHW
$WGHUVWWHVPHUHIRNXVSnKROGQLQJHULUHODWLRQWLOGHYLONnUGHUHNVLVWHUHUSnDUEHMGVPDUNHGHWRJ PXOLJKHGHUQHIRUDWIRUHQHHWDNWXHOWOQDUEHMGHPHGQVNHWRPHQGDJDWEOLYHVHOYVWQGLJHOOHU DWIn¨QVNHMREEHW¨
$W GHU XGDUEHMGHV RJLYUNVWWHVKDQGOLQJVSODQHUSnGHHQNHOWHVNROHUPHGKHQEOLNSnDWJH EHVNIWLJHOVHQIRUGLPLWWHQGHUQH
,QWHUQDWLRQDOLVHULQJ
,HYDOXHULQJVUDSSRUWHQVHVGHUQUPHUHSnHQUNNHLQGLNDWRUHUIRULQWHUQDWLRQDOLVHULQJRJGHW IUHPJnU DW GHW HU PHUH DOPLQGHOLJW DW XGHQODQGVNH VWXGHUHQGH RSKROGHU VLJ YHG GH GDQVNH GHVLJQXGGDQQHOVHUHQGRPYHQGW'HUQVWVHVGHWPRGVDWWHIRUKROGIRUOUHUQHVYHGNRPPHQGH 9HG DUNLWHNWXGGDQQHOVHUQH HU GHW DOPLQGHOLJW DW OUHUQH KDU OQJHUHYDUHQGH RSKROG L XGODQGHW PHQV GHW NXQ HU YHG $UNLWHNWVNROHQ L $DUKXV DW PDQ DQYHQGHU JVWHOUHUH L OQJHUHYDUHQGH SHULRGHU
(YDOXHULQJVJUXSSHQDQEHIDOHUSnGHQEDJJUXQGDWGHVLJQXGGDQQHOVHUQHVW\UNHUGHQLQWHUQDWLRQDOH XGYHNVOLQJ DI XQGHUYLVHUH RJ VWXGHUHQGH 6nYHO GHVLJQVNROHUQH VRP DUNLWHNWVNROHUQH Pn L VWUUH JUDG HQG KLGWLO SULRULWHUH DQYHQGHOVHQ DI JVWHOUHUH IUD XGODQGHW RJ VJH DW JH DQWDOOHW DI VWXGHUHQGHVRPLHQSHULRGHOVHULXGODQGHW
$QDO\VHRJDQEHIDOLQJHU
(YDOXHULQJVRJVHOYHYDOXHULQJVSURFHVVHQ
(YDOXHULQJHQ DI GHVLJQXGGDQQHOVHUQH XQGHU .XOWXUPLQLVWHULHW EOHY LJDQJVDW L IRUnUHW PHG SODQODJWDIVOXWQLQJVHQHVWGHQPDUWV3nEDJJUXQGDIHWQVNHIUDGHVLJQXGGDQQHOVHUQH RP IRUOQJHOVH DI VHOYHYDOXHULQJVSURFHVVHQ PHG DIOHYHULQJ DI VHOYHYDOXHULQJVUDSSRUWHUQH GHQ GHFHPEHUEOHYHYDOXHULQJHQVDIVOXWQLQJXGVNXGWWLOGHQPDM
(YDOXHULQJHQ DI GHVLJQXGGDQQHOVHUQH KDU YUHW SUJHW DI HQ UNNH IRUKROG VRP NRUW VNDO RSVXPPHUHV
• 'H IHP XGGDQQHOVHUV VWRUH IRUVNHOOLJKHG RJ GHUPHG HYDOXHULQJVRPUnGHWV EHW\GHOLJH KHWHURJHQLWHW )UD VNROHU PHG XGVSULQJ L KnQGYUNHW RJ WLO DUNLWHNWVNROHUQH PHG GHUHV PHUH WHRUHWLVNHRJIRUVNQLQJVEDVHUHGHWLOJDQJ
• (QYLVHYDOXHULQJVWUWKHGSnDUNLWHNWVNROHUQHVRPIOJHDIWLGOLJHUHHYDOXHULQJHU'HQQHWUWKHG RSOHYHGHHYDOXHULQJVJUXSSHQW\GHOLJVWSn.XQVWDNDGHPLHWV$UNLWHNWVNROH
• *ODV .HUDPLNVNROHQSn%RUQKROPHUVnQ\DWGHQHQGQXLNNHKDUXGGDQQHWGLPLWWHQGHU
• 'H VWRUH QGULQJHU VW\ULQJVPVVLJW RUJDQLVDWRULVN RJ LQGKROGVPVVLJW Sn GH WR GHVLJQVNROHU VDPW.XQVWDNDGHPLHWV$UNLWHNWVNROHLQGHQIRUGHVLGVWHSDUnU
• )RU WUH DI VNROHUQHV YHGNRPPHQGH SUREOHPDWLVN LQGGUDJHOVH DI GH VWXGHUHQGH L VHOYHYDOXHULQJVSURFHVVHQ 3n .XQVWDNDGHPLHWV $UNLWHNWVNROH GHOWRJ GH VWXGHUHQGH LNNH Sn
$UNLWHNWVNROHQL$DUKXVGHOWRJGHVWXGHUHQGHLNNHDNWLYWRJSn*ODV .HUDPLNVNROHQSn%RUQKROP YDUGHVWXGHUHQGHXWLOIUHGVHPHGIRUOEHWDIVHOYHYDOXHULQJVSURFHVVHQ
(YDOXHULQJVFHQWHUHWIRUGH9LGHUHJnHQGH8GGDQQHOVHUVWRGLIRUHQHYDOXHULQJDIDUNLWHNWXGGDQQHOVHQL
$DUKXV RJ L IRU HQ HYDOXHULQJ DI .XQVWDNDGHPLHWV $UNLWHNWVNROH , EHJJH WLOIOGH PHG .XOWXUPLQLVWHULHW VRP RSGUDJVJLYHU 'HVXGHQ HU IRUVNQLQJHQ YHG .XQVWDNDGHPLHWV $UNLWHNWVNROH EOHYHW HYDOXHUHW L EHJ\QGHOVHQ DI RJIRUVNQLQJHQYHG$UNLWHNWVNROHQL$DUKXVEOHYHYDOXHUHWL
(QYVHQWOLJGHODIHQHYDOXHULQJVRPGHQQHHUVHOYHYDOXHULQJVSURFHVVHQ'HQWLOYHMHEULQJHULNNH EORW HW GRNXPHQWDWLRQVJUXQGODJ IRU HYDOXHULQJHQ L IRUP DI HQ VHOYHYDOXHULQJVUDSSRUW PHQ SURFHVVHQWMHQHURJVnVRPHWYLJWLJWUHGVNDEWLONYDOLWHWVXGYLNOLQJSnGHQHQNHOWHLQVWLWXWLRQ6NDO VHOYHYDOXHULQJVSURFHVVHQ KDYH NRQVHNYHQVHU L GDJOLJGDJHQ HU GHW GHUIRU HQ YVHQWOLJ IRUXGVWQLQJDWDOOHJUXSSHUDISHUVRQDOHRJVWXGHUHQGHLQGGUDJHVLSURFHVVHQ
'HW Pn NRQVWDWHUHV DW VNROHUQH KDU JUHEHW VHOYHYDOXHULQJVSURFHVVHQ PHJHW IRUVNHOOLJW DQ LNNH PLQGVW L IRUKROG WLO KYRU EUHG LQGGUDJHOVHQ DI SHUVRQDOH RJ VWXGHUHQGH KDU YUHW 6RP WR
\GHUOLJKHGHU L GHQQH KHQVHHQGH NDQ QYQHV Sn GHQ HQH VLGH 'DQPDUNV 'HVLJQVNROH GHU KDU VDWVHWSnHQPHJHWEUHGLQGGUDJHOVHDIVnYHOSHUVRQDOHVRPVWXGHUHQGHLKHOHSURFHVVHQRJSn GHQDQGHQVLGH.XQVWDNDGHPLHWV$UNLWHNWVNROHGHULNNHKDULQGGUDJHWGHVWXGHUHQGHLSURFHVVHQ 3n GHQ EDJJUXQG YDU GHW NQDSW Vn RYHUUDVNHQGH DW GH VWXGHUHQGH Sn 'DQPDUNV 'HVLJQVNROH PHQWH DW VHOYHYDOXHULQJVUDSSRUWHQ JDY HW SUFLVW RJ I\OGHVWJUHQGH ELOOHGH DI VNROHQ PHQV GH VWXGHUHQGH Sn .XQVWDNDGHPLHWV $UNLWHNWVNROH KDYGH VYUW YHG DW IRUKROGH VLJ WLO GHUHV VNROHV VHOYHYDOXHULQJVUDSSRUW
)OJHJUXSSHQ Sn 'DQPDUNV 'HVLJQVNROH KDU EHVWnHW DI W\YH SHUVRQHU UHSUVHQWHUHQGH VnYHO DQVDWWHOHGHOVHVRPVWXGHUHQGH+HUXGDIEOHYHQUHGDNWLRQVJUXSSHSnIHPSHUVRQHUQHGVDWPHG GHW IRUPnO DW VNULYH VHOYHYDOXHULQJVUDSSRUWHQ 6HOYH VHOYHYDOXHULQJVSURFHVVHQ EHJ\QGWH PHG VXVSHQVLRQ DI DO XQGHUYLVQLQJ PDQGDJ GHQ VHSWHPEHU IRU DW VWXGHUHQGH RJ DQVDWWH NXQQH DIKROGH VWRUPGH YHGUUHQGH HYDOXHULQJHQ L &LUNXVE\JQLQJHQ L .EHQKDYQ +HUHIWHU IRUWVDWWH JUXSSHU DI DQVDWWH RJ VWXGHUHQGH PHG DW GUIWH VSUJVPnO IUD YHMOHGQLQJHQ WLO VHOYHYDOXHULQJHQ 5HGDNWLRQVJUXSSHQ KDU Sn EDJJUXQG DI GLVVH GLVNXVVLRQHU RJ IHP DIKROGWH PGHU L IOJHJUXSSHQ VNUHYHW VHOYHYDOXHULQJVUDSSRUWHQ GHU HU XGGHOW WLO VDPWOLJH DQVDWWH RJ VWXGHUHQGH 6HOYHYDOXHULQJVUDSSRUWHQ IUD 'DQPDUNV 'HVLJQVNROH HU HQ JUXQGLJ RJ JHQQHPEHDUEHMGHWUDSSRUWGHUDQDO\VHUHUEnGHVYDJHRJVWUNHVLGHUYHGXGGDQQHOVHQ
)OJHJUXSSHQ Sn 'HVLJQVNROHQ L .ROGLQJ KDU EHVWnHW DI HOOHYH SHUVRQHU UHSUVHQWHUHQGH VnYHO VWXGHUHQGH DQVDWWH VRP OHGHOVH )OJHJUXSSHQ KDU L VHOYHYDOXHULQJVUDSSRUWHQ YDOJW HQ JUXSSHRSGHOW IUHPVWLOOLQJ GHU HU HQ RSVXPPHULQJ DI GH IRUVNHOOLJH JUXSSHUV KROGQLQJHU YHGUUHQGH RPUnGHW KYLONHW JU GHW YDQVNHOLJW IRU XGHQIRUVWnHQGH DW LGHQWLILFHUH XGVDJQHQHV UHSUVHQWDWLYLWHW%HVJHWHIWHUORGGRJGHWLQGWU\NDWUDSSRUWHQGHUHUXGGHOWWLODOOHSnVNROHQ JDY HW GNNHQGH ELOOHGH DI XGGDQQHOVHQ 6HOYHYDOXHULQJVUDSSRUWHQ IUHPVWnU GHUIRU VRP QRJHW XIUGLJ RJ GHU PDQJOHU VDPOHGH YXUGHULQJHU RJ UHIOHNVLRQHU VRP NXQQH KDYH JHW GHQV LQIRUPDWLRQVYUGL
$UNLWHNWVNROHQL$DUKXVQHGVDWWHHQIOJHJUXSSHSnVHNVSHUVRQHUEHVWnHQGHDIVnYHODQVDWWHVRP VWXGHUHQGH 'H WR UHSUVHQWDQWHU IRU GH VWXGHUHQGH KDU LNNH NXQQHW DIVH WLG WLO DW GHOWDJH L
IOJHJUXSSHQV PGHU PHQ HU OEHQGH EOHYHW RULHQWHUHW RP VHOYHYDOXHULQJVUDSSRUWHQV LQGKROG )OJHJUXSSHQ KDU Sn EDJJUXQG DI PGHUPHGXQGHUYLVHUUHSUVHQWDQWHUIUDGHVLJQDIGHOLQJHUQH VNUHYHW UDSSRUWHQ 6HOYHYDOXHULQJVUDSSRUWHQ GHU HU XGOHYHUHW WLO DOOH Sn VNROHQ HU HQ JUXQGLJ UDSSRUWGHUDQDO\VHUHUEnGHVYDJHRJVWUNHVLGHUYHGXGGDQQHOVHQ
)OJHJUXSSHQ Sn .XQVWDNDGHPLHWV $UNLWHNWVNROH EHVWRG DI VHNV SHUVRQHU IUD XQGHUYLVHUJUXSSHQ GHU KDU VWnHW IRU XGIRUPQLQJHQ DI VHOYHYDOXHULQJVUDSSRUWHQ 6HOYHYDOXHULQJVUDSSRUWHQ IUD .XQVWDNDGHPLHWV $UNLWHNWVNROH YDU PHJHW JHQHUHO RJ LNNH WLOVWUNNHOLJ SUFLV L IRUKROG WLO GH GHVLJQIDJOLJHSUREOHPVWLOOLQJHU'HWWHEOHYGHUWLOHQYLVJUDGHIWHUIOJHQGHUHWWHWRSSnIRUVnYLGW DQJnU YLVLRQHU RP XGYLNOLQJHQ DI GHQ XGGDQQHOVHQV IDJOLJH SURILO 3n EHVJHW JDY VnYHO XQGHUYLVHUH VRP VWXGHUHQGH XGWU\N IRU DW GH LNNH KDYGH YUHW LQGGUDJHW L VHOYHYDOXHULQJVSURFHVVHQ
3n *ODV .HUDPLNVNROHQ Sn %RUQKROP EHVWRG IOJHJUXSSHQ DI V\Y SHUVRQHU UHSUVHQWHUHQGH VnYHO DQVDWWH OHGHOVH VRP VWXGHUHQGH )OJHJUXSSHQ KDU L IOOHVVNDE VWnHW IRU XGDUEHMGHOVHQ DI VHOYHYDOXHULQJVUDSSRUWHQ 6HOYHYDOXHULQJVUDSSRUWHQ DQDO\VHUHU L QRJHQ JUDG GH VYDJH RJ VWUNH VLGHUYHGXGGDQQHOVHQLGHWRPIDQJGHWJUXQGHWVNROHQVDOGHUHUPXOLJW
'H VWXGHUHQGH JDY L IRUELQGHOVH PHG EHVJHW XGWU\N IRU DW GH L IRU ULQJH RPIDQJ KDYGH YUHW LQGGUDJHWLSURFHVVHQRJHQGGDKDYGHKDIWSUREOHPHUPHGDWInDOPLQGHOLJDGJDQJWLOGHQIUGLJH UDSSRUW'HVWXGHUHQGHGHUKDYGHOVWUDSSRUWHQPHQWHLNNHDWGHQJDYHWUHWYLVHQGHELOOHGHDI VNROHQ
'HVLJQEHJUHEHW
'HUILQGHVIRUVNHOOLJHGHILQLWLRQHUDIEHJUHEHWGHVLJQ(QDOPLQGHOLJDQYHQGWGHILQLWLRQVRPVYDUHU WLOGHQHQJHOVNHEHW\GQLQJDIRUGHWHU¨DWJLYHIRUP¨(QDQGHQRJODQJWPHUHVQYHUGHILQLWLRQ PDQ NDQ PGH HU ¨IRUPJLYQLQJ DI LQGXVWULHOW SURGXFHUHGH YDUHU¨ RJVn NDOGHW SURGXNWGHVLJQ , UHJHULQJHQV 'HVLJQUHGHJUHOVH NDQ PDQ OVH DW ¨'HVLJQ HU XGWU\N IRU HQ NUHDWLY SURFHV GHU LQWHJUHUHU HW SURGXNWV I\VLVNH HJHQVNDEHU PHG VWHWLVNH KHQV\Q¨ 0HQV PDQ Sn DUNLWHNWXGGDQQHOVHUQH NDQ PGH GHQ RSIDWWHOVH DW GHVLJQ HU ¨HW DVSHNW DI DUNLWHNWXUHQ GHU
'DQPDUNV (YDOXHULQJVLQVWLWXW PRGWRJ VLGVW L MDQXDU HW EUHY IUD SURIHVVRU $QGHUV %UL[ 3HGHUVHQ KYRU GH QUPHUHKHQVLJWHUIRU$IGHOLQJHQIRUGHVLJQEOLYHUSUVHQWHUHW(IWHUIOJHQGHHUGHWEOHYSUFLVHUHWDWGHULNNH NXQHUWDOHRPHWLQGLYLGXHOWELGUDJPHQDWLQGKROGHWGNNHUDIGHOLQJHQVV\QVSXQNWHU
(UKYHUYVPLQLVWHULHW§5HJHULQJHQV'HVLJQUHGHJUHOVH¨V
SULRULWHUHU W\SHQ IUHP IRU VWHGHW¨ 1REHOSULVWDJHUHQ +HUEHUW 6LPRQ KDU VDPPHQOLJQHW GHVLJQIDJHW PHG RJ GHUPHG DIJUQVHW GHW IUD QDWXUYLGHQVNDEHUQH YHG DW SnSHJH DW KYRU GLVVH
¨DUHFRQFHUQHGZLWKKRZWKLQJVDUH¨VnHUGHVLJQ¨FRQFHUQHGZLWKKRZWKLQJVRXJKWWREHZLWK GHYLVLQJDUWLIDFWVWRDWWDLQJRDOV¨
(YDOXHULQJVJUXSSHQQVNHULNNHDWOJJHVLJIDVWSnHQEHVWHPWGHILQLWLRQPHQYLOEORWSUFLVHUH DW GHU HU PHJHW IRUVNHOOLJH RSIDWWHOVHU DI GHVLJQEHJUHEHW 'RJ V\QHV GHW NODUWDWGHVLJQKDQGOHU RPPnOUHWWHWIRUPJLYQLQJPHGVnYHOI\VLVNHVRPVWHWLVNHDVSHNWHU
.XOWXUPLQLVWHULHW RJ 0LOM RJ (QHUJLPLQLVWHULHW QHGVDWWH L GHF HW XGYDOJ WLO VLNULQJ DI GHW WUDGLWLRQHOOHKnQGYUN,UDSSRUWHQIUHPKYHVGHWDW¨KnQGYUNVIDJHQHVNDOSUJHXGYLNOLQJHQ DIGDQVNGHVLJQLQGHQIRUUHOHYDQWHRPUnGHU¨RJDW¨PHJHWGHVLJQKYLOHUSnKnQGYUNVPVVLJH IUGLJKHGHU¨(QGHOLJILQGHUXGYDOJHW¨DWKnQGYUNVHOHPHQWHWVNDOLQGJnVRPSDUDPHWHULGHQ HYDOXHULQJHQDIGHVLJQXGGDQQHOVHUQH¨
(YDOXHULQJVJUXSSHQHUHQLJLDWKnQGYUNHWHUHQYVHQWOLJGHODIGHVLJQIDJHWPHQGHWHUOLJHVn YLJWLJWDWSRLQWHUHDWGHVLJQGHOVRJVnKDUDQGUHUGGHUGHOVHUHWIDJPHGHQVHOYVWQGLJSURILO 'HW YDU VnOHGHV L IRUELQGHOVH PHG EHVJHQH Sn VNROHUQH VUGHOHV W\GHOLJW DW NRQVWDWHUH GHVLJQIDJHWVRSKDYLKKYKnQGYUNHWRJDUNLWHNWIDJHWPHQVGHWWLOV\QHODGHQGHNQLEHUPHUHPHG DWDFFHSWHUHGHVLJQVRPHWIDJPHGHQVHOYVWQGLJSODWIRUP
+YRUPDQSnDUNLWHNWVNROHUQHVHUGHVLJQVRPHQSURFHVGHUIRUHJnUUHODWHUHWWLOUXPVnEHWRQHUGH VNROHU GHU KDU VROLGH UGGHU L KnQGYUNHW PDWHULDOHUQH RJ KnQGYUNHW VRP GHW DIJUHQGH XGJDQJVSXQNWIRUIRUPJLYQLQJRJSURGXNWXGYLNOLQJ6DPWLGLJEHILQGHUGLVVHXGGDQQHOVHUVLJLHW VDPIXQG KYRU GHU VNHU HQ VWDGLJW KXUWLJHUH IRUDQGULQJ L V\QHW Sn KYDG HW SURGXNW HU KYDG GHU VNDEHU YUGL KRV NXQGHUQH RJ KYRUGDQ GH EDJYHGOLJJHQGH SURFHVVHU VNDO XGIRUPHV 'HWWH VNDO VDPPHQKROGHVPHGDWVWDGLJIOHUHLQGXVWULSURGXNWHULGDJSnGHQHQHVLGHIUHPVWLOOHVLDQGUH PDWHULDOHU HQG GHP VRP NXQVWKnQGYUNHUQH QRUPDOW DUEHMGHU PHG RJ Sn GHQ DQGHQ VLGH IRUEUXJHVHOOHUDQYHQGHVLHWPHUHGLIIXVWUXPMIGHWJOREDOHPDUNHG'HWWHXGJULNNHPLQGVWHQ XGIRUGULQJIRUGHVLJQHUHIRUKYHPGHWDWIRUVWnRJGHWDWYUHPHGWLODWXGIRUPHSURGXNWHUWLOHQ IUHPWLGLJXVLNNHUNRQWHNVWIRUYHQWHVDWYUHHQNHUQHNRPSHWHQFH'HWWHIRUGUHUEODQGWDQGHWDW GHVLJQHUHQ TXD VLQ XGGDQQHOVH RSQnU YLGHQ LQGHQIRU RPUnGHU VRP SURGXNWLRQV RJ SUVHQWDWLRQVWHNQLNNHU
6HOYHYDOXHULQJVUDSSRUWHQIUD.XQVWDNDGHPLHWV$UNLWHNWVNROHVIUHPKYHUGHQQHGHILQLWLRQ
7KH6FLHQFHVRIWKH$UWLILFLDOS
6W\UNHOVHDIGHWJRGHKnQGYUN¥,GHHURJIRUVODJ
'RV
8GIUDEHVJHQHSnVNROHUQHDWGPPHYLUNHGHGHWVRPRPDWPDQNXQLEHJUQVHWRPIDQJKDU DUEHMGHWPHGVWUDWHJLHUIRUIRUDQGULQJHULGHVLJQSURIHVVLRQHQ2JLGHQQHIRUELQGHOVHV\QWHVGHW OLJHOHGHVDWYUHHWJHQHUHOWSUREOHPDWLQIRUPDWLRQVWHNQRORJLSnVNROHUQHLKMHUHJUDGRSIDWWHV DOHQH VRP HW Q\W UHGVNDE L JDPOH SURFHVVHU IUHP IRU QRJHW GHU RJVn LQGHEUHU IRUDQGULQJHU L GHVLJQSURFHVVHUQH)RUGHVLJQHUHQLQGHEUHU,7KHOWQ\HPXOLJKHGHULIRUKROGWLOLQWHUDNWLYLWHWRJ IXQNWLRQDOLWHW (QGYLGHUH InU GH VWXGHUHQGH PHG ,QWHUQHWWHW WLOVYDUHQGH Q\H PXOLJKHGHU IRU DW DUEHMGHVDPPHQRJNRPPXQLNHUH
)DJOLJSURILOYLVLRQHURJSULRULWHULQJHU
)RNXV L GHWWH DIVQLW YLO YUH Sn GH IRUVNHOOLJH XGGDQQHOVHVLQVWLWXWLRQHUV IDJOLJH SURILO RJ Sn KYRUOHGHV GLVVH XGPQWHU VLJ L YLVLRQHU RJ SULRULWHULQJHU IRU GHQ SnJOGHQGH XGGDQQHOVHVLQVWLWXWLRQ
'DQPDUNV'HVLJQVNROH
'DQPDUNV 'HVLJQVNROH RSVWRG VRP IOJH DI HQ VDPPHQOJQLQJ L DI 6NROHQ IRU
%ROLJLQGUHWQLQJRJ6NROHQIRU%UXJVNXQVWKYRUVLGVWQYQWHKDUUGGHUWLOEDJHWLOEOD7HJQHVNROHQ IRU .YLQGHU RJ.XQVWKnQGYUNHUVNROHQ'DQPDUNV'HVLJQVNROHKDUVnOHGHVVROLGH UGGHU L HQ KnQGYUNVPVVLJ WUDGLWLRQ KYLONHW GHQ QVNHU DW KROGH IDVW L VDPWLGLJ PHG DW GHVLJQIDJHWV WHRUHWLVNH RJ PHWRGLVNH DVSHNWHU SULRULWHUHV KMHUH EOD JHQQHP NXQVWQHULVN XGYLNOLQJVDUEHMGHRJIRUVNQLQJ
0HG GHQ Q\H UHNWRU VRP HQ GULYNUDIW KDU PDQ Sn VNROHQ IUD VLGVWH GHO DI VDPWLGLJ PHG VHOYHYDOXHULQJVSURFHVVHQLJDQJVDWHQIRUDQGULQJVSURFHVPHGEHW\GHOLJEUHGGHRJG\EGH0nOHQH IRU GHQQH SURFHV HU DW VNUSH XGGDQQHOVHQV IDJOLJH RJ WHRUHWLVNH SURILO DW IRUEHGUH XQGHUYLVQLQJHQV SGDJRJLVNH NYDOLWHW RJ DW GHPRNUDWLVHUH EHVOXWQLQJVSURFHVVHUQH 'HU HU LYUNVDW HW RPIDWWHQGH VWUDWHJLVN DUEHMGH VRP EODQGW DQGHW L LQGHYUHQGH nU VNDO PXQGH XG L ILUHnULJHKDQGOLQJVSODQHU
'DQPDUNV 'HVLJQVNROH VHU GHW VRP VLQ PLVVLRQ ¨DW JLYH GH VWXGHUHQGH NYDOLILNDWLRQHU LQGHQ IRU GHVLJQRPUnGHUQH 5XP 3URGXNW RJ .RPPXQLNDWLRQ PHG KHQEOLN Sn SURIHVVLRQHO EHVNIWLJHOVH LQGHQ IRU GLVVH RPUnGHU 'HVXGHQ KDU 'DQPDUNV 'HVLJQVNROH VRP YLGHUHJnHQGH XGGDQQHOVH WLO RSJDYH DW IUHPPH XGYLNOLQJ RJ IRUVNQLQJ LQGHQ IRU GHVLJQ¨ 0nOHW HU VnOHGHV NODUW 'DQPDUNV
*VWD.QXGVHQWLOWUnGWHVLQVWLOOLQJSUGHQDXJXVW
'HVLJQVNROH QVNHU DW XGYLNOH VLJ WLO HQ YLGHUHJnHQGH XGGDQQHOVHVLQVWLWXWLRQ PHG IRUVNQLQJ IRU GHUYHGDWYUHQDWLRQDOWRJLQWHUQDWLRQDOWNRQNXUUHQFHG\JWLJRJIRUGHUYHGDWNXQQHDUEHMGHSn VDPPHYLONnUVRPVLQHVDPDUEHMGVSDUWQHUH
6NROHQ KHU HQGQX LQJHQ NODU GHVLJQSURILO PHQ GHQ QUH NRQWDNW WLO PDWHULDOHU RJ GH JRGH YUNVWHGHUDQVHVVRPHQVW\UNHYHGXGGDQQHOVHQ
(Q UHNUXWWHULQJ DI IRUVNHUH RJ HW VWUUH VDPDUEHMGH PHG IRUVNQLQJVPLOMHU HU HIWHU HYDOXHULQJVJUXSSHQV YXUGHULQJ HQ IRUXGVWQLQJ IRU HQ PHUH SHUPDQHQW SULRULWHULQJ DI GHW WHRUHWLVNHRJDQDO\WLVNHDUEHMGH'HQPDQJOHQGHIRUVNQLQJVPVVLJHVWDWXVJLYHUSUREOHPHURJVnL IRUKROG WLO DQGUH GHVLJQVNROHU L XGODQGHW KHUXQGHU PHULWPVVLJH SUREOHPHU IRU GH VWXGHUHQGH 6DPDUEHMGHW PHG RJ NRQWDNWHQ WLO HUKYHUYVOLYHW V\QHV DW YUH UHODWLYW XXGYLNOHW KYLONHW HIWHU HYDOXHULQJVJUXSSHQVYXUGHULQJNDQYUHHQGHODIIRUNODULQJHQSnGHQVWRUHDUEHMGVOVKHGEODQGW GLPLWWHQGHUQH
8QGHU EHVJHW Sn 'DQPDUNV 'HVLJQVNROH ILN HYDOXHULQJVJUXSSHQ HQ UNNH IRUVNHOOLJH VYDU Sn VSUJVPnOHW RP KYLONH RSJDYHU GHU LVU VNXOOH SULRULWHUHV L GHQ QUPHVWH IUHPWLG , XSULRULWHUHW UNNHIOJHEOHYIOJHQGHIUHPKYHW
• %HKRYHW IRU VWUXNWXUQGULQJHU KYRU PDQ EOD KDYGH RYHUYHMHW HQ PRGHO KYRU PDQ VDPOHGH LQGKROGHW L GHW QXYUHQGH EDVLVPRGXO PHG IOOHVODERUDWRULHW PHG KHQEOLN Sn DW HWDEOHUH HQ
¨VWDPPH¨ LJHQQHP KHOH VWXGLHW WLO VLNULQJ DI GHW WHRUHWLVNH PHWRGLVNH RJ NRPPXQLNDWLRQVPVVLJHIOOHVVWRI
• %HKRYHWIRUPHUHIRUVNQLQJRJNXQVWQHULVNXGYLNOLQJVDUEHMGHWWSnGHHQNHOWHIDJRPUnGHUPHG GHWSHUVSHNWLYDW'DQPDUNV'HVLJQVNROHEOLYHUHQIRUVNQLQJVLQVWLWXWLRQGRJXGHQDWGHWWHEOLYHUSn EHNRVWQLQJDIKnQGYUNHW'HWWHVHVVRPHQIRUXGVWQLQJIRUDWWHRULGDQQHOVHRJYLGHQVXGYLNOLQJ NDQILQGHVWHGVRPJUXQGODJIRUXQGHUYLVQLQJHQ
• 6WUUHLQGGUDJHOVHDIRJEHGUHNRPPXQLNDWLRQPHOOHPDOOHJUXSSHUDISHUVRQDOHRJVWXGHUHQGHSn VNROHQ
• (QXGYLGHWDQYHQGHOVHDILQIRUPDWLRQVWHNQRORJL
• (WJHWVDPDUEHMGHPHGHUKYHUYVOLYHW
)OOHVODERUDWRULHW YDUHWDJHU EODQGW DQGHW GHQ WYUIDJOLJH XQGHUYLVQLQJ VDPW DIKROGHU NXUVHU LQGHQ IRU PHUH WHRULSUJHGHIDJVRPILORVRILNXQVWRJGHVLJQKLVWRULH-YQIUDIVQLWRPGHWIDJOLJHLQGKROG
+HUWLONRPPHUGHSUFLVHULQJHUVRPIUHPJnUDIVHOYHYDOXHULQJVUDSSRUWHQPHGKHQV\QWLOEHKRYHW IRU HQ IRUEHGULQJ DI OQ RJ DQVWWHOVHVIRUKROG VnOHGHV DW PDQ NDQ WLOWUNNH NYDOLILFHUHW DUEHMGVNUDIW VDPW XGYLNOLQJ DI HQ SHUVRQDOHSROLWLN GHU VLJWHU PRG HIWHUXGGDQQHOVH RJ HW JRGW DUEHMGVPLOM6NROHQQVNHULGHQIRUELQGHOVHDWGHULQGOHGHVIRUKDQGOLQJHUPHG.XOWXUPLQLVWHULHW PHGKHQEOLNSnLQGJnHOVHDIHQUHVXOWDWNRQWUDNW
(YDOXHULQJVJUXSSHQHUHQLJPHGVNROHQLSULRULWHULQJHQDIGHRYHQIRUDQIUWHPnOLGHWGHWGRJPn EHPUNHV DW HQ SULRULWHULQJ DI IRUVNQLQJHQ YLO KDYH NRQVHNYHQVHU IRU EDODQFHQ PHOOHP WHRUL RJ SUDNVLV
'HW DQEHIDOHV DW GHW VWUDWHJLVNH DUEHMGH IDVWKROGHV RJ DW GHU L IRUELQGHOVH PHG GH SODQODJWH KDQGOLQJVSODQHUIRUPXOHUHVNODUHVXFFHVNULWHULHUVDPWSURFHGXUHUIRURSIOJQLQJ
'HVLJQVNROHQL.ROGLQJ
, MDQXDU EOHY GHQ GDYUHQGH .XQVWKnQGYUNHUVNROH XQGHU .ROGLQJ 7HNQLVNH 6NROH HQ VHOYVWQGLJVHOYHMHQGHLQVWLWXWLRQPHGHJHQUHNWRUEHVW\UHOVHRJYHGWJWHU6NROHQOJJHUYJW Sn GHW PDWHULDOH RJ KnQGYUNVPVVLJH XGJDQJVSXQNW IRU GHVLJQXGGDQQHOVHQ RJ SULRULWHUHU VDPVSLOOHWPHGHUKYHUYVOLYHWRJGHWYULJHVDPIXQGKMW0DQRSIDWWHUGHVLJQVRPIRUPJLYQLQJDI PDWHULDOHU RJ VLJWHU JHQHUHOW PRG GHW SHUVRQOLJH XGWU\N Sn HW KMW NXQVWQHULVN QLYHDX 6NROHQ WLOVWUEHUDWIRUHQHGHWIDJVSHFLILNNHXGJDQJVSXQNWPHGWYUIDJOLJWVDPDUEHMGHRJOJJHURSWLO DWGHVWXGHUHQGHNDQYOJHDWRULHQWHUHVLJLUHWQLQJDIVnYHOXQLNDVRPLQGXVWULHOSURGXNWLRQ, IRUKROG WLO WLGOLJHUH V\QHV GHW KnQGYUNVPVVLJH GRJ DW YUH EOHYHW QHGWRQHW WLO IRUGHO IRU HQ YJWQLQJDIGHWSHUVRQOLJHGHVLJQPVVLJHXGWU\N
6DPWLGLJHUVNROHQVLJPHJHWEHYLGVWDWXGGDQQHOVHQSnGHQHQHVLGHVWnUPHGHQVUOLJNXOWXU YLGHQRJNXQQHQVRPGHWHUYLJWLJWDWEHYDUHRJYLGHUHXGYLNOHRJSnGHQDQGHQVLGHPHGQRJOH
QGUHGH YLONnU KYRU GHW EOLYHU YLJWLJW DW YOJH GH FHQWUDOH YUGLHU XG RJ NREOH GHP WLO GH Q\H EHKRYRJIRUXGVWQLQJHULHQIRUWOEHQGHXGYLNOLQJVSURFHV
*HQHUHOWV\QHVGHUDWYUHHQJRGRYHUHQVVWHPPHOVHPHOOHPLQIRUPDWLRQRPXGGDQQHOVHQRJGHQ GDJOLJH XQGHUYLVQLQJ PHQ GHW HU GRJ XKHOGLJW DW VNROHQ NDOGHU HW IDJRPUnGH IRU ¨NHUDPLN RJ JODV¨RJGHUPHGXGDGWLOVLJQDOHUHUDWPDQKDUHQJODVDIGHOLQJQnUGHQVWXGHUHQGHLNNHNDQWDJH DIJDQJLJODVPHQNXQNDQLQGGUDJHJODVVRPHWVXSSOHPHQWWLONHUDPLNXQGHUYLVQLQJHQ
0HGKHQV\QWLOIDJRPUnGHUMYQIUDIVQLW
0DQ EU GHUIRU HIWHU HYDOXHULQJVJUXSSHQV PHQLQJ QGUH QDYQHW Sn ¨$IGHOLQJ IRU .HUDPLN RJ JODV¨WLOGHWLIRUKROGWLOYLUNHOLJKHGHQSnVNROHQPHUHNRUUHNWHQDYQ¨$IGHOLQJIRU.HUDPLN¨
6NROHQVPDQJOHQGHIRUVNQLQJVPVVLJHVWDWXVRSOHYHVVRPHWSUREOHPSnVNROHQEODLIRUKROGWLO DQGUHHXURSLVNHVNROHUKHUXQGHULIRUELQGHOVHPHGGHVWXGHUHQGHVPHULWPXOLJKHGHU0HQGHWHU LIOJH HYDOXHULQJVJUXSSHQ HW nEHQW VSUJVPnO KYRUGDQ IRUVNQLQJ L JLYHW IDOG NDQ IRUHQHV PHG VNROHQV KnQGYUNVPVVLJH XGJDQJVSXQNW RJ PHGDUEHMGHUVWDEHQV QXYUHQGH IDJOLJH IRUXGVWQLQJHU
6LGHQ WLOE\GHV,QGXVWULHO'HVLJQVRPHQIXOGIHPnULJXGGDQQHOVHRJGHWV\QHVVRPRPDW OUHUNUIWHUQH HU EHYLGVWH RP GHQ PRGVWQLQJ GHU NDQ YUH PHOOHP ¨GHW SHUVRQOLJH XGWU\N¨
VRP VNROHQ L YULJW OJJHU PHJHQ YJW Sn RJ Vn GHW IRNXV Sn EUXJHUQHV SUIHUHQFHU VRP DOPLQGHOLJYLV HU NDUDNWHULVWLVN LQGHQ IRU ,QGXVWULHO 'HVLJQ )DJRPUnGHW ,QGXVWULHOW 'HVLJQ KDU IUHPIUW DW GH VWXGHUHQGHV IDJOLJH SURILO LNNH L WLOVWUNNHOLJ JUDG UXVWHU GH VWXGHUHQGH WLO HW VDPDUEHMGHPHGLQGXVWULHQ
(YDOXHULQJVJUXSSHQDQEHIDOHUGHUIRUDWGHWIDJOLJHLQGKROGIRU,QGXVWULHOW'HVLJQLVUVHWLO\VHW DI DW GHQQH HU Q\ ¥ UHWWHV WLO VnOHGHV DW XGGDQQHOVHQ L KMHUH JUDG JLYHU GH VWXGHUHQGH GHQ QGYHQGLJHEUHGHUHIDJOLJHSURILOLIRUKROGWLOGHWIUHPWLGLJHDUEHMGVPDUNHG
$UNLWHNWVNROHQL$DUKXV
'HVLJQGHOHQSn$UNLWHNWVNROHQL$DUKXVHUGHOVIRUVNQLQJVEDVHUHWGHOVSUJHWDIGHQDQDO\WLVNHRJ KHOKHGVRULHQWHUHGH DUNLWHNWIDJOLJH WLOJDQJ WLO GHVLJQEHJUHEHW 0DQ RULHQWHUHU VLJ L VWLJHQGH JUDG PRGGHWLQGXVWULHOOHSnEHNRVWQLQJDIGHWKnQGYUNVPVVLJHRJSULRULWHUHUVDPDUEHMGHKHUXQGHU WYUIDJOLJWVDPDUEHMGHKMW
,QVWLWXWLRQHQV PLVVLRQ HU VRP KMHUH XGGDQQHOVHVLQVWLWXWLRQ Sn NXQVWQHULVN RJ YLGHQVNDEHOLJW JUXQGODJ DW JLYH XGGDQQHOVH L DUNLWHNWXU LQGWLO GHW KMHVWH QLYHDX VDPW DW XGYH NXQVWQHULVN XGYLNOLQJVYLUNVRPKHGRJSnYLGHQVNDEHOLJWJUXQGODJDWGULYHIRUVNQLQJLQGHQIRUDUNLWHNWXUHQ
*UXQGOJJHQGHPHQWHIOJHJUXSSHQDWDOWKYDGDUNLWHNWHUODYHUHUGHVLJQRJDWJUQVHQPHOOHP DUNLWHNWXU RJ GHVLJQ HU IO\GHQGH 0HQ HQ DIJUHQGH IRUVNHO L IRUKROG WLO GHVLJQVNROHUQH KQJHU VDPPHQ PHG JUXQGXGGDQQHOVHQ Sn DUNLWHNWXGGDQQHOVHQ GHU JLYHU GH VWXGHUHQGH HQ JUXQGOJJHQGHDUNLWHNWRQLVNRJPHUHWHRUHWLVNDQDO\WLVNWLOJDQJ
,QGHQIRUGHVLJQRPUnGHWHUGHWSnNRUWVLJW$UNLWHNWVNROHQL$DUKXVªVWUDWHJLDWRSE\JJHVWUNH IRUVNQLQJVPLOMHU LQGHQ IRU ,QGXVWULHO 'HVLJQ .RPSRQHQW 'HVLJQ .RPPXQLNDWLRQV 'HVLJQ RJ
E\JQLQJVUHODWHUHW GHVLJQ LGHW GLVVH IRUVNQLQJVIHOWHU DOOHUHGH HU LQGDUEHMGHW L 'DQVN &HQWHU IRU ,QWHJUHUHW'HVLJQ&,'
0DQ KDU VWRUH IRUYHQWQLQJHU WLO GHWWH Q\H WYUIDJOLJH RJ WYULQVWLWXWLRQHOOH VDPDUEHMGH 3n VLJW NDQ GHWWH LQLWLDWLY YUH PHGYLUNHQGH WLO HQ Q\WQNQLQJ RJ RPVWUXNWXUHULQJ DI GHQ WUDGLWLRQHOOH DUNLWHNWIDJOLJHRJGHVLJQIDJOLJHRSGHOLQJ
(YDOXHULQJVJUXSSHQ VHU GHW Q\H FHQWHU IRU LQWHJUHUHW GHVLJQ VRP HQ PHJHW SHUVSHNWLYULJ Q\VNDEHOVH LQGHQIRU GH GDQVNH GHVLJQXGGDQQHOVHU GHU NDQ GDQQH IRUELOOHGH IRU Q\H PnGHU DW DUEHMGHVDPPHQSnSnWYUVDIIDJRJLQVWLWXWLRQHU
'HW DQEHIDOHV DW $UNLWHNWVNROHQ L $DUKXV QMH IOJHU GHW Q\ FHQWHUV XGYLNOLQJ VWWWHU FHQWHUHW UHVVRXUFHPVVLJWRJDWGHRSQnHGHUHVXOWDWHUIRUPLGOHVVnYHOLQWHUQWVRPHNVWHUQW
.XQVWDNDGHPLHWV$UNLWHNWVNROH
2JVn Sn .XQVWDNDGHPLHWV $UNLWHNWVNROH EHWRQHGHV GHW DW PDQ XGGDQQHU DUNLWHNWHU RJ LNNH GHVLJQHUH'HVWXGHUHQGHGHUYOJHUGHVLJQDVSHNWHWEOLYHUVnOHGHVDUNLWHNWHUPHGDIJDQJLGHVLJQ 'HVLJQGHOHQ Sn .XQVWDNDGHPLHWV $UNLWHNWVNROH HU GHOV IRUVNQLQJVEDVHUHW GHOV SUJHW DI GHQ DQDO\WLVNH RJ KHOKHGVRULHQWHUHGH DUNLWHNWIDJOLJH WLOJDQJ WLO GHVLJQEHJUHEHW 9HG EHVJHW IUHPKYHGHIOJHJUXSSHQHQGYLGHUHDWWYUIDJOLJKHGHQHUDUNLWHNWXGGDQQHOVHQVDGHOVPUNH
,QVWLWXWLRQHQEUHUSUJDIDWEHILQGHVLJLHWYDGHVWHGKYRUPDQSnGHQHQHVLGHNQ\WWHUDQWLOHQ JDPPHO DUNLWHNWIDJOLJ WUDGLWLRQ RJ Sn GHQ DQGHQ VLGH HU VNROHQ QX L IUG PHG DW IRUQ\
XGGDQQHOVHQVLQGKROGRJIRUP
$IGHOLQJHQIRU'HVLJQHUHWDEOHUHWYHGVDPPHQOJQLQJDIGHWLGOLJHUHDIGHOLQJHUJUDILVNGHVLJQ LQGXVWULHO GHVLJQ RJ PEHO RJ UXPNXQVW , XQGHUYLVQLQJVSODQHQ IRU VWXGLHnUHW SRLQWHUHVGHWDWGHUHU¨EHKRYIRUDWHWDUNLWHNWIDJOLJWKHOKHGVV\QSUJHUGHVLJQGHEDWWHQ3n GHQQH EDJJUXQG KDU DIGHOLQJHQ IRUPXOHUHW HW LQWHJUHUHW WYUIDJOLJW GHVLJQRPUnGH Sn DUNLWHNWIDJOLJW JUXQGODJ 9RUHV V\QVYLQNHO VJHU DW SUFLVHUH IDJRPUnGHWV SRVLWLRQ YHG EOD IOJHQGH IRUPXOHULQJ ¨$IGHOLQJHQV IDJOLJH XGJDQJVSXQNW HU DUEHMGHW PHG GH DVSHNWHU DI DUNLWHNWRQLVNIRUPJLYQLQJGHUSULRULWHUHUW\SHQIUHPIRUVWHGHW¨¨
$UNLWHNWVNROHQ L $DUKXV HU YUWVLQVWLWXWLRQ IRU &,' GHU HU HW WYUIDJOLJW VDPDUEHMGH PHOOHP $UNLWHNWVNROHQ L
$DUKXV $DOERUJ 8QLYHUVLWHW RJ 'DQPDUNV 7HNQLVNH 8QLYHUVLWHW PHG GHW IRUPnO XG IUD HW PXOWLGLVFLSOLQUW XGJDQJVSXQNWDWXGYLNOHIUHPWLGHQVGHVLJQIDJOLJKHGLWWGLDORJPHGSURGXNWLRQVPHWRGHU,&,'VUHJLHUWUHSKG VWXGHUHQGHDQVDWSn$UNLWHNWVNROHQL$DUKXVRJWLOVYDUHQGHHUWUHSKGVWXGHUHQGHDQVDWSn$DOERUJ8QLYHUVLWHW
'HU HU VnOHGHV Sn GHVLJQDIGHOLQJHQ RYHUYHMHOVHU L JDQJ YHGUUHQGH HQ Q\ VWUXNWXUHULQJ DI DIGHOLQJHQVDUEHMGVIHOWRJLQWHJUDWLRQDIQ\HLQWHUHVVHVIUHU+YRUPDQWLGOLJHUHVNHOQHGHPHOOHP GH WUH RPUnGHU PEHO RJ UXPNXQVW LQGXVWULHO GHVLJQ RJ JUDILVN GHVLJQ IRUVJHU PDQ QX DW VDPPHQWQNH GLVVH XQGHU WR V\QVYLQNOHU KKY V\VWHPHWV V\QVYLQNHO RJ GHW DUNLWHNWRQLVNH UXPV V\QVYLQNHO'HVXGHQDUEHMGHVGHUPHGLQWHJUDWLRQDIQ\HLQWHUHVVHVIUHUVnVRPV\VWHPGHVLJQGHU GNNHU RYHU VDPPHQVDWWH SURGXNWV\VWHPHU VnYHO VRP JUDILVNH V\VWHPHU VWRUVNDODGHVLJQ GHU LQGVNULYHU VLJ L E\HQ RJ GHW nEQH ODQGVNDE GHVLJQ WLO GHW DUNLWHNWRQLVNH UXP VDPW LQIRUPDWLRQVJUDILNRJPXOWLPHGLHUVDQYHQGHOVHLIRUELQGHOVHKHUPHG6NROHQJUJOGHQGHDWGHQ VUOLJH VW\UNH OLJJHU L DW IRNXVHUH Sn IRUKROG VRP LQGVNULYHU VLJ L UXP RJ VWUUH VDPPHQVDWWH V\VWHPHU
(YDOXHULQJVJUXSSHQ DQEHIDOHU DW .XQVWDNDGHPLHWV $UNLWHNWVNROH LQWHQVLYHUHU IRUDQGULQJVSURFHVVHUQH JHQQHP HQ VW\UNHOVH DI VDPVSLOOHW PHOOHP GH IDJOLJH HQKHGHU VWXGLHDIGHOLQJHQ IRU GHVLJQ ,QVWLWXW IRU 'HVLJQ RJ ,QVWLWXW IRU 9LVXHO .RPPXQLNDWLRQ +HUYHG YLO GLDORJHQPHOOHPIRUVNQLQJHQRJXQGHUYLVQLQJNXQQHIUHPPHV
*ODV .HUDPLNVNROHQSn%RUQKROP
6NROHQ OLJJHU L 1H[ Sn %RUQKROP KYRU GHU GHOV HU HW EHJUQVHW XGEXG DI YLGHUHJnHQGH XGGDQQHOVHU GHOV HQ KnQGYUNVPVVLJ ORNDO WUDGLWLRQ RJ GHOV HW JHQHUHOW EHKRY IRU DUEHMGVSODGVHU'LVVHIRUKROGHUHQGHODIEDJJUXQGHQIRUGHQORNDORSEDNQLQJRJPHGLHPVVLJH RSPUNVRPKHG VRP XGJU HW VWRUW DNWLY IRU VNROHQ 'HQ RYHURUGQHGH PnOVWQLQJ PHG XGGDQQHOVHQO\GHUVRPIOJHU
¨*ODVRJNHUDPLNXGGDQQHOVHQKDUWLOIRUPnOSnEDJJUXQGDIHWVROLGWKnQGYUNVPVVLJW NHQGVNDEWLODUEHMGHWPHGWHNQLNRJPDWHULDOHUDWJLYHGHVWXGHUHQGHNYDOLILNDWLRQHUWLOSn HW KMW NXQVWQHULVN QLYHDX DW XGIRUPH RJ XGYLNOH JODV HOOHU NHUDPLNNXQVWKnQGYUNHW
*HQQHPHQSHUVRQOLJLQGOHYHQLKnQGYUNHWVNRQNUHWHDUEHMGHPHGPDWHULDOHUWHNQLNRJ IRUP RJ JHQQHP HQ WHRUHWLVN VNROLQJ VNDO GH VWXGHUHQGH JLYHV GH QGYHQGLJHWHNQLVNH
VWHWLVNH RJ DQDO\WLVNH IRUXGVWQLQJHU IRU DW NXQQH OVH RSJDYHU LQGHQ IRU JODV HOOHU NHUDPLNRJKHUPHGEHVOJWHGHPDWHULDOHU¨
6NROHQOJJHUYJWSnHQDYDQFHUHWPHQUHODWLYVQYHUKnQGYUNVPVVLJLQGOULQJPHGIRNXV Sn PDWHULDOHUQH RJ KnQGYUNHW , EHWUDJWQLQJ DI VNROHQV PnOVWQLQJ RP DW JLYH GH VWXGHUHQGH NYDOLILNDWLRQHU WLO Sn KMW NXQVWQHULVN QLYHDX DW XGIRUPH RJ XGYLNOH JODV HOOHU NHUDPLNHUNXQVWKnQGYUNHWJLYHUVNROHQLNNHLQGWU\NDIDWKDYHHWVUOLJWVSURJOLJWUHIOHNWHUHQGH
%HNHQGWJUHOVHQU
IRUPJLYQLQJVPLOM 5HNWRU XQGHUVWUHJHGH GD RJVn L IRUELQGHOVH PHG EHVJHW DW GH RSIDWWHU XGGDQQHOVHQ VRP HQ SURIHVVLRQVVNROH PHG VSHFLDOH L NHUDPLN RJ JODV KYRU IRNXV HU Sn IRUP RJ IDUYHVQDUHUHHQGSnLGpHUQHEDJXGIRUPQLQJHQ+HUWLOHIWHUO\VWHGHVWXGHUHQGHPHUHnEHQKHGRJ YLOMHWLOGHEDWIUDVNROHVVLGHYHGUUHQGHXGGDQQHOVHQVPnOVWQLQJHULQGKROGRJIRUP
(YDOXHULQJVJUXSSHQ DQEHIDOHU DW VNROHQ SULRULWHUHU GLDORJ PHOOHP DQVDWWH RJ VWXGHUHQGH RP XGGDQQHOVHQVPnOVWQLQJHULQGKROGRJIRUP
'HWHUHWQVNHIUDVNROHQDWVWXGLHWLGHQXGYLGHVIUDGHQXYUHQGHWUHnUWLOILUHHOOHUIHPnU'HQ HNVWUDWLGVNXOOHHIWHUOHGHOVHQVRSIDWWHOVHLJLYHWIDOGEUXJHVWLODWJHIOHNVLELOLWHWHQIRUG\EHOVHQ RJGHQSHUVRQOLJHXGYLNOLQJ
'HWHUHYDOXHULQJVJUXSSHQVYXUGHULQJDWGHUHUEUXJIRUDWVNROHQOJJHUVWUUHYJWSnWHRUHWLVN IRUG\EHOVH RJ SHUVRQOLJ NXQVWQHULVN XGYLNOLQJ VDPPHQ PHG HQ XGYLNOLQJ DI GHW KnQGYUNVPVVLJH'HUHUGHVXGHQEHKRYIRUDWVNROHQIRUDWNXQQHOHYHRSWLOEHNHQGWJUHOVHQV IRUPnOSUFLVHUHULQGKROGRJIRUPVRPYLGHUHJnHQGHNXQVWQHULVNXGGDQQHOVHVLQVWLWXWLRQ
(YDOXHULQJVJUXSSHQNDQLNNHSnGHWIRUHOLJJHQGHJUXQGODJDQEHIDOHHQVWXGLHWLGVIRUOQJHOVHPHQ DQEHIDOHULVWHGHWHWPHUHXGYLNOHWVDPDUEHMGHPHOOHPGHVLJQXGGDQQHOVHUQH
*HQHUHOOHDQEHIDOLQJHUYHGUUHQGHIDJOLJSURILOYLVLRQHURJSULRULWHULQJHU
'HW HU HYDOXHULQJVJUXSSHQV RSIDWWHOVH DW GHW EnGH DI KHQV\Q WLO GH VWXGHUHQGH YHG YDOJ DI XGGDQQHOVHVVWHGVDPWDIKHQV\QWLOEnGHQDWLRQDOHRJLQWHUQDWLRQDOHVDPDUEHMGVSDUWQHUHYLOYUH HQIRUGHOIRUGHQHQNHOWHLQVWLWXWLRQPHGHQVNDUSHUHSURILOHULQJRJHQNODUHUHLGHQWLWHW
(YDOXHULQJVJUXSSHQ DQEHIDOHU DW VnYHO GHVLJQVNROHUQH VRP DUNLWHNWVNROHUQH XGDUEHMGHU NODUHUH SURILOHUIRUGHUHVXGGDQQHOVHURJIRUDUNLWHNWVNROHUQHRJVnIRUGHUHVIRUVNQLQJSnGHVLJQRPUnGHW (QGYLGHUH ILQGHU HYDOXHULQJVJUXSSHQ DW HQ VWUUH JUDG DI VWUDWHJLVN EHYLGVWKHG PHG YDOJWH SULRULWHULQJHU XGIRUPQLQJ DI KDQGOLQJVSODQHU RSVWLOOLQJ DI VXFFHVNULWHULHU VDPW IDVWODJWH SURFHGXUHU IRU RSIOJQLQJ YLOOH VW\UNH GH VWUDWHJLVNH RYHUYHMHOVHU VRP HU LQGHKROGW L VHOYHYDOXHULQJVUDSSRUWHUQH
$UEHMGVGHOLQJRJVDPDUEHMGHPHOOHPGHVLJQXGGDQQHOVHUQH
'HWWH DIVQLW IRNXVHUHU Sn KYRUOHGHV HQ HYHQWXHO DUEHMGVGHOLQJ PHOOHP GH IRUVNHOOLJH GHVLJQXGGDQQHOVHUQHNDQXGYLNOHV
'HIHPXGGDQQHOVHUVnGDQVRPGHWDJHUVLJXGLGDJNDQVHVLIRUKROGWLOHWVSHNWUXPGHUJnUIUD HUKYHUYVVNROHU EDVHUHW Sn KnQGYUNVPVVLJ LQGOULQJ VRP GHQ HQH SRO RJ WLO DNDGHPLVNH XGGDQQHOVHUPHGYJWSnDQDO\VHWHRULRJIRUVNQLQJVRPGHQDQGHQSRO*ODV .HUDPLNVNROHQ Sn %RUQKROP OLJQHU VnOHGHV PHVW HQ WUDGLWLRQHO HUKYHUYVVNROH GRJ PHG HQ KM JUDG DI VSHFLDOLVHULQJ PHQV DUNLWHNWVNROHUQH XGJU GHQ PRGVDWWH SRO 'H WR GHVLJQVNROHU EHILQGHU VLJ KHULPHOOHPRPHQGGHLGLVVHnULIUGPHGDWEHYJHVLJKHQLPRGHQKMHUHSULRULWHULQJDIGHW PHUHJHQHUHOOHDQDO\WLVNHRJWHRUHWLVNH'HWHULGHQQHVDPPHQKQJDWEHKRYHWIRUIRUVNQLQJRJ IRUVNQLQJVEDVHUHWXQGHUYLVQLQJDNWXDOLVHUHV
6RP XGJDQJVSXQNW ILQGHU HYDOXHULQJVJUXSSHQ GHW NRQVWUXNWLYW DW GHU ILQGHV VnYHO DUNLWHNWVNROHU PHGHQDUNLWHNWIDJOLJRJPHUHJHQHUHODQDO\WLVNWLOJDQJWLOGHVLJQIDJHWVRPGHVLJQVNROHUPHGHW KnQGYUNVPVVLJW RJ PHUH PDWHULDOHRULHQWHUHW XGJDQJVSXQNW +YRUDUNLWHNWVNROHUQHLKMJUDG RULHQWHUHU VLJ L IRUKROG WLO UXPPHW KHOKHGHU V\VWHPHU RJ RIWH L VWRU VNDOD Vn DUEHMGHU GHVLJQVNROHUQHPHUHPHGWHNQLNPDWHULDOHURJIRUPJLYQLQJ¥LVNDODHQWLOHQ
1nUGHUWLOOJJHVHQORNDOLVHULQJVPVVLJVNHOQHQPHOOHPVWRJ9HVWGDQPDUNIUHPVWnUGHWVRP KHQVLJWVPVVLJW DW GHU ILQGHV VnYHO HQ DUNLWHNWVNROH VRP HQ GHVLJQVNROH L EnGH VW RJ 9HVWGDQPDUN
'HW GHU GHUIRU L KMHUH JUDG HIWHU HYDOXHULQJVJUXSSHQV PHQLQJ EU RYHUYHMHV HU KYLONH KnQGYUNVPVVLJHIDJRJVSHFLDOHUGHUEUYUHDGJDQJWLOSnGHILUHVNROHU0HJHWWDOHUIRUDW GHU NXQ EU YUH HQ DIGHOLQJ IRU PEHO RJ HQ DIGHOLQJ IRU YLVXHO NRPPXQLNDWLRQ L KKY
VWGDQPDUNRJ9HVWGDQPDUN'HWWHYLOOHLQGHEUHVnYHOUHVVRXUFHPVVLJHIRUGHOHVRPHWVWUUH VWXGHQWHUJUXQGODJSnLQVWLWXWQLYHDX
(Q PHJHW XGYLNOHW DUEHMGVGHOLQJ VNROHUQH LPHOOHP KYRU GH VWXGHUHQGH UHODWLYW OHW NXQQH EOLYH RYHUIUWPHOOHPVNROHUQHYLOOHIRUXGVWWHHQLQGE\UGHVWLOQUPHOVHDIVWUXNWXUHQIHNVLIRUPDI IRUVNHOOLJHYDULDQWHUDIHQWUHSOXVWRPRGHOVRPNHQGHVSn8QGHUYLVQLQJVPLQLVWHULHWVRPUnGH(Q VnGDQWLOQUPHWVWUXNWXUNXQQHnEQHRSIRUDWPDQHIWHUGHIUVWHWUHnUPHGHWDIVWLJQLQJVEHYLV
6H NDSLWHO PHG KHQV\Q WLO VDPDUEHMGHW PHG DQGUH XGGDQQHOVHVLQVWLWXWLRQHU VDPW GH VWXGHUHQGHV PHULWPXOLJKHGHU
6HQRWHV