S c d Pa #A IR2021:
T 22 d A a C c
A c a I R a c
Vi al E e / 13-16 Oc 2021
WHICH HUMAN FACES CAN AN AI GENERATE?
LACK OF DIVERSITY IN THIS PERSON DOES NOT EXIST
L ca N e Se ei a1
G A ificial I ellige ce a d A (GAIA) a d M lec la Scie ce c e, U i e i f Sa Pa l
B S. M e chi2
G A ificial I ellige ce a d A (GAIA) a d Fac l f A chi ec e a d U ba i m, U i e i f Sa Pa l
Vi ici A iel A da d Sa 3
G A ificial I ellige ce a d A (GAIA) a d P l ech ic Sch l f he U i e i f S Pa l
1. I
I hi ab ac e h he e l f a i e di ci li a e ea ch i hich e a di fake h ma face (dee fake ) ge e a ed b he eb i e Thi Pe D e N E i (TPDNE),
a d di c h hi em ca hel e e a e ma i i ie ed b a
de e de c a limi ed da aba e. O a al i f c e he defa l ge e ic face e c ea ed b e la i g a d m am le f fake face ge e a ed b TPDNE' alg i hm . I de e de l f he g f fake h ma face am led, he ame ge e ic hi e face al a a ea ed a a e l (Fig. 1).
3Vi ici A iel A da d Sa i a de g ad a e de f he P l ech ic Sch l f he U i e i f S Pa l , e ea che a G A ificial I ellige ce a d A (GAIA / C4AI / I a USP). E-mail:
i ici a iela da@ .b
2 B M e chi i a i al a i , d c al fell a U i e i f S Pa l Fac l f A chi ec e a d U ba i m, e i fell a he Ce e f A , De ig a d S cial Re ea ch (CAD+SR), membe f he Hi ie f AI: A Ge eal g f P e g (U i e i f Camb idge) a d c -c di a f GAIA. E-mail:
b m e chi@ .b
1 L ca N. Se ei a i a de g ad a e de f M lec la Scie ce a he U i e i f S Pa l , e ea che a G A ificial I ellige ce a d A (GAIA / C4AI / I a USP) a d j i e ea che a CP D, i he a ea f Na al La g age P ce i g. E-mail: l ca e @ .b
Suggested Citation (APA): Sequeira, L., Moreschi, B., and Arruda dos Santos, V., (2021, October). Which Human Faces Can an AI Generate? Lack Of Diversity In This Person Does Not Exist. Paper presented at AoIR 2021: The 22nd Annual Conference of the Association of Internet Researchers. Virtual Event: AoIR. Retrieved from http://spir.aoir.org.
Fig. 1: Defa l ge e ic face e la i g 3000 a d m fake face b TPDNE. C edi : Be a d F e a d L ca N. Se ei a / GAIA
Face alg i hmicall ge e a ed a e a f he im eme i he CV field i ce 2014, i h he de el me f a machi e lea i g i f a c e called Ge e a i e Ad e a ial Ne k GAN (G dfell e al., 2014). The i c ea e f ha d a e ce i g ca aci ie made he i e i e e f hi i f a c e iable, ece l all i g he c c i f m e eali ic image f h ma , a imal a d bjec f m he e GAN k a S leGAN (Ka a e al., 2019) a d S leGAN2 (Ka a e al., 2020; Nie e al., 2020).
C ea ed i 2019, TPDNE i he e l f i g hi la ecific GAN a d i a l achie able ha k he e f he da aba e f eal h ma face f m he Flick -Face -HQ da aba e, i h 70 h a d high defi i i image . E e ime he
eb i e i ef e hed, i AI e de a e (a d fake) h ma face.
The i e f CV ha m ed e e al deba e ch a he alg i hmic i j ice (N ble, 2018). I hi e ea ch e a e a ic la l i i ed b ecific a al i f da a e ch a
he a di i g ImageNe c d c ed b P abh a d Bi ha e (2020), a d al he cial a d c l al im ac ha h e em eflec i cie a achi g c l al a ec (Mi e al., 2019) ce ai he ic bg (B lam i i a d Geb , 2018).
Fig. 2: Sam le f fake h ma face ge e a ed b TPDNE.
2. M
We fi b il a da aba e i h 4100 fake h ma face ake f m TPDNE eb i e ia eb c a i g. The , e a al ed hem h gh a P h la g age c i c ea ed b , a d di c ed beha i ide ified i hi S leGa 2. O a al e a e ba ed he e f
ecific image c ea ed f m a b e f e la i g fake h ma face a ailable i da aba e. The e e l i g image , called he e cl e -image , e e made f m he
e la i g fNa bi a image ge e a ed b he TPDNE' alg i hm .
3. W a a ?
3.1. A a a a a
T e hi a eme , e fi c ea ed i cl e -image b e la i g fake face c a ed f m TPDNE (Fig. 3), each f med b diffe e am f image . We b e ed ha he e l i g cl e -image a e imila : a e i h hi e ki , da k e e a d b hai , cha ac e i ic ha a e e e ed ega dle f he mbe f diffe e fake face e la ed.
Fig. 3: Se f i image e l i g f m e la each i h a mbe f diffe e image . The cl e -image A, B, C, D, E a d F e e ge e a ed b , e ec i el , 10, 30, 100, 300, 1000 a d 3000 image ake f m da aba e a d ml a d
i h e e i i .
T a e h he e cl e -image a e imila each he , e c ea ed a e f 20 cl e -image f each al e f N. I each e cl e -image i h a fi ed N, e calc la ed he mea de ia i be ee he image i hem (Fig. 4). Thi c e e eal ha
he deg ee f imila i i c ea e e e iall f cl e -image ha e e ge e a ed i h a la ge mbe f da a. F m m e ha 1000 image , e ca a me ha he cl e -image ge e a ed a e he defa l ge e ic face f he da aba e (Fig.1; E a d F i Fig. 3). Thi e e ime h ha , he e alk ab he TPDNE' alg i hm , a la ge da a l me d e mea a i c ea e i he di e i f e l , b he c a .
Fig. 4: The cha e e e he mea a da d de ia i c e be ee e image a mi g diffe e al e f N, each ge e a ed b 20 diffe e cl e -image .
3.2. Fa b a a
We al a al ed h he image da aba e beha e i ela i ki e di e i . F hi , e elec ed all he face f black e le f m a e f 4100 TPDNE e l he e
e e l 54 face , hich e e e 1.4% f he h le e , hich e e h hi aciali a i a d hi e e f he da a e . Wi h he e image e ge e a ed a cl e -image (A). We al elec ed 54 a d m image f hi e face c ea e a he cl e -image (B) f c m a i (Fig. 5). Fi all , e c cl de ha he cl e -image ge e a ed b hi e e le i i all m e imila he defa l ge e ic face ha he
cl e -image ge e a ed b face f black e le. Thi i em i ical e ide ce ha he c ea i f fake face i i de e de f cha ac e i ic ch a hi e e im ed b
ce e ha a e al ead hi icall k .
Fig. 5: Cl e -image f e la : A (54 image f diffe e fake face f black e le) a d B (54 diffe e fake face f hi e e le).
4. W a a ?
The e e l i ig e beca e he a e a f a fi ge e a i f eali ic image c ea ed i de e de l f m he h ma e e, i ce he a e he i ed lel b alg i hm a e cha e i he hi f - h g a h (Beig elma , 2020). B al , a ic la l beca e he lack f di e i f TPDNE' ge e a ed face i a b g i
hi digi al i f a c e, b a ei f ci g a da d d amic ha hi icall eg la e b die , e i ie a d ac ice , f m hich he c m e cie ce a e em ed. I i b cha ce ha Beig elma (2020) al ai e he e i : a e fake h ma face he a ceme f a e e a f he e ge ic f image ?
Wh , f e am le, i he e a ge e ic mile defa l ge e ic face a d i all he fake h ma face i TPDNE? Wha d e i hide f m ? Th a d f eca i c d k (a T ke ) ga i i g he Flick -Face -HQ da aba e ma be e f he a e (Ka a e al., 2019). The fac ha hi S leGAN a l ible ha k he e f ama e
e al image ed Flick (Smi a d We e , 2021) a d ake i h he c e f i a h al ell a l ab h da a i made a ailable ai A ificial I ellige ce.
5. R
Beig elma , G. (2020). A e dade do deepfake . Re ie ed Ma ch 25, 2021, f m h ://b k.affec i g- ech l gie . g/a - e dade -d -dee fake /
B lam i i, J.; Geb , T. Gende hade : In e ec ional acc ac di pa i ie in comme cial gende cla ifica ion. I : C fe e ce fai e , acc abili a d
a a e c . PMLR, 2018. . 77-91.
G dfell , I.; P ge -Abadie, J.; Mi a, M.; X , B.; Wa de-Fa le , D.; O ai , S.; C ille, A.; Be gi , Y. (2014). Gene a i e Ad e a ial Ne o k . P ceedi g f he I e a i al C fe e ce Ne al I f ma i P ce i g S em (NIPS 2014). . 2672 2680.
Ka a , T., Lai e, S., & Aila, T. (2019). A le-ba ed gene a o a chi ec e fo gene a i e ad e a ial ne o k . I P ceedi g f he IEEE/CVF C fe e ce C m e Vi i a d Pa e Rec g i i ( . 4401-4410)
Ka a , T., Lai e, S., Ai ala, M., Hell e , J., Leh i e , J., & Aila, T. (2020).Anal ing and imp o ing he image q ali of legan. I P ceedi g f he IEEE/CVF C fe e ce C m e Vi i a d Pa e Rec g i i ( . 8110-8119).
Mi , A., G bb , B., Sil a, T., & Pili e , E. (2019, A il 17). In e oga ing Vi ion API . Re ie ed Ma ch 18, 2021, f m:
h :// . e ea chga e. e / blica i /332910402_I e ga i g_Vi i _API /
Nie, W., Ka a , T., Ga g, A., Deb a h, S., Pa e , A., Pa el, A., & A a dk ma , A. (2020, N embe ). Semi- pe i ed legan fo di en anglemen lea ning. I : I e a i al C fe e ce Machi e Lea i g ( . 7360-7369). PMLR.
N ble, S. U. (2018).Algo i hm of opp e ion: Ho ea ch engine einfo ce aci m. NYU P e .
P abh , V. U.; Bi ha e, A. (2020). La ge image da a e : A p hic in fo comp e i ion?. a Xi e i a Xi :2006.16923.
Smi , T.; We e , M. (2021) The Agenc of Comp e Vi ion Model a Op ical In men . Vi al C mm ica i . h ://d i. g/10.1177/1470357221992097.
