If you have a phone then you’ve probably played around with one of the trendy face-altering filters. Unless you’ve been living under a rock, then getting a glimpse at your elderly or baby self, changing your gender, or just slapping on rabbit ears is not something that would ‘WOW’ you. However, what may surprise you is that an Israeli developer has utilized Machine-Learning technologies to take the filter craze one step further, by turning your image into a Pixar cartoon character.
Training StyleGAN2 on cartoon characters
The Toonify model is based on Deep-Learning algorithms that will take your image input and transform it into a Pixar cartoon character. The modification model was developed by Israeli developer Doron Adler and his partner Justin Pinkney. Doron started the project at of pure amusement and fun, but once he connected with Pinkney on Twitter, the project grew legs and a new trend was born.
In a conversation with Geektime, Adler tells that his passion for generative models is what led him to create the filter model. “I really enjoy messing around and researching the possibilities that can be done with these tools. I really got excited the first time I saw Justin’s algorithm, using Nvidia’s StyleGAN2, for creating Ukiyo-e faces, and I wanted to see how it would work with a model that I had fine-tuned to faces from Disney, Pixar, and DreamWorks characters.”
StyleGAN2 is a framework that can generate a non-stop stream of faces of people who don’t really exist (you may remember the website thatpresondoesnotexist.com), Adler took the platform and fed it with cartoon faces, rather than real ones, in order to train the AI system on generating similar faces. He even once applied the practice of training StyleGAN2 on The Muppets’ images to create new characters (incredible but highly psychedelic).
The two took their combined efforts to the next stage after Pinkney saw how Adler used his work on his generative model. “He really loved it and asked me if I would want to take it ‘one step further’. The point was to train a model called Pix2PixHD that runs on CPU rather than GPU, this way any user could Toonify their own personal image,” explains Adler.
After generating troves of cartooned characters and training the Pix2PixHD model, the website went live. Adler notes that they were extremely successful in a short amount of time: “Just 20 hours after uploading the platform we already had thousands of users all at once. The costs started seeming worrisome, in less than a day we hit $100 and the site was just becoming more and more popular. So, we closed it down. During the short time that the site was live, close to a quarter-million images had been Toonified.” The website went dark for about a week and a half, though once Adler and Pinkney found Toonify a new home at DeepAI.org the site was up-and-running again.
So after checking out the end result of Adler’s creation, we had to ask him the million-dollar question - does he know how creepy the generated characters come out looking? And is he ok with it? According to Adler, it’s all due to the Uncanny Valley effect where imperfect artificial replicas of ourselves seem weird and cause us unease. “We’re pretty cool with it, interestingly enough it has become quite the gimmick for the site,” states Adler.
Continuing with their “day jobs”, Adler and Pinkney soldier on, developing the project together. They are working on creating different versions for the models that they developed, introducing an option to filter more than one face on Toonify in addition to further innovating the product - it’s important to remember that this is still a hobby for both of the developers and they only play around with it in their free time.
What about privacy? Similar apps have been previously persecuted for having sketchy privacy policies, however, Adler calmly claims that all images are erased after the the DeepAI model is done doing its magic.
