Draw This is an instant camera that draws cartoons.
Instant cameras are a classic that still delights today. There is something eternally amusing about a physical, unique image, that is uniquely different to digital. Playing with neural networks for object recognition one day, I wondered if I could take the concept of an instant camera one step further, and ask the camera to re-interpret the image, printing out a cartoon instead of a faithful photograph.
One of the fun things about this re-imagined camera is that you never get to see the original image. You point, and shoot – and out pops a cartoon; the camera’s best interpretation of what it saw. The result is always a surprise. A food selfie of a healthy salad might turn into a enormous hotdog, or a photo with friends might be photobombed by a goat.
The camera is a mash up of a neural network for object recognition, the google quickdraw dataset, a thermal printer, and a raspberry pi. Initially, I began with some experiments on my laptop. I set up an image processing pipeline in python to take pre-captured images and recognise the objects in them, using pre-trained models from google. At the same time, I explored the quickdraw dataset, and mapped the categories available in the dataset with the categories recognisable by the image processor. After writing some code to patch the two together, wrapping the lot in a docker image, and cobbling together some electronics, interspersed with some hair pulling moments of frustration, the camera was ready.
The outcome was a really fun way to get in to creative applications for neural networks. If you would like to make your own you can find code + instructions on github. The good folks at Kapwing have also put together a great online version for you to try yourself!
This project has been featured on TechCrunch, Engadget, Vice Motherboard, the Raspberry Pi Foundation Blog, The Verge, and Hackaday.