Animated Drawings: Meta’s new AI project lets you convert doodles into animation

The open-source project is based on a dataset of up to 180,000 annotated amateur drawings.
Mrigakshi Dixit
Representational image of Cubism style portrait.
Representational image of Cubism style portrait.

Dusan Stankovic/iStock 

In the face of increasing competition, it appears that Meta is significantly upping its artificial intelligence (AI) game. 

Following the release of the "Segment Anything Model," the tech giant has unveiled yet another interesting and fun AI-based project. The project, called Animated Drawings, allows you to turn your doodles into animations. And it could be the next big thing. 

This open-source project is based on a dataset of up to 180,000 annotated amateur drawings. The company has made this dataset and animation code available for AI researchers and creators to use and innovate further. The Meta blog also emphasizes that this is the "first annotated dataset" made of a large variety of artwork. 

“With this dataset and animation code, we believe that the domain of amateur drawings can inspire a new generation of creators with its expressive and accessible possibilities. We hope they will be an asset to other researchers interested in exploring potential applications for their work,” said the blog post. 

The process of turning drawings into animation

After the drawing is uploaded, a series of subtasks are performed to convert the sketch to animation. Human figure detection, segmentation, and pose estimation are some of the subtasks performed by the AI system. These features aid in identifying and filtering out unwanted elements such as shadows, paper creases, and light glare that are common in sketches. 

These subtasks allow computer vision to capture a digital version of a doodle or drawing. Following which, the system employs digital graphics techniques to convert the drawing image into an animated version. The system has been "trained on photographs of real-world objects," including children's drawings.

The Fundamental AI Research (FAIR) team first introduced the Animated Drawings Demo in late 2021. The demo invited people to contribute their amateur drawings in order to create a large dataset with their permission to ensure privacy. 

“The browser-based demo allowed people to upload images, verify or fix a few annotation predictions, and receive a short animation of their humanlike character within their drawing,” said the company’s blog

Approximately 6.7 million images were uploaded to this demo platform, and 3.2 million people from various parts of the world used this system.  

The company has also published a research paper, which can be accessed here.

Study Abstract:

Children's drawings have a wonderful inventiveness, creativity, and variety to them. We present a system that automatically animates children's drawings of the human figure, is robust to the variance inherent in these depictions, and is simple and straightforward enough for anyone to use. We demonstrate the value and broad appeal of our approach by building and releasing the Animated Drawings Demo, a freely available public website that has been used by millions of people around the world. We present a set of experiments exploring the amount of training data needed for fine-tuning, as well as a perceptual study demonstrating the appeal of a novel twisted perspective retargeting technique. Finally, we introduce the Amateur Drawings Dataset, a first-of-its-kind annotated dataset, collected via the public demo, containing over 178,000 amateur drawings and corresponding user-accepted character bounding boxes, segmentation masks, and joint location annotations.

Add Interesting Engineering to your Google News feed.
Add Interesting Engineering to your Google News feed.
message circleSHOW COMMENT (1)chevron
Job Board