Back

Meta uses AI to bring children's drawings to life

Meta Engineers (formerly Facebook) have developed an Artificial Intelligence (AI) system capable of identifying and automatically animating human-like figures in children's drawings.

The choice in favor of children's drawings in the company is explained by the variety of forms, colors, sizes and scales of "heroes" depicted by children, which are effective input data for learning AI.

266785441_1595147910829577_5871587137964044236_n.png

The first step in animating drawings is to distinguish the human figures from the background and other characters. The system allows the user to adjust the mask - to set the boundaries for the character of the drawing, to check the filling areas and connection points of the limbs.

Meta AI’s neural network-based object detection model, Mask R-CNN, is used to extract human-like silhouettes in the original drawings.

Next, Meta uses AlphaPose to identify key points on the human figures that can serve as hips, shoulders, elbows, knees, wrists, and ankles. This model was trained on images of real people; however, it was adapted for children's drawings and was trained to detect the character's pose in Meta.

Bringing 2D shapes to life with 3D motion capture takes the perspective of the drawing into account to correctly animate the character and make him move left or right, depending on the original intent.

267494493_674900687280601_9185796494071382734_n.png

The company is confident that in the future this technology will make it possible to create animated cartoons by animating several characters in one scene, and using AR glasses to interact with newly created characters.

"This project will bring us closer to creating an AI that can understand the world from a human point of view," - said in Meta.