AI System Creates 3D 'Motion Sculptures' Using Video Footage

MIT CSAIL have created a system called MoSculp that allows viewers to interpret how different parts of the human body react at different times.

MIT CSAIL has released some mesmerizing footage of a system that visualizes complex human motion via 3D motion sculptures. The video shows how the system creates a 3D structure swept by a human body as it moves through space.

The system, called MoSculp is first fed an input video. The system then embeds it back into the scene in a 3D-aware fashion. MIT CSAIL says users of the system can customize the sculpture design by selecting different materials and lighting conditions.

By visualizing movement with this system, human movement can begin to be understood in ways that are impossible when viewing purely with the naked eye. The motion sculptures reveal space-time motion information that allows viewers to interpret how different parts of the human body react at different times.


The system has potential to be used by both professional athletes and their coaches as well as doctors and other practitioners working to understand a human movement that could assist with the development of mobility aids and prosthetics.

“Imagine you have a video of Roger Federer serving a ball in a tennis match, and a video of yourself learning tennis,” says Ph.D. student Xiuming Zhang, lead author of a new paper about the system. “You could then build motion sculptures of both scenarios to compare them and more comprehensively study where you need to improve.”

The research team will present their paper on MoSculp next month at the User Interface Software and Technology (UIST) conference in Berlin, Germany.


Follow Us on


Stay ahead with the latest science, technology and innovation news, for free:

By subscribing, you agree to our Terms of Use and Privacy Policy. You may unsubscribe at any time.