Pilots & Experiments

Kinect Digital Puppetry: It’s here, it’s real and it works!

By February 9, 2011 January 30th, 2014 One Comment

Back in November I wrote about those Kinect digital shadow puppets that went viral. Well, Kinect hacking has come a long way in the few short months since then and the potential implications for digital puppetry are huge. I was at a small meet-up for Kinect hackers here in Toronto last night and tried a very rudimentary version of “Kinect puppetry” that an animator I know has been working on and it was very cool. Affordable, accessible digital puppetry is here, it’s real and if you already have a computer and a little technical know-how you can do it yourself for under $200.

Here’s an impressive demo of Kinect puppetry using the previously-blogged Japanese “Vocaloid” Hatsune Miku using the Kinect and free software from the VPVP Project:

Aside from the Kinect itself (which retails for around $150), most of the software needed to try this yourself can be downloaded for free. You will need some sort of 3D program likeBlender and a 3D game engine; fortunately “lite” or “learning” versions of commercial game engines like Motionbuilder and Unity can be downloaded for free and some people are usingOgre 3D, which is free and open source. The exact methods to make your own digital puppets will vary depending on which software you use, but lots of tutorials can be found by searching online.

Not all digital puppetry with the Kinect has to be three dimensional though. Here’s a neat 2D demo done in the style of shadow puppetry. It’s part of a project called Virtual Marionette:

Although I do think this is huge step forward for open, accessible digital puppetry there are still some problems to solve before Kinect can be used to create truly emotive, expressive characters. For one thing, the skeletal tracking in these demos is very simple without individual finger movements or any kind of facial expression. Those problems aren’t unsolvable though and I have some ideas on how to tackle them, but that’s another post.

In the meantime, if you’re interested in this rapidly emerging field, be sure to subscribe to my other blog Machin-X: Digital Puppetry, which focuses exclusively on digital puppetry, real-time animation and related technologies.

One Comment

Leave a Reply