The future in a box

It took two tries, but we got in to the Japan Media Arts Festival at the National Arts Center before it ended. Many supercool interactive exhibits, but surprisingly not a lot that felt brand new. This was the only one that I didn't feel like I'd seen some version of before. The box on the screen is just a solid paper-covered block a little smaller than a Rubix cube that the kid is holding a few inches in front of a camera. It has an abstract black and white mosaic with just a few tiles on each face.
On the screen, each face of the cube appears as a 3D room with a well-rendered man inside. He moves around when the cube is moved, going back and forth and up and down steps depending on how the cube tips. When he's maneuvered out the door he pops out in another room.It takes a moment to be translated to video when the cube moves suddenly, and the image stutters back and forth between the paper surface and the simple little 3D world inside, like Take on Me.
The kids were hogging it for 20 minutes and I kept waiting for their mother, who was watching, or the staff guy, who kept coming over and frowning at their backs, to say alright kiddies, move it along. But no. So, I waited my turn, and it was worth it.

I do have questions. What are the applications for this? How does it work? And where does the little man go when the camera is off?

The show is a collection of the winners and notable entries of an annual juried contest including hours and hours of amazing animation and video. Mark your calendar for next year. And don't wait til the last weekend. We could have gone in on the second to last day instead of the last. Even got in the long line since it was moving pretty fast. We got a peek into the jam-packed gallery as the line wound past and I'm afraid my suggestion that we should perhaps spend our time elsewhere was made in a rather unladylike way.