Thursday, July 25, 2013

Leap Motion: real-time 3D data is a leap into the future

I have had a Leap Motion device on order since July 2012. Earlier this week I received an email saying that my device has been shipped. Yay!

Even better: courtesy of a friend, I was able to obtain a developer version of the device last week and I have been coding for it ever since.

I am in heaven.

My on-going dream is to code around real-time data in a 3D world.

With the Leap Motion I am reveling in over abundance of 3D data.

Perhaps my brain is so happy because it is seeing me do what it does for itself incessantly without thinking!

I think my brain also likes it when a lot of it is being used on work that requires a lot of concentration. And, boy, coding for the Leap Motion devices requires an abundance of concentration as well.

So I have spend the last week pouring over the documentation and examples - and trying to get some code out. It's been both fun and frustrating. This is always what happens when you are a newb working in a new area.

And in upcoming posts I will talk in more detail about both these aspects.

But apart from coding for the device the other aspect is that device presents new possibilities for communicating via computer. In other words, for designers this is terra incognita. Most of us have very little experience with 3D interaction.

So far, I have taken a rather peculiar direction with learning about this device. I decided that I would not look at applications that are already published. I want my brain to come into this device free and unfettered by existing thinking and current dogma.

I want my brain and limbs to experience this thing and I want them to inform me about what is needed and where to go. Therefore I have only looked at code in the API docs and code in the examples. And simply playing with the device and trying to understand the numbers it sends out.

I am coming to see/feel several thought patterns.

1. This device is about using two hands. If you are only programming it for use by one hand then much could be done faster and more easily with mouse or touch pad or cursor keys.

2. The thing you are working on should be 3D. There is no advantage to using the Leap Motion for editing text or arranging photos and other flat stuff.

3. It's about moving things that are moving - particularly with a swinging or radial motion.

This device is for sculptors modelling cay, musicians playing drums, scientists folding proteins and, of course, for gamers.

If what you want to do can be done on a flat surface then finding a good use for the Leap Motion is going to be challenging. I am not saying it can't be done, but more that cool ways have not yet been thought of. And much as we all like Minority Report the great portion of what we see there could be done just as well with a touchpad. And furthermore the Leap Motion device has, as of this writing, only a very small field of vision and must be in a position where hand activity is just above the device.

The use that jumps out at me is where you pick up an object with one hand and do something to the object with the other hand. So think of a jeweler picking up and turning a ring with one hand and using a tool to add detail with the other hand. Or think of an engineer picking up the Starship Enterprise with one hand, rolling it around and inspecting for hull integrity with the other hand. Do remember that with computers we can make anything happen. Or think of a chemist holding a grain of salt in the palm of one hand while approaching it to the sun with the other hand.

The point is that there are going to be many uses for the Leap Motion device but they will only start appearing when we stop thinking with a paper and pencil mentality.

"Toto, I have a feeling we are not in Flatland anymore..."

Link to some of my code: