Tuesday, September 3, 2013

JavaScript App to Record, Save and Replay Leap Motion Gestures in 3D

The Leap Motion device leads the way into new computer interfaces designed for specifically for our hands. Why? Because all other devices (mice, pianos, steering wheels whatever) require that the hands touch, hold or manipulate some other things. The Leap Motion device is the first device that captures your hand and finger movements in a completely free and unfettered manner.

Being the first device of this kind has its issues. There is not a lot of software for the device. There are not many good tools for designing software. And there really isn't even a good idea as to what the best tools should be or should do.

Frankly, I think this is amazing. This is one of the very rare occasions when we have a 'green field site' that doesn't even have a green field.

So what fun things need too be addressed first? Well, one of the main things is that there is no way of recording the movement of your hands and fingers and then replying the gestures and being to read the numeric data as well as view a representation in 3D. And perhaps more interesting there is no simple, easy-to-understand FOSS method for recording motions.  Or maybe there is but I haven't seen it.

There was, however, a great first attempt. Kai Chung put together two code examples, 'Leap Motion Recorder' and 'Leap Motion Replayer', available from here:

http://js.leapmotion.com/examples.

There are issues with both the apps. The recorder app provides no method for saving data and the replayer app only replays just the one file it is hardwired to replay and ut has no method for selecting and opening files. And in terms of helping programmers, these apps are coded to work with an early beta version of the Leap Motion software and do not seem to work when linked to the current version of the JavaScript Leap library.

But now, as of today, there is 'jestPlay'. If you have a Leap Motion device, try out the app here:

http://jaanga.github.io/gestification/cookbook/jest-play/r1/jest-play.html

There is a replay only version in the works - so that people without the device can replay gestures. It should be available shortly.

The jestPlay app enables you to record your hand movements by saving data from the device to your computer as JSON files. Once saved, you can open these files and watch a full 3D replay of the movements.

The app is a 'cookbook' style app. It is not a fully-featured or even a finished app. It does, however, provide you with a simple working example in just over two hundred lines of very simple JavaScript code that you can use to start developing your own code.

The app provides full access to your operating systems file save and file open dialog boxes
- which are features not normally found in JavaScript as they were recently introduced in HTML 5.

Based on the Three.js library, the jestPlay app allows you to zoom, pan and and rotate the views of the replays - so from another person's point of view - you can see your handiwork .

Source Code
https://github.com/jaanga/gestification/tree/gh-pages/cookbook/jest-play