Wednesday, October 9, 2013

Leap + Three.js: Phalanges R7 Video


Description

The goal is to build a web app with the procedures required to display - correctly and in real-time - a user-manipulated 3D hand - or claw - or appendage. This demo shows what is still a work in progress.

Source Code here: https://github.com/jaanga/gestification/tree/gh-pages/cookbook/phalanges

Live demo here: http://jaanga.github.io/gestification/cookbook/phalanges/r7/phalanges.html
- Requires a Leap Motion device

The motion is captured using a Leap Motion device. See http://leapmotion.com

The 3D graphics are generated using the Three.js JavaScript library. See http://threejs.org

The video was recorded using CamStudio. http://http://camstudio.org/ There needs to be work on capturing data at a better frame rate.


Phalanges R7 - Requires Leap Motion Device to operate

Transcript

Hello this is Theo. And You're looking at the new Phalanges Release 7
Phalanges is Latin term for finger bones
It's October 8th 2013,  here in San Francisco
What you're seeing is the movements of my hand recreated in a 3D space
I'm using the Leap Motion device to capture the actual movements of my hand and fingers as I speak
The graphics you see in the video are being generated on screen using the three.js JavaScript library
The issue in all this is that the Leap device cannot see all your fingers all the time
So whenever one of the  colored block disappears, it means that the Leap Device cannot see that finger
The objective of the code is to keep all the fingers  - the gray box-like objects - visible at all times.
The second objective have fingers *not* go off in crazy directions.
As you can see there's a fairly good connection, but it's not perfect.
I can make my hand  pitch - roll  - and Yaw
I can wiggle my fingers
Mostly the fingers the visible and not too crooked
And it's a lot better than Release 1
Anyway, All of this very much a work in progress.
What you are looking at is example or cookbook code.
It's a program intended to be used as the basis for further development
So it's not a thing beauty.
For example, you can see All the dummy objects to make sure the fingers point in the right direction
They are just here for testing and won't be visible in later programs
Speaking of later programs
The next generation of code based on this work will be out very soon'
Two major features will be getting into this code:
First, You will be able to use this these algorithms to save data in the industry-standard BVH file format.
Secondly, you'll be able use this code to display human-like hands, or animal claws or robot appendages or whatever'
So there's a lot more to be coming out out of this code.
But for the moment, this is Theo, saying 'Bye for now...'