Old 2.0: How Old becomes the New New

The San Francisco WebGL Developers Meetup held on 15 May at CBS Interactive was in many ways even more mind-boggling than our normal meetup.

The usual type of things we see at the meetup is some sort of visualization that we have all seen before. The only difference is that we last saw the thing in a mega movie such as Minority Report or on a $75 game DVD running on a computer with dual GPUs. And now we are seeing it in a browser, for free, no plugin necessary. It's so much deja view all over again that we can hardly Yogi Berra it.  It's huge and we think its normal.

Things are moving so fast and yet it feels like we are standing still. Is there a reason for this?  The present and the future are fast moving, but we actually live slowly in the present and present. Is it because we live in our own legacy?

And that was the fun fact of this WebGL meetup. Every demo was trying to speed up the past, to bring our legacy up to modern capability, to make the past go faster miles an hour.

Let me count the ways.

First up: Tony Parisi - along with Remi Arnaud and Fabrice Robinet - showed off glFT. What does glFT do? It's like a hose that sucks 3D models out of people's hard disks and gets them up into the cloud. If we are going to move from Computer Aided Design to Internet Aided Design then we have to get the last twenty years of design data online - quickly, easily and cheaply - without coders always having to rescue stuff. And Tony gave us a presentation of an open-source and probable industry-standard method for doing so - truly a free glFT.

After Tony, up came Aleksandar (Aki) Rodic who showed a 3D editor built using Three.js and connected to the cloud via Google Drive. I logged in to the online demo while Aki talked and happily added an icosahedron and scaled torus knots while he demoed. So what was the old bit here? The editor did not have much power. Apart from the amazing collaboration ability - it was more like AutoCAD 2.18 from 1983 than an editor of 2013. But that's not the point - and I will come back to Aki's demo at the end of this post.

Then came Iker Zugaza from Ludei. OMG. Now we are not just talking old. We are talking archeology. Iker showed us the Ludei game system running on old Blackberry tablets, early Kindles and even an iPad 1.  Each of these ancient devices displayed 3D graphics with webGL-like speed even if the device did not support WebGL.

More than that: you can use your wacky Android dev tool or Objection-C or whatever. Just get your stuff to run in a browser, send the blob to Ludei and they will send you back the code for just about any platform out there - living or dead. For free (well until you get enough users).

Finally there was Robin Willis from sunglass.io.  Think GitHub for designers. Use the tools you like to use. SketchUp. Inventor SolidWorks or whatever, and then easily and quickly obtain a representation that can be shared, and viewed and commented (and blamed) by the whole design team online. Your project is no longer stuck on a hard disk in some office but is available to the world.

So all of the demos gave the old legacy data and devices a kick in the pants and said "back to the future with you!" All very cool.

But the demo I keep thinking about is Aki Rodic's collaborative 3D edit demo.

And I think I am beginning to understand why.

When we think of the past we think of all that data from 3D Max, Maya, ProE, SolidWorks, Rhino and whatever. trillions of point and faces.

But the future is not about data, it's about code.  You don't send monster data down the interpipes, you send the code for building and animating monsters. Architects don't send you buildings, they send you the instructions for creating the building. Your DNA does not contain a miniature you, it only holds the the code for re-creating you.

Three.js is not a tool for building CAD models, it's a library of WebGL tools. It's the equivalent of DNA. What Aki is sending around is just the instructions on how to use the DNA. He's transferring the Internet equivalent of RNA. I think the future of 3D on the web will be based on techniques that move code as much as data. It's a fast and proven idea.

Speaking of fast and proven, hats off to Tony Parisi for yet again showing that he 'gets' what happening in 3D and 'puts' on a great event.

FGx Globe r3.1 ~ Bug Fix Release - Nonetheless Very Nice Bugs

Here is the description on GitHub

The drop-down lists have received a lot of attention. It should now be a fairly straightforward process to select a different theme or map. Also what gets closed stays closed. This includes windows that have been closed and flights that have been terminated. More world maps have been added to the globe and the TBD panel is beginning to have a healthy set of issues to fix and features to add. 

Work has also continued on bringing in 3D models of airplanes. The current issue has to with dealing with what to do while waiting during the loading times for the 3D models, but this appears to be a solvable issue.

And there's a lot more that could be said.  I had a look at the app in FireFox and Internet Explorer. Given that I code on and for the Google Chrome browser there are a ton of issues.

The nice thing is that Three.js - the premier WebGL library for dummies - is sort of running on Internet Explorer. It's just not very snappy. The beauty of this scenario is that people can actually see what they are missing by being on IE.

The real issue, the lesson-learned of today, is something quite different.  I know that I suck at using the Chrome JavaScript developer console. I am working on this. Last night I even fell asleep watching a Paul Irish video on this very topic. But it turns out that I suck even more at the FireFox and Internet Explorer developer consoles. And don't even begin to ask me about Safari and Opera developer consoles.

So not only are the languages interpreted differently on each browser but also the available debugging consoles are radically different.

Even more interesting was that some of the issues definitely had to do with jQuery. And jQuery is meant to be the solid working-class tool for cross-browser support.

Would anybody kindly console me?

FGx Globe r3 ~ FlightGear Planes Now In the Sky

One of my several projects over the last few weeks has been the development of a flight simulator visualizer.  You may well ask what is a 'flight simulator visualizer'?

FlightGear is a popular FOSS flight simulator program. Players like to share their in-the-air status online. A JSON feed for this data is available from FGX's Crossfeed. Their current visualizer for this flight simulator data looks like this. I thought is might be fun to wrap the data around a globe.

I have just started rev 3 of my FGx Globe. Here's is what I just wrote about the new rev:

This revision begins to feel as if its getting somewhere. The code is simpler, shorter and does more than before. In particular there is much better separation between the user interface and the 3D world. The user interface is now built using jQuery UI while the 3D all resides in a separate file embedded in an iframe. 

The user interface now provides a table that displays all the flights currently underway. For each flight there is a clickable button that opens a separate window. Multiple windows may be opened - each of which displays the current position and navigation data for a single plane. You may select from a variety of map data sources. The user interface is theme-able with all the standard 20+ jQuery UI themes available at the click of a button. All significant settings are savable as 'permalinks'. 

On the 3D side, the data-handling procedures have been re-written in a much more simple and more straightforward fashion. Also, there first steps have been taken introduce more realistic planes. 

There is still much unfinished work in this build. In particular the only way to remove terminated flights id to reload the window and the only way to completely delete a window is to reset the permalink. And there are several issues with the drop-down lists. Please note that all building and testing has been on Google Chrome. Other browsers will have issues. Other issues are listed in the TBD panel of the main menu.

So the new thing here is that I am using jQuery. I have gone over to the dark side. I have drunk the Kool-Aid.   I say this because, for the most part I code in a very simple and spare style.

The Three.js site says this about the code:

The aim of the project is to create a lightweight 3D library with a very low level of complexity — in other words, for dummies. 

Well, I have tried to match that style. In some ways this is because I resemble the categorization and in other ways because the style is simple and pure. But perhaps the most telling reason is because I am not a programmer.  I actually don't try to write code or build a program. I am a designer. What I am trying to design is a brain visualization thingy or a stock market snooper or whatever.

I could do this using pencil and paper or clay or some such traditional media. But I just happen to like to use JavaScript as my design media.

I plan to write more about coding while being a designer, but suffice it to say that much of my work sticks to "for", "if" and "=" along with parentheses, commas and semi-colons.

So the move into adding jQuery to my repertoire is a big one for me.  I think I will most likely become schizophrenic: Writing in Three.js style on some days and jQuery style on other days.

But not to worry, on both types of days my brain will very likely be placed squarely in the middle of the Three.js target market.

Jaanga Pivots Again

Much as I like the thought and idea of blogging in 3D, it looks like I don't do enough of it.  I am already doing more than enough 3D stuff on GitHub, so the work here on the blog is being neglected.

And, on the other hand, there are a lot of matters that I would like to be discussing that relate to coding and 3D and designing that I am not writing enough about.

So here's what I am thinking of doing for the time being:

Jaanga.com - this site - becomes all writing and then all the demos and code are on GitHub - at jaanga.github.io and gitihub.com/jaanga.

Well, the demos and code are already there there. So all I need to do is link to these from time to time from posts here in this blog.

I envisage writing short text-only posts. This could easily be a Tumblr thing, but since I have enough material on Blogger I don't feel the urge to do a big switch. And let's not get into a Blogger versus WordPress thing. I love them all. All of them are adding so much to the blogosphere both in terms of content but also in ways of processing that content. Hats-off to all three.

Easy Way to Embed Three.js into Blogger: Use an Iframe

An easy way to add Three.js to Blogger is to embed a Three.js page as the source in an IFRAME.

And you can even edit the code in the post causing the animation to update.

For example, change camera.position.z to 800 or color: to 0cff0000.

Here's the code for the iframe that causes the Three.js online editor to appear here.

<iframe height="40%" src="http://mrdoob.com/projects/htmleditor/" style="border: medium solid rgb(0, 0, 0);" width="100%"></iframe>

It's best to keep the iframe  height well under 100%, otherwise the frame can obscure your code while you are editing the post.

Using Three.js with DAT.GUI User Interface Controls

Silly me. I spent the day building an app to show you how Three.js can make good use of DAT.GUI. But now I cannot get the app to run inside Blogger without taking over the top of the screen.

So you will only be able to view my 3D efforts for the day after the jump.

WebGL on Chromebook

demo broken here. but runs OK when post is opened

The new Google Chromebook landed on my desk today. This is the new $249 laptop from Google. It's a very nice little laptop. OK, so it's not a MacBook Air, nevertheless it compares with $2,000 ultrabooks of a couple of years ago.

The Brain of Richard App

Use your pointing device to zoom, pan & rotate the brain of Richard

Link
http://jaanga.github.com/brainofrichard/

My friend Richard has a brain tumor. To be more precise he has an oligodendroglioma in his posterior cingulate gyrus, that is to say Brodmann area 23. His brain has been scanned on numerous occasions using an MRI scanner at UCSF. Richard has been to obtain the scan data and made it available to me - all are 2D images in DICOM format. DICOM is a frequently used data format in medical imaging as it allows patient and other alphanumeric and binary data to be stored along with digital images.

One of the many scans of the brain of Richard

The ZIP file I received was for a scan taken on 18 January 2012 and has over 12 hundred images. These are all 2D images and mostly look like the image just above. The images are in a variety of series of sections or slices and each scans appears to be about two millimeters apart from the next. There's a series of horizontal scans, vertical slices from front to back and side to side.

The question that Richard and I posed each other is would it be possible to design and build an app that could somehow recombine images and display them in some 3D manner? And, if so, would the result be useful in anyway. It should be pointed out that neither Richard nor I have any experience with medical imaging though we both have good experience with 3D and digital imaging.

Our first thought was to see if we could reassemble the images in 3D in an approximation of the original and then to reduce the opacity of scans so as to be able to see through a series of scans. This post is the first report on the results of the experiment.

The software we used is Three.js - the WebGL framework that enables GPU accelerated 3D graphics in a browser. Three.js is able to read and display 2D images as 'textures' applied to 3D objects. Three.js can read PNG, JPG and GIF only.

The first operation operation was to convert DICOM files to PNG (the most open format). On a Windows computer I was able to do the conversions using Irfanview and XnView - both are popular and readily available Windows apps available at no charge. They both handle converting, renaming and making the background transparent.

The next operation was to load the images into a web page using Three.js. Below is the code I used. I won't explain all the variables. The critical elements were setting the material to be double-sided and transparent. Suffice it to say, it's not rocket science.

 
brainApp.buildBrain = function() {
  if ( scans ) { scene.remove(scans); }
  geometry = new THREE.PlaneGeometry( 200, 200, 10, 10 );
  scans = new THREE.Object3D();
  scans.current = 0;
  for (var i = 1, l = hack.count; i <= l; i++) {
    map = THREE.ImageUtils.loadTexture( hack.dir + i + '.png' );
    material = new THREE.MeshBasicMaterial( { map: map, opacity: hack.opacityDefault, side: THREE.DoubleSide, transparent: true} );
    mesh = new THREE.Mesh( geometry, material );
    mesh.rotation.x = hack.angle;
mesh.position.set(0, hack.startY + i * hack.deltaY, hack.startZ + i * hack.deltaZ);
   scans.add( mesh );
  }
  scene.add( scans );
};

It's hard to describe the thrill I had when I dealt with enough enough issues so that the images finally appeared. I am a designer. When I design something I have a perception of what I am designing in my brain. Nevertheless there is that moment when a thing you are working on begins to transform itself from raw material into the thing it's meant to be, begins to come alive. In architecture, what was a the pile of wood begins to be a home. The lump of marble become a sculpture. In this case, there I was a hitting the enter key and then watching a bunch of JavaScript code transform into Richard. Magic.

Over the last two or three weeks I have spent a number of hours on the project and identified a number of interesting things:

There is a website that displays MRI scans of vegetables. Have a look at the wonderful InsideInsides.com. The developer, Andy Ellison, has kindly authorized me to take apart his animated GIF files and reassemble them in 3D. I have added a cactus and artichoke scan to the File Open menu

The right side menu is built using the FOSS dat.gui - A lightweight controller library for JavaScript. This project was my first foray into the work of dat.gui. The interesting thing here is that the dat.gui that comes with Three.js is an older version. The dat.gui from the home site has many more features and fewer bugs.

There is a playable demo included which is part of an ongoing effort to "Put Theo in a Bottle". The demo was built using the HTML5 audio element. The interesting thing here is that the audio element has a ton of cool features that allow it to be used as a timeline controller.

All of these and more are worth a double-click. So over the next few days, I plan to write a detailed post about each of these thoughts.

But more than that, we are beginning to consider where to go next with all this. There is a huge amount of data to explore. The data may contain all sorts of fascinating things to play with. For example MRI scans may contain anisotropic data, Three.js supports anisotropic rendering. Could that be a useful thing to explore?

After I pointed Andy Ellison - an MRI technologist at Boston University Medical School - to the app, he responded as follows

I'm super impressed. Usually we need the original DICOM format to do that. Very clever, and built into a browser, wicked.

If Andy has this thought regarding Richard's and my first little baby steps, then it might be fun to see what somebody who actually what they doing working on this sort of project...

Link
http://jaanga.github.com/brainofrichard/


Link to code for demo in header
http://jaanga.github.com/blode/#jaanga.githb.com/Blode/Brain-of-Richard-App


See also
http://www.nytimes.com/2012/10/09/health/labs-seek-new-ways-to-look-inside-the-body.html

Notes
The scans can take quite a while to load and then after the load they have to attain their transparency. Some of the files have over 30 megapixels of transparency to sort out. So it can take a minute or so so settle down. If you see a lot of black then the model has not finished loading. Note: There's a lot that can be done to optimize the process.

The other issue is trying to deal with 3D using a touchpad instead of a mouse move the models. If you are having issues with using the touchpad, try clicking on 'Zoom', 'Rotate' or 'Pan' in the right side menu. And that is clicking on not sliding the indicators. The 3D user interface is still at an early stage.

3D in Blogger with Three.js ~ Day 4: We Have Callbacks!

What's new today for blogging in 3D?

There's a new 'in your face' masthead to your right. Links to last year's apps have been lost for the moment, but a link to the new jaanga organization in GitHub has been added.

The really fun thing was that I added the ability for every post to have its own unique animation. Yesterday, every post could have different data or models but everything just spinned in an identical manner.

Book Review: WebGL: Up and Running

Book cover with the tonyBot (in running mode) superimposed

I have just finished my first reading of WebGL: Up and Running written by Tony Parisi.

Note and disclaimer: The author is my very good friend. I prepared demonstration material for the book. Tony mentions me by name three times in the book in glowing terms. Therefore you can assume that my point of view is not particularly neutral. On the contrary I am definitely biased in favor of Tony and his work.