Michael Frank Deering: Virtual Reality Projects
This page describes several Virtual Reality Hardware and Software projects I’ve designed or played a major role in. (There is some intentional duplication of filing for some of these projects with my “Graphics Software” section.)
Sun: Java 3D
I (with others) architected a 3D scene graph API for Java: Java 3D. This software has been and still is widely available for a number of platforms: Sun, HP, SGI, Apple, Linux, and Microsoft. One of the “firsts” for Java 3D was the built-in support of complex VR viewing models (without the application writer or end-user having to deal with most of the details).
This was a last minute submission to the VRML II competition. (Currently, the file is in inverse page order!)
This was a research project I designed to see what would happen when a high-resolution virtual reality environment was always available. In the main, HoloSketch was a system for creating, editing, and animating 3D objects all in VR. But HoloSketch was also used as a 3D playback environment, a demo system, and a research platform for debugging many 3D graphics concepts: compressed geometry, 3D scene graphs, and per-pixel programmable shading hardware. There was never and intent to release HoloSketch as a commercial product, but a sub-set of HoloSketch was released as a real-time demo with all Creator3D systems, and the design of Java 3D was heavily influenced by our experience with HoloSketch. The second version of the Virtual Portal was run by HoloSketch, and several HoloSketch and HoloSketch related papers were produced.
This was a research project to demonstrate the ability to convert either RenderManTM or real world cameras output to 3D micro-polygons to compressed geometry. Streams of per-frame compressed geometry could then later be played back in a head-tracked stereoscopic VR environment.
Sun: Experimental Displays
Many of my research projects at Sun involved putting together displays with multiple video projectors. This projects collects several of them together.
HoloCam was a research prototype 32-channel live video camera I designed and built at Sun to experiment with large array of camera data. It has always been amusing that because my commercial products used polygon rendering, I was considered “anti-image based rendering”, while at the same time my HoloCam prototype is often referenced as one of the first experiments in image based rendering. Many other people did commercialize large arrays of cameras for commercial products: a film camera for “frozen in time” effects; but nearly all of these came after my first HoloCam work prior to 1992. More recently Marc Lavoy and his students at Stanford has been doing many interesting things with 1 and 2 dimensional arrays of cameras that go well beyond the early HoloCam work.
Sun: The Virtual Portal
The Virtual Portal was (simultaneously with the CAVETM) the world’s first multi-screen projective head-tracked immerse stereo display environment. Initially based on three Sun GT accelerators, several versions of the Virtual Portal were built over the years.
Sun: The Virtual Lathe
The Virtual Lathe was an early Virtual Reality demo allowing a user to use a 3D wand tip to grind away at a virtual piece of spinning material.
Sun: High Resolution Virtual Reality / Augmented Reality
My first Sun paper on Virtual Reality was about my “High Resolution Virtual Workstation” project. The paper and some additional images and materials are collected on this page.
Schlumberger: Virtual Reality fish tank and Head Mounted Display
During the mid 1980′s I did my first Virtual Reality work at Schlumberger Palo Alto Research with several other lab members. This page describes some of this early work, most of which was never published.
Press, TV shots, Corporate Demos
Like any good VR system of the day, my various VR demos (as described above) got a fair amount of both print articles and video segments in the popular press. Most of these were light pieces showing VR technology to the public at large. I remember a sequence on HoloSketch making it onto the SciFi channel in its early days; I was upset because my cable operator didn’t yet carry the channel, so I never saw the final piece. Press came in from all over the world to make their own shoots (OK, some of them were already in the area for other shots). You could tell the higher value production teams by the size: was their a separate audio person and camera person? A really high end team had its own makeup person (I look better with lots of powder, something about the specular power of refraction of skin on the top of the head). I also did several production shoots of my own with rental chimerical video equipment. These were used to make internal Sun videos, videos for paper submissions, and to make B-roll available for press that may have not gotten all the shots they wanted. The press generally has high ethical standards; whenever possible they would shoot their own original material; the corporate supplied “B-roll” was a back-up that they could use to fill in gaps.
From a Sun point of view, the internal and trade-show demos of my VR work was primary useful in educating customers that Sun had a graphics story. I remember one customer demo day, in which the (large corporate) customer had spent most of the day looking at the traditional Sun products, with the VR demo a “cake” at the end of the day. After seeing the VR demo, the customer expressed that he hadn’t realized that Sun had sophisticated 3D graphics line, and went back into discussions the next day and bought a number of (my) 3D workstations. No, he didn’t buy the VR, but the VR got point across.