We highlight two recent rechnology components we implemented for virtual environments.

For more information, contact 5581web_info@nrl.navy.mil.

Configurable Semi-Autonomic Animated Animal Characters in Interactive Virtual Reality Applications

A close up view of a virtual dog character
A close up view of a virtual dog character
Sequence of animated dog images
Animation sequence of the "Search" command. Top left: dog in the "Stop" state in the top-level state machine and in the "Sit" state in the animation state machine. Top center: transitioning to a "Search" state in the top-level machine requires a transition in the animation machine to a "Stand" state. Top right: the animation state machine has transitioned to a "Turn" state to orient the dog properly for the "Search" state in accordance with the user command to search in the direction away from the user's position. Middle row: the dog has reached the "Run" state in the animation machine. Bottom row: the dog accelerates between the left and right images. Animation blending makes the acceleration smooth.

In many virtual reality (VR) simulation and training applications, it is desirable and even critical to have computer controlled characters that can behave with a high degree of realism. The realism can be measured in two aspects. One is behavioral realism, or how real the characters act and respond to the commands and environments. The other aspect is visual realism.

Many efforts have been directed at animating human characters in applications such as interactive computer games. However, more research is needed for realistic animation of animal characters. Animals exist in environments for which security personnel are being trained, so we need to include realistic behavior and visual representations of animal characters in our training applications.

This study is focused on animal behavior and animation for VR applications. Synthetic animal characters are included in the system so that their behavior can be programmed and controlled digitally. A two stage state machine is used in the system. The top level state machine controls the animal behavior. The secondary state machine controls animation. Animation blending and procedure control are used to make the animation smooth.


Performance Measurements for the Microsoft Kinect Skeleton

Three data streams provided by the Kinect for Windows SDK, from left: depth with user index, skeleton, and video.
Three data streams provided by the Kinect for Windows SDK, from left: depth with user index, skeleton, and video.

The Microsoft Kinect for Xbox 360 ("Kinect") provides a convenient and inexpensive depth sensor and, with the Microsoft software development kit, a skeleton tracker. These have great potential to be useful as virtual environment (VE) control interfaces for avatars or for viewpoint control. In order to determine its suitability for our applications, we devised and conducted tests to measure standard performance specifications for tracking systems. We evaluated the noise, accuracy, resolution, and latency of the skeleton tracking software. We also measured the range
in which the person being tracked must be in order to
achieve these values.

Graph of noise vs. distance
The mean and maximum noise as a function of distance from the sensor showed an exponential fit. The shaded region denotes the optimal depth range.
Graph of pixels with player index vs. frame rate
We measured latency relative to a USB mouse. The frame rate depended on the number of pixels within human silhouettes.

Publications

2012 Ai, Z., and M. A. Livingston, "Configurable Semi-Autonomic Animated Animal Characters in Interactive Virtual Reality Applications", 5th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS 2012), IEEE Virtual Reality Conference, Orange County, California, 03/2012.  (208.41 KB)
2012 Livingston, M. A., J. Sebastian, Z. Ai, and J. Decker, "Performance Measurements for the Microsoft Kinect Skeleton", IEEE Virtual Reality, Orange County, California, 03/2012.  (114.9 KB)
2006 Schmidt, G. S., D. G. Brown, E. B. Tomlin, J. E. Swan II, and Y. Baillot, "Probabilistic Algorithms, Integration, and Empirical Evaluation for Disambiguating Multiple Selections in Frustum-Based Pointing", Journal of Multimedia, 06/2006.  (1.06 MB)
2006 Ni, T., G. S. Schmidt, O. G. Staadt, M. A. Livingston, R. Ball, and R. May, "Survey of Large High-Resolution Display Technologies, Techniques, and Applications", IEEE Virtual Reality, Alexandria, Virginia, 03/2006.  (609.92 KB)
2005 Livingston, M. A., and E. V. Herbst, "Interactive Operations for Visualization of Ad-hoc Sensor System Domains", IEEE Intl. Conference on Mobile and Ad-hoc Sensor Systems, Washington, D.C., 11/2005.  (1.7 MB)
2005 Klein, E., G. S. Schmidt, E. B. Tomlin, and D. G. Brown, "Dirt Cheap 3D Spatial Audio", Linux Journal, 10/2005.  (1.88 MB)
2005 Klein, E., G. S. Schmidt, E. B. Tomlin, D. G. Brown, and P. Violante, "Low Cost 3D Spatial Audio", ACM Symposium on Interactive 3D Graphics and Games, Washington, D.C., 04/2005.  (2.34 MB)
2005 Livingston, M. A., and E. V. Herbst, "Interactive Operations for Understanding Embedded Sensor Domains", ACM Symposium on Interactive 3D Graphics and Games, Washington, D.C., 04/2005.  (28.59 KB)
2003 Swan II, J. E., J. L. Gabbard, D. Hix, R. S. Schulman, and K. Pyo Kim, "A Comparative Study of User Performance in a Map-Based Virtual Environment", IEEE Virtual Reality 2003, Los Angeles, California, pp. 259-266, 03/2003.  (235.95 KB)
2001 Maxwell, D. B., R. King, and A. Butler, "Design Synthesis in a Virtual Environment", Proceeding of the 2001 ASME Design Engineering Technical Conference, Pittsburgh, PA, 09/2001.  (468.58 KB)
2000 Rosenblum, L. J., "Virtual and Augmented Reality 2020", IEEE Computer Graphics & Applications, vol. 20, issue 1, pp. 38-39, 01-02/2000.  (209.57 KB)
1999 Gabbard, J. L., D. Hix, and J. E. Swan II, "User-Centered Design and Evaluation of Virtual Environments", IEEE Computer Graphics & Applications, vol. 19, issue 6, pp. 51-59, 11-12/2009, 1999.  (395.33 KB)
1999 Cohen, P., D. McGee, S. Oviatt, L. Wu, J. Clow, R. King, S. J. Julier, and L. J. Rosenblum, "Multimodal Interaction for 2D and 3D Environments", Projects In VR, IEEE Computer Graphics and Applications, vol. 19, issue 4, pp. 10-13, 07-08/1999.  (114.04 KB)
1999 Kuo, E., M. Lanzagorta, R. Rosenberg, S. J. Julier, and J. Summers, "VR Scientific Visualization in the GROTTO", IEEE Virtual Reality '99, Houston, Texas, 1999.  (13.99 KB)
1999 Hix, D., J. E. Swan II, J. L. Gabbard, M. McGee, J. Durbin, and T. King, "User-Centered Design and Evaluation of a Real-Time Battlefield Visualization Virtual Environment", IEEE Virtual Reality '99, Houston, Texas, IEEE Computer Society Press, pp. 96-103, 1999.  (338.45 KB)
1999 Julier, S. J., R. King, B. Colbert, J. Durbin, and L. J. Rosenblum, "The Software Architecture of a Real-Time Battlefield Visualization Virtual Environment", IEEE Virtual Reality '99, Houston, Texas, IEEE Computer Society Press, 1999.  (449.5 KB)
1998 Durbin, J., J. E. Swan II, B. Colbert, J. Crowe, R. King, T. King, C. Scannell, Z. Wartell, and T. Welsh, "Battlefield Visualization on the Responsive Workbench", IEEE Visualization 1998, Research Triangle Park, North Carolina, IEEE Computer Society Press, pp. 463-466, 10/1998.  (521.16 KB)
1998 Durbin, J., S. J. Julier, B. Colbert, J. Crowe, B. Doyle, R. King, T. King, C. Scannell, Z. Wartell, and T. Welsh, "Making Information Overload Work: The Dragon software system on a Virtual Reality Responsive Workbench", 1998 SPIE AeroSense Conference, vol. 3393, pp. 96-107, 04/1998.  (2.26 MB)
1997 Tate, D.., L.. Sibert, and T. King, "Using Virtual Environments for Firefighting Training", IEEE Computer Graphics & Applications, vol. 17, issue 6, pp. 23-29, 11/1997.  (413.13 KB)
1997 Rosenblum, L. J., "Virtual Reality: Out of the Laboratory", Virtual Systems and Multimedia (VSMM) '97, Geneva, Switzerland, 09/1997.  (297.12 KB)
1997 Durbin, J., B. Doyle, D.. Tate, and R. King, "Situational Awareness Using the VR Responsive Workbench", IEEE Computer Graphics & Applications, vol. 17, issue 4, pp. 12-13, 07-08/1997.  (8.88 KB)
1997 Rosenblum, L. J., J. Durbin, and B. Doyle, "The Virtual Reality Responsive Workbench: Applications and Experiences", British Computer Society Conference on Virtual Worlds on the WWW, Internet, and Networks, Bradford, UK, 04/1997.  (60.83 KB)
1997 Tate, D.., L.. Sibert, and T. King, "Virtual Environments for Shipboard Firefighting Training", IEEE 1997 Virtual Reality International Annual Symposium, Albuquerque, New Mexico, IEEE Computer Society Press, pp. 61-68, 03/1997.  (413.13 KB)
1997 Rosenblum, L. J., and R. A. Cross, "Challenges in Virtual Reality", Visualization and Modelling: Academic Press, pp. 325-339, 1997.
1996 Rosenblum, L. J., J. Durbin, U. Obeyeskare, L.. Sibert, D.. Tate, J. Templeman, J. Agrawal, D. Fasulo, T. Meyer, G. Newton, et al., "Shipboard VR: From Damage Control to Design", IEEE Computer Graphics & Applications, vol. 16, issue 6, pp. 10-13, 11/1996.  (1.17 MB)
1995 Cross, R. A., and L. J. Rosenblum, "Virtual Reality: Research Issues and Applications", British Computer Society Conference on Visualization and Modelling, Leeds, UK, 12/1995.
1995 Rosenblum, L. J., S.. Bryson, and S. K. Feiner, "Virtual Reality Unbound", IEEE Computer Graphics & Applications, vol. 15, issue 5, pp. 19-21, 09/1995.
1995 Earnshaw, R. A., and L. J. Rosenblum, "Virtual Reality, Visualization, and their Applications", Interface '95, 1995.