High-Presence, Low-Bandwidth, Apparent 3-D Video-Conferencing With a Single Camera

Executive Summary
University of Virginia researchers have developed a novel method for enhancing users’ video-conferencing experience by enabling apparent three-dimensionality with the use of a single camera. High-presence and direct eye-contact conversations are achieved under low-bandwidth requirements without the need for expensive motion-capture technologies or multi-camera arrays.

Background
Video-conferencing has become an increasingly popular way to engage in personal and business-related communications, with noted drawbacks stemming from participants’ lack of physical co-presence and technical demands for high bandwidth.

Typically, due to the physical displacement between a user’s camera and screen, participants cannot simultaneously watch and make eye contact with other users on-screen. This physical displacement alters micro-facial expressions that would normally occur in face-to-face meetings and may make participants feel uncomfortable, potentially damaging conversations and relationships.

While current technologies exist to enhance co-presence, they often consist of elaborate motion-tracking systems and multi-camera arrays, which are expensive and demand high bandwidth requirements, making them undesirable in many video-conferencing situations.

Invention Description
Developed by quantitative psychology researchers, the invention is a method for providing high-presence, real-time video-conferencing through simulated three-dimensionality. Using a single camera, each participant’s face and movements are captured, tracked and reduced to a few data points using statistical modeling. In this way, important facial features are captured and transmitted over very low bandwidth (a few hundred bytes per frame).

At the remote device, motion parallax is used to reconstruct a simulated 3-D image that is modified to reflect the viewing angle and location of the recipient viewer, creating a near-photorealistic video of each participant’s face. This feature also allows apparent eye contact to be made and broken, as it is in in-person conversation.

View a Demo
View a brief demo movie of the technology in action.

Advantages
The invention offers several advantages over conventional technologies:

• High-presence video-conferencing is achieved through a single camera, minimizing costs and the need for any special equipment
• Direct eye contact allows for more life-like conversation
• Use of statistical facial modeling allows for high-quality video transmission over low bandwidths, making it ideally suited for mobile technology

Potential Applications
The invention can be used to enhance a wide range of applications for which effective communication is imperative, such as:

• Video communications: enhancing conversations while reducing bandwidth, especially attractive for mobile devices
• Online gaming and animation: allowing users to manipulate avatar facial expressions
• Motion-picture industry: allowing easy post-alterations to an actor’s facial expressions, better reflecting a dubbed recording
• Medical, marketing and sports industries: increasing the consumer’s sense of physical presence at an event