Nimble Sense: Bring Your Hands into Virtual Reality & Beyond
Nimble Sense: Bring Your Hands into Virtual Reality & Beyond
Nimble Sense is a breakthrough depth sensing camera that captures your hands for VR input. Optimized for the Oculus Rift, but flexible.
Nimble Sense is a breakthrough depth sensing camera that captures your hands for VR input. Optimized for the Oculus Rift, but flexible. Read more
About this project
The Nimble Sense captures a 3D point cloud across a 110 degree field of view. Combined with the world’s most robust skeletal hand tracking software, the Nimble Sense delivers low-latency, accurate hand input to provide the simple experience of having both hands in VR. The Nimble Sense is perfectly suited for integration on an Oculus Rift or other head mounted display, but can also be mounted above a monitor or sit on your desk.
“I’ve tried just about every VR control scheme out there, and I have to say that the Nimble Sense is by far the most promising I’ve tried so far.” - Blair Renaud, Iris VR, makers of TECHNOLUST
"Nimble brings a level of interactivity to Ethereon that I've only dreamed of so I'm very excited to continue developing with this exciting new technology and seeing what incredible new experiences we can offer everyone in the future." - Tony Davidson, Innervision, makers of ETHEREON
"Having your hands in there with you in virtual reality takes immersion to a completely new level. I'm excited by how much scarier Spectre can become when you can literally reach out with your undead hands and steal the souls of your victims." - Ryan Anderson, Proscenium, makers of SPECTRE
“We've worked closely with Nimble VR to optimize time of flight for virtual reality. The result is a breakthrough in cost, accuracy, and power consumption!” - Bernd Buxbaum, pmdtechnologies, inventor of photonic mixer devices / time of flight technology
The Nimble Sense uses time-of-flight technology -- composed of a photonic mixer device from pmdtechnologies and a powerful, but eye-safe laser to capture a detailed 3D point cloud and an infrared image of the world. This 3D point cloud is interpreted by our skeletal hand tracking software to track the location, identity and joint angles of each finger. All of this information is exposed through an API, allowing software developers to design applications that allow you to pick up pieces on a virtual game board, control and manipulate virtual objects, and experience being in virtual reality like never before.
Our skeletal hand-tracking technology is accurate, low-latency and robust. We believe it's the best in the world.
Nimble Sense is a depth sensor the size of a pack of gum that can be mounted on an Oculus Rift DK2, an Oculus Rift DK1 or on a desk. If you are using the Nimble Sense with the Oculus Rift we have a gaming-grade USB cable that is flexible and long enough for use in VR.
We have been working to build the absolutely best hardware and software system for skeletal hand tracking, and we are ready to get Nimble Sense into the hands of developers and innovators. Our Kickstarter will give you early access to this breakthrough camera system, software development kit, and code examples so that you can develop breakthrough experiences in and out of VR.
We’re also releasing our VR application demos as open source to kickstart development.
Who we are:
Nimble VR is focused on building the best skeletal hand tracking software and hardware in the world for VR. Our team include experts in computer vision, computer graphics, and visual effects. Because we’ve supported every depth sensing camera on the market, we knew that we needed to create a new sensor to suit the needs of VR.
Risks and challenges
We realize that Nimble VR is a small team and that manufacturing hardware is a difficult process. We only felt confident about launching the Kickstarter after working with three experienced camera module makers in the United States and Asia to obtain quotes for manufacturing and the cost of goods sold. Two of the module makers were able to quickly build prototypes of time-of-flight cameras that met our spec.
While our product is simple to use, it combines cutting-edge hardware with sophisticated software. Creating such a product involves coordinating the supply of lasers, optics and very specialized ICs, as well as the implementation of advanced computer vision algorithms for understanding hands from a point cloud. We know we can do this through our interactions with module makers with extensive experience manufacturing depth sensing cameras and through the maturity of the software that we have already developed for skeletal hand tracking. Nevertheless, there is a risk of delay.
With the support of the Kickstarter developer community, we feel confident in our ability to deliver on the Nimble Sense.Learn about accountability on Kickstarter
Yes. With every Nimble Sense, we will include a 7-foot, flexible USB cord. The 14-foot gaming grade cable is optional.
Compared to gloves and controllers:
Gloves and controllers certainly have their place in VR input. However, we believe that there are many interactions where putting something on or even holding something in your hand is unnecessary and detracts from the user experience. This development kit is an opportunity to explore these experiences.
Compared to other markerless hand-tracking systems:
Our time-of-flight camera captures a dense 3D point cloud every 20 milliseconds that can be used to bring not only hands, but arms, legs, and even your desk into VR. The 3D point cloud of the real world is rendered exactly at the right scale and location in VR -- regardless of your IPD. The point cloud can even be shared and visualized by other users for multiplayer / social VR experiences.
Some stereo systems may only provide two infrared images, rather than dense 3D geometry. It's harder to visualize a stereo image pair from a different user's perspective, and stereo also assumes a specific IPD based on the spacing of the cameras.
Finally, we believe our skeletal hand-tracking technology is the best in the world, and will be rock solid by the time we deliver on the hardware. Here is how it currently compares with the LEAP Motion skeletal tracking:
We'll be showing our current prototype and software at meet-ups all over the Bay area (SFVR, SVVR Nov 6th) and in NY (NYVR Nov 20th). If you're around, come give it a try!
Like the pmd Camboard Pico (Nimble UX), we're using the Infineon / pmd sensor chip--the photonic mixer device. However, the laser, laser optics and receiving optics, as well as the electronics differ from the Camboard Pico. We're designing and building a new camera module optimized for VR, with a much wider field of view, longer range, and higher accuracy for hand-tracking.
We are an independent company from pmdtechnologies gmbh. However, we've worked closely with pmd on both the software for Nimble UX and the hardware of the Nimble Sense.
Unfortunately, not at launch. There are things we can do to the firmware and software to prevent multiple cameras from interfering with each other, but that's not something we can promise right now.
We want to support Android in the future, although frankly, that's not a small project. It likely will involve some image processing computation on the camera itself, bluetooth over to a phone and more processing on a phone. Basically, supporting Android will involve additional hardware and software. We're at least a year out for Android support at this point, but that's certainly on our roadmap.
Support this project
- (46 days)