About this project
What people are saying about ZapBox
What is ZapBox?
ZapBox is an affordable way to experience Mixed Reality and Room-Scale Virtual Reality using the power of your smartphone.
ZapBox combines physical components with advanced software to provide magical Mixed Reality experiences. Insert your smartphone into the ZapBox headset, start the ZapBox app, and step into a whole new world of interactive content.
The ZapBox app displays the live feed from your smartphone’s camera so you can see what’s around you in the room. The magic part is that it can also display virtual objects that appear anchored to the real world, so you can change your viewpoint by simply moving around.
That’s pretty awesome already, but that’s not all. ZapBox also works out the 3D position and orientation of the included pair of handheld controllers to offer natural interaction with content simply by reaching out and touching it.
This combination of a view of the real world combined with virtual objects that appear to be anchored in 3D space is known as Mixed Reality. For just $30 including shipping ZapBox unlocks this entirely new medium by making it accessible, affordable, and enjoyable for all. Huzzah!
You may have heard about Mixed Reality (MR) before, perhaps from Magic Leap or Microsoft HoloLens, but the technology is not currently very widely accessible - a HoloLens Developer Edition is now available but will set you back $3,000.
We set ourselves the challenge of creating a much more affordable MR solution. How could we make Magic Leap, “Magic Cheap”? :)
Inspired by Google Cardboard, we realised we could combine the power of existing smartphones with some simple physical components to offer MR experiences. With ZapBox, the equation is simple: More Cardboard = More Awesome.
What will I be able to do with ZapBox?
We’ll provide free experiences with ZapBox at launch so you can jump into Mixed Reality right away. The video at the top shows some of the content you’ll be able to try for yourself, including:
- Painting - ZapBrush, where you can paint in 3D space and walk around your creations
- Mini-golf - play mini-golf in your hallway complete with an animated windmill
- Dancing mini-game - Wii Fit meets brain training, a fun mini-game to test your dexterity
- Xylophone - Play a virtual musical instrument
- Explore Mars - Send a probe to the surface of Mars and discover fun facts
The content we build here at Zappar HQ for ZapBox is only the start - ZapBox is also fully open to developers. Using our ZapWorks platform we’re empowering the creative community so they can make their own experiences and easily share them with ZapBox users, straight into the ZapBox app - there’s no need to submit and manage separate app submissions for each piece of content.
All of that, in a box, including shipping, for just $30. We’re crazy good to you guys.
To keep shipping costs down we’re planning to flat-pack the headset and controllers but don’t let that scare you - assembly will be straightforward and full instructions will be provided. You’ll be ready to go in no time.
What’s not in the box, but is still important?
ZapBox App - The free ZapBox app is your one-stop shop to experience ZapBox content. You’ll use the app to build a map of the codes you’ve placed around your room, and then to browse and experience content.
ZapWorks Studio - ZapWorks is our content authoring platform which allows developers to create their own content for ZapBox, and to instantly preview and share their content with other users through the ZapBox app. ZapWorks Studio is available for free for non-commercial use.
Smartphone - You'll need a smartphone to use ZapBox. For iOS users an iPhone 6 or later is recommended. On Android, most higher-end devices from the last few years should be fine, but see the FAQ at the bottom of the page for more details.
A closer look at the stretch goals
How ZapBox works
ZapBox includes a set of physical markers (you guessed it, made of cardboard) that you place around your room. We call these markers “pointcodes” and we’ve designed them to be detected and identified in camera images very quickly, even when they are small or if there are many of them in view at once.
Thanks to the joy of maths [or math for our American friends] and our crack team of software wizards the ZapBox app is able to build a map of where all the pointcodes are in your room, and then use that map to work out your exact position and viewing direction using the pointcodes it can see at any time. We also use pointcodes on the ZapBox controllers to allow us to fully track them in 3D whenever they are visible to the camera.
As well as Mixed Reality, ZapBox can also offer Room-Scale Virtual Reality experiences. The virtual content can completely cover up the camera feed shown in the background, putting you into an immersive, entirely virtual environment that will react to your movement in space. Any motion of the controllers in the real world will also be accurately mapped into the VR view to provide natural interaction with content.
ZapBox vs Other Devices
There’s a lot of devices out there promising brand new Mixed Reality or Virtual Reality experiences. ZapBox occupies a unique position in terms of the amount of functionality available at a previously unheard of price-point.
Check out the table below for how the different devices stack up. We may be slightly biased, but you can’t deny ZapBox ticks a lot of boxes for just $30!
How do I get started with my ZapBox?
Getting set up with ZapBox is super-simple. Place your pointcodes, get the app, and away you go. You’ll be enjoying sweet Mixed Reality content within minutes. Check out the infographic below for some more detailed steps.
How do I develop content for ZapBox?
With your support and the imagination of the creative developer community the possibilities really are endless.
The good news is that our R&D team have cracked the heavy lifting stuff of making the underlying technology work from a smartphone (what else were the PhDs for?!).
Now the fun really begins. We’re developing the ZapBox app; plumbing the code into our ZapWorks platform; building the content experiences to ship with ZapBox; and finalising the product specification for manufacture with our partners.
If we hit our goal we plan on having ZapBox in your hands by 31st April 2017.
Who are we, and how do we know so much about this stuff?
Glad you asked!
We're Zappar, a young and friendly company based in (sometimes) sunny London. We are world leaders in mobile AR & VR, with many years of experience both in research that furthers the capability of the technology and in campaigns that harness its current potential.
We have a lot of experience working with both big and small companies. We recently partnered with Rovio, the creators of Angry Birds, in the launching of the largest AR campaign to date, and just last month completed a project for a local school free of charge simply to inspire more children to read.
One learning that we've taken from our wide ranging experience is that the audience for this brave new interactive world is as diverse as it is global and we fully believe it to be a wonderful thing. In our view, great technology shouldn't be something that is only available to a few and likewise we want anyone & everyone to be able to enjoy the thrill of experiencing “zaps”.
We’re on a mission to democratize AR and VR, and now MR too thanks to ZapBox! If another two letter acronym ending in R pops up we’ll probably be all over that too!
We’re an established company with a track record of producing high-quality software and content, but we haven’t attempted to make a physical product until now. To be able to offer ZapBox at such a low price-point and print so much cardboard (we <3 cardboard) we need to order the various components in bulk, and with your pledges we’ll have the confidence to make such a bulk order and know the demand for ZapBox is out there.
More importantly, the community of users and developers is going to play a fundamental role in the success of ZapBox as a whole. We think Kickstarter is a fantastic way to build and love that community, in addition to making ZapBox available to the world!
We want to make Mixed Reality accessible, affordable, and enjoyable for all; whether you're an experienced developer, hobbyist creator, or someone looking to experience the cutting edge of what's possible in the comfort of your living room.
We really need your support to make ZapBox happen. If you think this sounds cool we’d love to have you as a backer so together we can get on with making the future.
Risks and challenges
We already have a working prototype of the ZapBox app, so we have full confidence in the ability of our software and pointcode solution to deliver awesome MR experiences on current smartphones.
We held off launching the campaign until we'd reached this milestone so we could guarantee that our solution would work. We also wanted to demonstrate ZapBox truthfully to potential backers by showing real footage captured live from a smartphone.
Creating amazing digital content is something that we're extremely well versed in. Given the advanced stage of our ongoing development any risks associated with the software and digital experiences are now minimal :-)
The physical components have been kept purposefully simple, both to help hit our cost target but also to reduce the likelihood of manufacturing issues further down the line. There are no electronic components in ZapBox which helps to simplify our manufacturing and quality assurance workflows.
We've got some experience customizing an off-the-shelf Google Cardboard design with Mr Cardboard who are world leaders in production of cardboard headsets and are lending that expertise to ZapBox. We're also working with Ship Naked who have a wealth of experience in Kickstarter fulfilment and customs friendly worldwide shipping.
We are extremely happy to be working with such great names in these fields and are fully confident in our combined experience to deliver this project to the highest quality to all backers worldwide.
All this being said, delivering ZapBox to you all is a mammoth task that has many moving parts (in the process - not the cardboard!). We'll keep you in the loop with the production process and if there are any unforeseen delays we will openly and honestly let you know about them and how we're working to resolve them.Learn about accountability on Kickstarter
Great question. The way Kickstarter is geared up allows each backer to select one pledge level and also to set their personal pledge amount.
While we think our current pledge tiers cover the majority of cases, the combination of a bundle of multiple ZapBox kits combined with early developer access is not currently covered.
If that describes what you want then please pledge at the bundle level (which will get you a nice multiple-unit discount), add $10 to your pledge, and send us a message to let us know. We'll then include the additional benefits from the developer bundle tier with your reward.
We do need to limit the number of backers with early access, so we can only offer one set of early access benefits per backer, even if you're pledging for multiple units.
For an optimal experience, we recommend a screen size of at least 4.5 inches with a resolution of at least 720p. The device must also have a gyroscope sensor. Although ZapBox would work on other devices, a smaller screen impacts the field of view that can be displayed in the headset and so offers a sub-optimal ZapBox experience.
ZapBox supports iPhone 6 or later, excluding the iPhone SE. On Android the sheer number of devices means it’s impossible to give an exhaustive list of supported phones, but it is possible to give some guidance. ZapBox requires Android 4.1 or later, but we would recommend Android 5.0 or later as they expose more manual controls for the camera.
If you’re able to download the “Google Cardboard” app from the Play Store and the “Cardboard Demos” content renders smoothly you should be good to go.
Take a look at the comparison chart on the page to see the differences.
The unique part about ZapBox is the additional cardboard components and the underlying software means it offers significantly more guaranteed functionality over the existing Google Cardboard ecosystem which will empower developers to create exciting new content experiences that are not possible with Cardboard alone.
With Cardboard the only guaranteed means of input is head rotation - many headsets don’t have any form of input button. ZapBox on the other hand offers full 3D tracking of both the user and a pair of handheld controllers, along with integrated MR rendering capabilities.
The headset is fully compatible with any other Google Cardboard apps so if you’ve yet to try out existing mobile VR, ZapBox is a great way to do that too.
Many Google Cardboard or similar smartphone VR headsets don’t have openings for the camera. Even those that do allow camera access won’t typically support the wide angle lens adapter on the phone at the same time.
You’re of course welcome to modify any other headsets to accept your phone with the lens adapter and to use that with ZapBox, but in the interests of keeping the pledges simpler we thought it wasn’t worth adding a pledge level without the headset at all. You can always give it away to a friend if you really don’t want it, or let your gran experience a taste of VR :-)
Never fear, the content will be rendered in stereo when in the headset. We thought it was less confusing to show it in non-stereo mode for the video (Magic Leap and HoloLens do the same for their videos).
It’s hard to get across in a video but the combination of stereo rendering and full 3D tracking of the user’s position delivers a really compelling experience.
There are two different ways to combine real and virtual scenes - optical see-through or video see-through.
HoloLens and Magic Leap use an optical see-through setup, where you view the real world directly and only the virtual content comes from the display. This gives a more natural view of the real world but does have some drawbacks; it’s harder to give a large field of view for virtual content, and showing completely solid content without dimming the entire real world view isn’t possible. It’s also challenging to keep the virtual content locked in place relative to the real world as the user moves around; it turns out the speed of light is pretty quick!
With ZapBox we take a video see-through approach; the user’s view of the real world comes from a live camera feed which is rendered in the background, with the virtual content is rendered on top. This offers a less natural view of the real world and is unlikely to be something you would want to wear all the time, but does have some benefits - it permits fully opaque content and a high field of view, and smooth transitions from MR to VR. It’s also easier to completely align the real world and virtual world as the user moves around.
If you think MR sounds a lot like Augmented Reality (AR), you’d be right. We’ve used the term Mixed Reality to describe ZapBox primarily because the experiences it delivers are similar to those shown on HoloLens and Magic Leap, and Mixed Reality is the term they use to describe their content. AR is popular on phones, but the more immersive, larger-scale experiences delivered through headsets feel different enough to warrant a new term.
If that all sounds a bit wishy-washy marketing speak to you, a more technical distinction between MR and AR can be made by considering the contextual relevance of the content in the real world environment. AR is all about context - Pokémon GO characters appear in relevant places in the world, and visual tracking enables things like maintenance instructions overlaid on a specific model of printer. In MR, the real world is used as a stage for virtual content but the context is less relevant. Content appears anchored in the real world to allow natural exploration and interaction, but it doesn’t really “belong” there - mini golf doesn’t really belong in our office, but it’s sure fun to play!
AR gives a great sense of connection between the real and virtual, but you need to go in search of the objects or places with associated content. With the MR approach of ZapBox the content can come to you! This makes browsing, sharing and experiencing content much easier for users. For hobbyist developers the MR paradigm allows content to be shared with the audience of other ZapBox users without them needing to print out or find specific “target images” for your content.
We’re convinced there’s a big future for both AR and MR, and ZapWorks is an ideal platform for creating content for either paradigm.
ZapBox really does tick all of those boxes in the comparison chart for $30.
Achieving such an affordable price-point meant avoiding any additional sensors or electronic components and limiting ourselves to the cameras and sensors available on existing smartphones.
While you shouldn’t be expecting ZapBox to offer an experience exactly on par with HoloLens, we’ve been able to make it 100 times cheaper! Our aim was not to make the perfect MR device, but to push the boundaries of what was possible with our existing smartphones and offer a genuinely affordable entry-point to this entirely new class of experiences.
The exact amount of latency from the camera is device-dependent. If you spin around quickly you will probably notice a delay between your motion and the rendered view.
It’s important to bear in mind that ZapBox offers a new class of experiences. With the 360-degree video content popular with Google Cardboard headsets the user is almost encouraged to spin around quickly in case they’re missing some cool content going on behind them, and that’s why there’s been a push to minimize motion-to-photon latency in the mobile VR.
ZapBox content is fully anchored in 3D and we find this encourages much slower user motion. The content tends to be more “outside in” where you walk around things of interest rather than “inside out” where there are things happening all around you. These “outside-in” experiences are perfectly suited to ZapBox and are much less latency-sensitive than 360-degree videos.
Usually headsets offering video-see through AR or MR would feature a camera for each eye so the live feed can also be shown in stereo.
As ZapBox understands the world geometry from the map-building process we are able to use that knowledge to produce two different renders of the camera image, so that content does appear correctly anchored to the world in the stereo view.
Initially ZapWorks Studio will be the only supported tool for publishing content for ZapBox. It supports importing assets made in other tools such as videos or animated 3D content (from tools like Blender, 3DS Max or Maya).
ZapWorks and the Zappar platform is perfectly suited to creating and publishing short-form experiences that don’t require app submissions for each piece of content. This provides a significant benefit both to users wanting to discover content and developers wanting to share what they’ve built. We believe that this integrated approach will allow a ZapBox ecosystem to develop and thrive.
Although theoretically possible to develop plugins to expose the underlying technology to other tools (eg Unity or Unreal) the resulting experiences would need to be distributed as standalone apps. This would offer significantly more friction to the user experience, and hence is not currently a priority.
Content shared through the ZapBox app will be available for free. The ZapBox app is not an “app store” - we wouldn’t be allowed to distribute it if it was.
It’s possible that in future you will be able to embed the ZapBox technology into standalone apps which you could submit to the app stores as a paid app if you want.
We expect the early days of ZapBox to be about smaller, bite-sized experiences as we collectively experiment with what can be done with the platform. The ability to easily share these experiences directly into the ZapBox app without needing to make separate app submissions should really encourage this experimentation and we’re really excited to see what you come up with.
ZapWorks has a seat-based licensing model for commercial use. You’ll need a “Pro Seat” for access to ZapWorks Studio to build ZapBox content. See https://zap.works for the details.
Generally, content will be available free to users, but if you are creating custom ZapBox MR experiences for events or giveaways, you will require a commercial subscription to ZapBox.
This might get technical...
SLAM stands for Simultaneous Localisation and Mapping. SLAM algorithms can determine the user's position with reference to a map of "natural features" - things that are in the real world already - that it builds up as the user moves around. HoloLens uses SLAM to allow tracking without world markers.
In fact the "map building" part of ZapBox employs very similar math[s] to work out how the pointcodes are laid out. So why don't we use SLAM in ZapBox?
When we first started thinking about ZapBox we had two core requirements - we wanted to support Mixed Reality experiences, so needed to show the camera feed, but we also wanted natural independently tracked interactions using handheld controllers. That combination means the controllers themselves have to be trackable when small in the image (otherwise they'd cover the view of the rest of the world and prevent true "MR" experiences).
We developed pointcodes as a new marker design that can be very efficiently detected on mobile hardware even when small in the image. A constellation of these markers on each controller allow us to accurately detect them from any angle and when only some of them are visible. The same basic concept is also used for our pointcode world tracking. Using the same approach for both is great for performance - a single pointcode detection step across the entire image is sufficient to track both the world and the controllers.
Monocular (single-camera) RGB SLAM also has certain limitations; scale is ambiguous, it is difficult to cope with other objects moving in front of the camera (such as people's arms / controllers when interacting with content), and it requires sufficient image texture in the world.
The requirement for sufficient texture is especially challenging in smaller indoor home environments combined with low-FOV cameras on existing smartphones.
HoloLens is built from the ground up around SLAM-based world tracking. That's why it has 3 wide-angle RGB cameras pointing in different directions and a depth camera (which presumably both helps tracking when close to a surface and allows an occlusion mask to be passed in to the SLAM algorithm to prevent arm motion disrupting SLAM). They also have some custom DSPs to process all that data.
In summary on mobile hardware we believe using pointcodes for world tracking is the right tradeoff between robustness, performance and ease-of-setup.
See the following question for where other natural image features may fit in...
As the videos are recorded live and to give a true impression of the performance that ZapBox offers, they represent the current state of our development.
Using the wide-angle lens adapter increases the field of view of the camera and hence reduces the required density of codes to robustly track the same area. There are also continuing improvements to both the tracking quality of individual codes, along with sensor fusion with device accelerometer and gyroscope, which we expect to provide solid tracking with fewer codes in view.
Another possibility that we will investigate as ZapBox gets closer to launch is employing other natural features in the image to constrain the user position, either through visual odometry or full SLAM. See the question above for why we didn't start with SLAM and why we don't consider that a suitable primary solution for robust world tracking in the ZapBox context.
Finally we are also planning to investigate blurring or in-filling the codes in the camera texture, in which case you won't even see them in the experience itself (if they're on a solid background...).
We think they look cool though :)
We asked ourselves the same thing, as we’ve always known him as Jeff!
Jeff heads up our US operation and is the named person on the campaign so we meet Kickstarter’s requirements for US projects. This allows us to have all our pledges listed in US Dollars for the convenience of international backers.
Support this project
- (30 days)