Share this project

Done

Share this project

Done
ShoeHaiku: Game/VR controllers that fit in your shoes!'s video poster
Play

ShoeHaiku sensors slip into a pair of shoes and detect walking, running & jumping - this generates mouse moves and key-strokes via USB. Read more

Austin, TX Hardware
Share this project
4
backers
$153
pledged of $10,000 goal

Funding Canceled

Funding for this project was canceled by the project creator on December 4, 2013.

ShoeHaiku sensors slip into a pair of shoes and detect walking, running & jumping - this generates mouse moves and key-strokes via USB.

Austin, TX Hardware
Share this project

About this project

Haiku: A short but evocative poem.

ShoeHaiku: Poetry in motion.

The ShoeHaiku sensors slip into a regular pair of shoes and plug into the USB port of your computer as a "HID" keyboard+mouse.  The sensors detect the pressure of your feet within the shoe as you walk in place, lean, stomp or jump.  The controller box analyzes that motion and turns it into keystrokes and mouse commands to move you around in the virtual world. It works with almost any game and computer that supports keyboard and/or mouse control.

Our final prototype.
Our final prototype.

ShoeHaiku is fun to use no matter how you play games.  But in many virtual reality settings it solves a serious problem.  In those situations, your eyes are typically covered by some kind of headset, such as the Oculus Rift (or the recently under development: Valve Steam and the  Infiniteye headsets) so you can't see your hands - which makes using a keyboard or a mouse almost impossible. There are plenty of hand-controllers for aiming and shooting - but the only options for movement are unnatural joysticks, nunchuck-controllers or large and costly treadmill contraptions.

ShoeHaiku costs under $100, it fits inside your own shoes and has a tiny control box that clips onto your belt.  It needs no special drivers and works with almost every game and software application on any computer that allows you to use a USB "HID" keyboard or mouse.   We can't think why you'd want to play Tetris using just your feet...but you could.

The prototype electronics...fortunately, I *won't* be the person soldering the reward sets!
The prototype electronics...fortunately, I *won't* be the person soldering the reward sets!

As far as the computer is concerned, ShoeHaiku is just a normal keyboard and mouse.  (And if we get enough backers to cover the additional development work, we'll make it work like a joystick too!)

A Google-Chrome "app" is used to tell the ShoeHaiku controller which mouse/keyboard commands you want it to send for each combination of foot motions (we call them "gestures").  Those commands are stored inside the controller box and stay in effect even if you unplug it and plug it into a different computer.

For example, you might tell ShoeHaiku to hold down: 'W' for walking, "Shift-W" for running, "E" for sprinting, Spacebar for jumping, "A" and "D" for leaning left and right and "S" for leaning backwards.  That would be a good setup for most "WASD" games.  You can also tell it to hit a key only briefly, do "key repeat" (hitting the key repeatedly while you're doing some gesture), click any of the mouse buttons and even move the mouse and operate the scroll-wheel!

There are special modes for games where you may want to hold down multiple keys (eg holding down two arrow keys to move diagonally) - and for platformers where you need a different keystroke for running to the left and right and to use a "stomp" gesture to switch between them.

I'm sure we'll come up with more control modes as our conversation here develops and we get feedback and suggestions from early adopters.  This ability for us to interact with our backers is the real power of Kickstarter!

Because your computer accepts commands from both the real keyboard/mouse and the ShoeHaiku, you can have your feet control walking, running and jumping and let your regular keyboard and mouse (joystick, nunchuck, head-tracker, whatever) control aiming, shooting, switching weapons and so forth.  If you have a head-tracked display, then that takes over the direction control, and your feet do the running and jumping so that your hands are free for shooting and so forth.   For this specific application, we have a stretch goal to include two simple push-button hand-controllers to the ShoeHaiku so that you can use it with a head-tracked display without the need for any other input devices.

How It Works:

The ShoeHaiku controller is programmable from a web-based app to allow it to generate just about any keystrokes or mouse motion/clicks from a variety of patterns of footsteps, stomps and body-weight shifting - so it's adaptable with almost any game or other software and many different control styles.  There is specific built-in support for the common control pattern of WASD games, side-scrolling platformers, driving and flying games - but you can easily customize the controls to work with almost everything...you can play Tetris (on a cellphone!) using just your feet if that's the kind of thing you want!

The Controller box.
The Controller box.

Because the computer thinks that ShoeHaiku is just an add-on keyboard and mouse, it works with any flavor of Windows, Linux or MacOS - and any other software/hardware that supports a standard "HID" external keyboard and mouse.  You can even use a USB-OTG cable and connect ShoeHaiku to a variety of phones and tablets - although, unfortunately, not many mobile games support keyboard input.

The ShoeHaiku sensors fit into your regular shoes using cushioned sole inserts that you cut to size.  If the sole insert ever wears out, or if you want to change shoe sizes, just buy a pair of insoles for a couple of bucks at your local pharmacy and you're good to go.

The Shoe Insert
The Shoe Insert

Configuring the way you use your ShoeHaiku sensors and adjusting the calibration for your body weight is done with a web-base app in conjunction with the shoes themselves in a special setup mode that's enabled by a switch on the controller box.  You can even adjust the shoes and switch what they do while your game is running.

SMASH - the Chrome App that configures your ShoeHaiki
SMASH - the Chrome App that configures your ShoeHaiki

The "SMASH" application is only need to configure your ShoeHaiku for a particular game setup - and once configured, you can plug the controller into a different computer to actually play the game.  So even if your game computer doesn't run Chrome, you can still use ShoeHaiku.

SMASH can also be left running while you're actually playing a game, and use it to dynamically switch between settings without leaving the game.  This might be useful if your game entails swimming, skiing, parachuting or some other activity as well as walking.

The software in the ShoeHaiku can be upgraded in the future using an additional Google-Chrome App - so early adopters won't be left out if new software goodies are added later on.

One problem with our HID-keyboard/mouse approach is that the ShoeHaiku sends these commands continuously as you move your feet - which is a bad thing when you're not actively playing the game (eg if you want to enter your name into the high-score table and you don't want it covered in W's, A's, S's and D's as you do your usual victory dance!).   For that reason, there is a switch on the controller box that shuts off the HID functionality for as long as you need.

The controller and shoe sensors are powered through the USB port - so no batteries or external power supplied are needed.

What You Get:

If you pledge support for our project at the $94 level or higher, you'll get a set of thin-film sensors that stick inside your shoes with adhesive pads, and foam insoles to cover them up and provide a comfortable layer for your feet.  A 3.5mm jack pops out at the back of each shoe and plugs into a standard headphone-type cable that runs up to a tiny controller box at your waist.  A USB cable goes from there to your computer.  If we can end up with pledges for at least 500 units, we'll add in a two meter coiled USB cable for extra freedom - and at yet higher levels, we'll develop a BlueTooth interfaced version that will not require a cable to the host computer at all.

The Basic Pledge box contents.
The Basic Pledge box contents.

When all deliveries are completed, we will release the source code for the controller software as OpenSource code under the GPL.  At the "developer" levels, we'll include beta copies of the source code under a more restrictive license as soon as you get your reward.

The controller can also be placed into a mode where it sends raw pressure data from the shoe sensors - which enables you to develop your own software that goes beyond the features we provide in the controller box.  In this mode, it sends a simple ASCII string many times each second with the pressure readings from the toe and heel sensors in each shoe expressed in hexadecimal - in this mode it does not act as a HID device but as a standard USB serial port, so these data blocks do not interfere with normal keyboard input.

BONUSES & STRETCH GOALS

If the project does well, we plan several additional goals and some free bonuses for people who already pledged at the basic $94 level:

  • $50,000 - BONUS: We'll add a free, extra-long, coiled, 2 meter USB cable to give you more room to move around.
  • $100,000 - STRETCH GOAL: We'll make two hand-held push-button controllers which will plug into an expanded controller box to provide the ability to do things like switching weapons and firing (Reward price: TBA).
  • $150,000 - BONUS: We'll develop a joystick emulation mode so that you can control any software/device that supports a standard USB HID joystick and include this for free at all ShoeHaiku pledge levels.
  • $200,000 - STRETCH GOAL: We'll design a version of the controller that is battery operated and supports BlueTooth communications instead of USB so you can be untethered from a BlueTooth-equipped PC.  This version should also work with phones and tablets - any device that supports a standard BlueTooth joystick, keyboard and/or mouse. (Reward price: TBA)
  • $300,000 - BONUS: We'll include the two hand controllers *FREE* at every reward level that includes the ShoeHaiku.  People who already pledged for the $100,000 stretch goal will get an improved version (TBA).
  •  $400,000 - STRETCH GOAL: We'll develop a pair of tiny ZigBee radios - one for each shoe - that will completely eliminate all of the wiring (Reward price: TBA).

Additional stretch goals and bonuses will be announced should we surpass $500,000.

NOTE: Please do not attempt to 'game the system' by pledging large amounts in order to push the project over a goal - with the plan of reducing your pledge after the bonus is unlocked.  This is very uncool and it won't work because the bonuses announced above only apply if the FINAL pledge total exceeds the amounts indicated.

Project Timeline.

The original idea for this project came about somewhere around 10 years ago when I was working with virtual reality headsets as a part of a helicopter simulation project for the US Army.  These headsets were used in full cockpit simulators.   It became clear to me that if virtual reality were to become widespread, you'd want to be using your feet as controllers.

Sometime later, while working as a videogame developer with Midway games, I was messing around in my spare time interfacing a Wii Balance board with a home project.  It's annoying that Nintendo don't allow their developers to encourage people to jump or do other vigorous activities with the board because of the risk of falling off of it.  It occurred to me that placing the pressure sensors INSIDE your shoes instead of in some external contraption would fix that.

But at the time, small, cheap computers such as the Arduino were not very capable - and I'd need significant horsepower to do "gesture" recognition inside a portable controller box.   I knew I could do that processing on the main CPU - but that would require special drivers and the resulting system would not be portable between computers or operating system variants.

The idea has been floating around ever since.  When I recently lost my job - I had time to push the idea further.

Hand-soldered Mk II circuit board and block diagram
Hand-soldered Mk II circuit board and block diagram

Over a couple of weeks in mid-October, I researched available pressure sensors - and came up with a thin-film sensor in a slimline package that would slide comfortably into a shoe.  Measurements with an oscilloscope suggested that this approach was really quite workable.   For the next few weeks I went through several design revisions for the controller box - using progressively smaller electronics packages from an Arduino Leonardo to a "Pro-Micro" controller that's about the size of your thumb.

Here is a photo of our first controller box - in a rather ugly laser-cut case:

Mk I controller.
Mk I controller.

 The Mk II controller uses a smaller computer module and fits into a small plastic enclosure:

MK II Controller.
MK II Controller.

 Aside from my time spent in identifying vendors for all of the parts we'll need - the largest part of the time has been in writing the software which comes in two parts:

* A C++ program that runs on the controller-box computer to convert pressure readings from the shoe sensors into "gestures" and then to map the gestures onto keystrokes or mouse movement commands.

* A Javascript "app" that runs as a "Chrome" browser application to allow the end user to configure the ShoeHaiku controller to produce appropriate keystrokes for different games and to download those settings to the controller box.  

It's taken about 200 hours of work to get these working well together and to test the system with a wide variety of games and game genres.  The biggest problems lay in the areas of gesture recognition and in the interfacing to USB via the JavaScript app.

A moment in the software development process.
A moment in the software development process.

So now the project is at the point where my hand-soldered messes need to be turned into proper printed circuit boards and parts ordered in mass quantities in order to get the manufacturing costs down.  Most of the bulk discounts and circuit board manufacturing minimums suggest a minimum production run of 100 units - and at $94 each, that comes to a $10,000 project goal.

Risks and challenges

Although Renaissance Innovation is a brand new business venture, Steve & Renee Baker also own and operate a sister company: Renaissance Miniatures (http://renaissanceminiatures.com) which has had two highly successful Kickstarters with hundreds of satisfied backers and ZERO unhappy ones! Go ahead - check out our history! So we're quite familiar with the joys, tripfalls and gotcha's of Kickstarter-funded businesses. We're also enthusiastic Kickstarter backers: Our laser cutters are "lasersaurs", our initial prototypes used the "$9 Arduino" computer reward and so forth.

We have our own in-house rapid prototyping capability (laser cutters, 3D printers, small-scale circuit board production, etc) and we know the ropes when it comes to frequent backer updates and the importance of quick, clear and open communications. Renee is a great "people person" and insists on keeping every single backer happy, no matter what.

Steve has programmed computers since they were large enough to fill a room and has even designed integrated circuits for a living - and he's spent years working with head-mounted displays for military applications. So we have the tech side of things nailed down too.

The shoe sensors and controller box are already working, we've made many prototypes - they are robust, reliable and easy to assemble. The software works with every PC-based game we've tested against - so we have no technological risks for the base system.

We have suppliers lined up for all of the parts so we have no unknown parts pricing or supply risks.

We have yet to manufacture the printed circuit board for the controller - but to minimize risk, we use an off-the shelf microcontroller so all that the circuit board does is to add some protection resistors, a switch and audio jack sockets. If the quantities are very large, we'll integrate the microcontroller onto the main circuit board - but in that case we'll have sufficient funds to employ a professional circuit design service to do that.

The assembly process is the most difficult part to plan for because we don't yet know the quantities involved. For very small numbers, up to a couple of hundred units, we'll hand-build the controllers ourselves. For larger quantities - up to perhaps a couple of thousand units, we'll find local workers to help us to assemble them and we'll do our own QA. If the project goes crazy-viral, we'll need to make deals with outsourcing agencies to manufacture them in those larger numbers.

We're sure that the price point we've set will support all of those options - but you'll note that we've built in the possibility of a little time-slip in the event that we have to find additional assembly help.

Early-bird and developer pledges will be hand-built no matter what to ensure you'll get your ShoeHaiku on-time.

Some of the stretch goals entail significant additional software work - and this will take time - but at those funding levels, we can take on additional engineering help if we need it. We will provide software updates for Early-Bird backers should that prove necessary.

Learn about accountability on Kickstarter

FAQ

Have a question? If the info above doesn't help, you can ask the project creator directly.

Ask a question

Rewards

  • You selected
    $1 reward

    3 backers

    Our gratitude for your interest.

    Follow what we're doing, get in on the conversation and see backer-only updates. Maybe we can tempt you to increase your pledge later on: Kickstarter lets you increase your pledge at any time up to the project end date.

    Estimated delivery:
  • You selected
    $5 reward

    0 backers Limited (200 left of 200)

    A small token of our gratitude. A limited edition laser-etched outline of a shoe, made from wood with a hole in it so you can hang it from your shoelace or keychain for inspiration while jogging.

    Estimated delivery:
    Ships anywhere in the world
  • You selected
    $93 reward

    0 backers Limited (100 left of 100)

    EARLY BIRD: One Pair of ShoeHaiku sensors plus controller box, and cables. We'll also send you one of our limited-edition laser cut shoes as thanks for being amongst the first to take the plunge.

    Estimated delivery:
    Ships anywhere in the world
  • You selected
    $93 reward

    0 backers

    REGULAR SHIPMENT: One Pair of ShoeHaiku sensors plus controller box, and cables. Delivery date is estimated to be in June but may extend later into the year if we are overwhelmed with backers.

    Estimated delivery:
    Ships anywhere in the world
  • You selected
    $150 reward

    1 backer Limited (19 left of 20)

    LONE DEVELOPER ACCESS: One hand-built Pair of ShoeHaiku sensors plus controller box and cables. Software updates will be provided for free if needed after delivery. Controller may be in a prototyped laser-cut case.

    Estimated delivery:
    Ships anywhere in the world
  • You selected
    $1,000 reward

    0 backers Limited (5 left of 5)

    SERIOUS DEVELOPER PACK: This level of pledge gives you access to the circuit schematics and source code for the project and three months of email access to our developers for project support. We'll send you a hand-built pair of ShoeHaiku sensors plus controller box, cables, etc. Controller boxes may be in a prototype laser-cut case. We will upgrade you to the production set at no cost as soon as it becomes available and if you need a modest number of additional hardware sets for your developers, we will work with you to provide what you need.

    Estimated delivery:
    Ships anywhere in the world
  • You selected
    $1,500 reward

    0 backers Limited (5 left of 5)

    Come to the Austin, Texas area (at your own expense) and we'll spend the day with you discussing the product and showing you how it's made. You'll also get everything from the SERIOUS DEVELOPER pledge level and we'll make you an utterly unique, one-of-a-kind laser-etched shoe tag with whatever Haiku you want.

    Estimated delivery:

Funding period

- (27 days)