Share this project

Done

Share this project

Done
A glove-based input device that provides full mouse/keyboard control built for wearable/mobile computing and handicapped users.
A glove-based input device that provides full mouse/keyboard control built for wearable/mobile computing and handicapped users.
95 backers pledged $12,474 to help bring this project to life.

New Keyglove firmware, KGAPI protocol, and working Bluetooth

2 likes

How do I update thee? Let me count the ways...

Since my last post, I have created a lot of documentation, written a lot of new code, done just a bit of yak-shaving, and published all of the new code and docs to the Keyglove Github repository and a new Arduino firmware documentation page on the project website. These accomplishments involved some ancillary activities, and there are also some other points to mention, so as usual I will try to address each main topic in turn.

New Keyglove firmware Arduino code

The most significant news is the updated and largely rewritten Arduino firmware source code that powers the microcontroller at the core of the Keyglove design. Since this is an open-source project, it has always been my intention to publish the complete source so that anyone can understand or modify the functionality however they might wish to. However, I am a bit of a neat freak when it comes to code. Well, "a bit" may be an understatement. It's hard for me to work with or around code that isn't organized, and/or formatted, and/or documented well, and it's even harder for me to publish it knowing that others may need or want to use it in their own designs. Before I feel good about handing something to the world, it must pass my own tests for organization, readability, and documentation.

In light of this, a significant portion of my work on the Keyglove project over the last few months has been making all of my code consistent, splitting it up as logically as possible into modular files, and finally documenting the heck out of it. I am proud to say that these efforts have paid off, and I've published the not insignificant results to the Github repo, and all of the documentation to go with it.

Keyglove Arduino documentation
Keyglove Arduino documentation
Keyglove Arduino file list
Keyglove Arduino file list

It's great to write so-called "self-documenting" code, and I have tried to do that. I don't want people to need to refer to documentation any more than is absolutely necessary, and well-written, well-commented code can go a long way. But at the same time, there is no substitute for good documentation. The code that makes perfect sense to me because I wrote it might make no sense at all to someone else. The documentation linked above is my initial attempt to make sure that doesn't happen. It will undoubtedly be improved over time, but for now, it's a good start.

So what exactly does the current published Keyglove firmware support?

  • Teensy++ 2.0 microcontroller
  • USB (wired) HID and serial connectivity
  • Bluetooth (wireless) HID and serial connectivity
  • Up to 37 touch sensors
  • MPU-6050 6-axis motion sensor
  • Blinking LED feedback
  • RGB LED feedback
  • Vibration motor feedback
  • Piezo buzzer feedback
It is up to the user whether and how each component is used in the whole system. The firmware has been written as a generic platform that can do all kinds of things (including things I haven't imagined), but doesn't necessarily need to. The totally default non-customized behavior is that it boots into an "idle" state after running through the initial setup process, and then simply waits for instructions from an external host. But with only a few lines of code in one single file (application.h for the coders reading this), you can include any autonomous behavior you want, even going so far as to make host control totally unnecessary by putting all relevant application logic right on the Keyglove itself.

Think of it as a keyboard + mouse + joystick + motion sensor combo device which you can load your own custom code into, and/or fully control from another device. It's extremely customizable. This brings us to...

Keyglove API protocol (KGAPI)

As I alluded to in the last update, I've now implemented a straightforward protocol that you can use to control the Keyglove. This is made up of the following basic types of communication:

  • Commands: sent to the Keyglove to make stuff happen
  • Responses: sent back from the Keyglove after commands are sent, acknowledging or reporting errors
  • Events: sent from the Keyglove when things happen that aren't directly triggered by commands, such as touch sensor changes or new motion data
This is the same kind of structure that you will find in many protocols used by hardware peripherals, so there's nothing really special here.

However, one important point to mention is that while you can use the KGAPI protocol from an external device like the PC to which you've connected the Keyglove, you can also use exactly the same "vocabulary" of commands, responses, and events right in your custom application code in the firmware. This gives you a lot of power and flexibility that you won't find in other similar devices; rather than just making external control possible, the Keyglove implementation makes it as easy and intuitive as possible to tweak the firmware itself (remember application.h?), even going so far as to use the same command names, parameters, etc.

What does this mean practically? Let's say that you write a Python script which talks to the Keyglove using KGAPI, and handles the "touch_status" event (which happens whenever a touch sensor changes state), and inside that event handler you use the "motion_set_mode" command to turn on and of the motion sensor based on the touch status. Rather than making the PC do that work, you could use the exact same set of commands and events to make that behavior part of the firmware itself.

You could further offload basically anything from the PC onto the Keyglove itself--touch interpretation, motion processing, Bluetooth connection automation, feedback, you name it--so the only thing the connected host sees is the actual control signals over the human input device (HID) interface. This is in fact how I assume many people will want to use the Keyglove, but this alone is too limiting. Not everyone has the same implementation vision as I do, and I want to empower as many different use cases as I possibly can with this platform. My last visit to Maker Faire powerfully communicated the importance of that goal to me, and now, it has become a reality.

Another feature worth mentioning is the ability to extend the KGAPI protocol with your own commands, responses, and events if desired. This is more than most people will want to mess with, probably, but for those who do, I've made every effort to make it easy. You can find reference code and comments in the custom_protocol.h file.

I am still working on a KGAPI reference manual, but it isn't quite ready yet. It's coming along, but it's still too bare. In the meantime, the previously linked documentation includes descriptions for all of the commands (functions beginning with "kg_cmd_..."), and there is a handy reference file for all of the possible event handlers you might need (the application_stubs.h file).

I also have a Python implementation of the whole API including a parser and a few test scripts, and I am working on a simple cross-platform graphical testing application, also written in Python and using the wxPython GUI toolkit:

Keyglove GUI Test Software
Keyglove GUI Test Software

This will be published as soon as I consider it to be stable. There are only a few fixes and additions remaining before it will reach that point. However, the core KGLib parser library and a console-based test script are already available on the Github repo.

Working Bluetooth system

So what's all this about Bluetooth? As some of you may know, the Bluetooth wireless system has been one of those lingering features that I couldn't quite manage to nail down for one reason or another. Well, I nailed it down. I've got a robust connection and data management system in place, built around the Bluegiga WT12 Bluetooth module and its powerful iWRAP firmware.

The solution to this longstanding problem involved much of the "yak shaving" that I mentioned earlier. (For a brilliant and hilarious explanation of yak-shaving, see this animating GIF.) In short, I totally rewrote the iWRAP parser library that I began years ago in order to make it function properly and fit nicely with the Keyglove API structure. The end result of that effort is a cleaner and much more intuitive interface that I and anyone else can use to talk to Bluegiga's classic Bluetooth modules, like the WT12 that I have in the Keyglove. While it technically stands on its own and isn't part of the Keyglove firmware, it dovetails perfectly between both the Keyglove and my day job at Bluegiga, and it has already been useful for other people in other applications.

In short, the new iWRAP code cleaned up one of the messiest and most complex parts of the Keyglove firmware source, and is now both much easier to understand and much easier to maintain (not to mention the more important fact that it actually works correctly).

With the right configuration, the Keyglove can now automatically reconnect to one or more previously paired devices, even going so far as to support simultaneous connections. This means, for instance, that you could connect to your PC and your smartphone at the same time, and have one set of touches trigger behavior on the PC while a different set controls the media functionality on your phone. Yep, it's that flexible.

Keyglove kit development

All of the above means that the main Arduino firmware is in a state where quite literally everything officially supported (so far) is totally working, at least as far as I have been able to test. There will undoubtedly be more coding required, but not right now. It's reached a stable milestone that I am satisfied with. Therefore, I feel free to focus on solving other problems that have been on the back burner for some time--most importantly, creating an easily reproducible kit which can be assembled with (ideally) not much effort using as many off-the-shelf parts as possible.

I have been in contact recently with some of the people who have worked on similar glove projects that heavily involve e-textiles, and I'm hopeful about the outcome of those efforts. Previous discussions with other individuals and companies haven't panned out yet, but I've actually met these people in person before, so as long as they have the time and I can make it worth their while, I will remain optimistic.

I am simultaneously taking a new approach to solving the very difficult problem of mounting and reliably wiring so many sensors on the glove. Honestly, 37 sensors is a lot. Most people won't want or need to use that many. Having 37 also tremendously complicates the sensor wiring, to say nothing of the tedious mounting process. So, in the interest of moving the football down the field, as they say, I'm creating a simplified variant of the design that cuts the sensor count down to a mere five: one for each fingertip. Since multitouch and touch order permutations are still easily possible, there are still many possibilities for customized behavior. Full alphabet input is probably not included in this unless you are extremely clever and/or patient, but that's okay for now.

Much like what I just finished with the firmware source, I'm trying to focus on the next milestone that I know I can achieve even if I don't have any help. Of course, I'm still pursuing the more ambitious goal for which I'll need help from others, but since I don't have any guarantees for how that will go and I still have rewards to deliver, I've dialed back the requirements for reaching the next step so that it is actually within my grasp. Once I make that goal, I'll set a new one and pursue it while starting from my new vantage point, so that one way or another I can achieve the final goal.

On the horizon

What can you expect in the next couple of months? Here's my high-level to-do list:

  • Create 5-sensor version of Keyglove kit with glove, battery, feedback, and wireless support
  • Continue discussion of complex e-textile implementation with contacts
  • Add content to KGAPI Reference Guide and publish
  • Finish initial release of Keyglove GUI testing software

Can I manage it by the end of September? I will do my best. As always, for more frequent but brief updates on the project (there have been 50 since the last Kickstarter post), be sure to keep an eye on the Keyglove Twitter feed and/or Facebook page.

Thanks for your continuing support!

BorisKourt and Adriaan Wormgoor like this update.

Comments

Only backers can post comments. Log In
    1. Jeff Rowberg Creator on

      @Adriaan,

      That's definitely the goal, yes. The simpler kit will most likely be available before anything else, but I won't stop until the finished product is ready. Thanks for your comments about the code and documentation!

    2. Missing avatar

      Adriaan Wormgoor on

      Great post Jeff and very well-documented codebase! Much looking forward to being able to order a keyglove. By the looks of it that's where things are headed at the moment?