Flowton Technologies: Control your home with gestures+voice
Flowton Technologies: Control your home with gestures+voice
First true Natural Interface-(NI)-Controller for your home! Control any TV, lights, thermostat, etc. with only body gestures and voice.
First true Natural Interface-(NI)-Controller for your home! Control any TV, lights, thermostat, etc. with only body gestures and voice. Read more
About this project
If you only have 2 minutes, please watch our quick feature video below
Do you like what you see here ? Please share this page with your facebook and twitter friends, and tell the world about the future of User Interfaces!
Even if you are unable to back us, you would really be helping out by spreading the word!
Your hand is now the remote
How many of you have at least one remote at home? We bet most of you have 3 or more. Have you ever lost one of them, buried deep in the recesses of your sofa? Or maybe you brought one to the kitchen when you went on that break and forgot it, right there on the kitchen counter. Isn't it such a pain to have to go retrieve it AFTER you've settled back on the couch to watch your show?
What about always having to get up just to turn off a light switch, adjust your thermostat or dim the lamp lights? Wouldn't it be cool to not be faced with those issues any more? How awesome would it be to be able to control all those things and more with just your body? Yes, just your body!
Well, we are pleased to present to you our solution – The Flowton Controller.
It's here! The Flowton Controller allows you to use your body as a universal controller for all your electronics. The software inside works by capturing image and sound from it's environment, and converts this raw data into commands that you may use to control anything from your TV, DVD, to thermostat, lights and more.
Use your hands, your body or your voice to execute commands, replacing switches, buttons, and other artificial devices.
What is a Natural Interface (NI) you ask? Well it pretty much means allowing your natural body to execute commands. There is nothing to touch. You simply push a "virtual button" in empty space in front of you, and the 9-point controller springs up around your hand.
From here you can control your television volume, switch channels, or select other connected devices, such as window blinds, and many others.
Just as well, you may call out a voice command - like "Kitchen lights ON", and the computer responds, swiftly performing the action you want it to - illuminating your way before you ever left your comfortable seat.
Think of it as your digital butler. You can either tell it, or show it, and it does.
Amazing right? We certainly think so!
Well, now that you've decided that you want to back us, we can guess exactly what you are thinking...
What reward option should I choose?
In short, if you just want the Controller in your home to use with TV, DVD, wireless thermostat, wireless blinds, etc., then you should go for the 219$ early standard version (limited #).
If however you have many electronics such as table lights, lamps, decorations, alarm clocks, or other objects that you wish to control with Natural Interaction, then go for either the 249$ early extended bundle (limited, comes with 1 smartplug) or the 299$ complete bundle (comes with 2 smartplugs).
The reward for developers is pretty obvious so if you're a developer, get one of those today.
And if you just want to help out and see this product made, you can still back us in the lower rewards. We count everything and acknowledge all contributions!
Exclusive to Kickstarter!
You will not get this anywhere else. It simply doesn't yet exist.
There is nothing on the market that will allow you true Natural Interaction control over every day objects. We are not just talking PC and iPhone here. It's the things you interact with most daily - TV, lights, etc.
Kickstarter has become a one-of-a-kind portal for innovation and groundbreaking products, and we are happy to be a part of it.
If you pre-order now, you will be one of the first people to get their hands on the finished product in all it's glory! You will be the first to bring NI into your home, and puzzle all those around you with random gestures in the air... that somehow now have an effect on the surrounding devices. This is as close to magic you will get for another few years. Believe me, we checked.
And the best part is - we are setting up an exclusive portal just for our backers, so you can get in on the development and evolution of this incredible technology! You will get unprecedented look behind the scenes, and witness this technology become the future.
It is our goal to work closely with our backers and bring the most utility out of this capable device, so we will keep our ears firmly to the ground. However, there is only so much advice and feature requests we can take and implement. Thus, being selective with who we bring in to throw around ideas with, we must be sure that it's people who are passionate about NI and dedicated to making this product all it can be.
So we invite you to come and invent with us!
Is the future here yet?
Flowton unit is a device built for advanced Human Computer Interaction (HCI). Certain forward technologies have lagged behind in the research and development stages. They remain the stuff of science fiction, and our imagination.
We were all promised something extraordinary when the likes of Microsoft® Kinect™ came onto the market. It was now more affordable than ever to own devices that can understand our body. To see it. To hear it. Kinect™ being the prime example consequently became the "fastest selling consumer electronic device" in the world! Tens of millions of people bought into the vision of the next stage of computers. Well, we are happy to inform you that your patience paid off. And here is the result...
The next user interface revolution...
The internet of things. Ubiquitous computing. Things that think. You have heard it all before. And the time has come.
We are surrounded every minute of the day, by computers, everywhere. Ones that only a decade or so ago would have been called supercomputers. It’s time we start putting these concepts to work!
Flowton Controller (command)
So how does it actually work, you ask?
The Flowton Controller receives commands through a built-in color RGB (Red/Green/Blue) camera and 2 microphones. Both need an initiator command for the unit to "start paying attention".
Once the Controller starts capturing image (after performing a gestural initiator command) or sound (after speaking a voice keyword), you may issue follow-on commands addressing a particular device. Such as in example where you initiate the TV controller with your wrist, and do a rightward swipe to change the channel. Or when you say "Bedroom Thermostat Up 5 degrees", you first address the device, and then issue a follow up command.
The Flowton Controller also is able to work with existing depth sensors such as the Microsoft® Kinect™, and others (listed below). By utilizing a different method to capture images, proper depth cameras such as remove any limitations that are dim lighting lighting, color noise, etc. This is why we strongly advocate it's purchase, however it is not mandatory.
Some of the current choices available for purchase:
- Microsoft Kinect – 109$
- ASUS Xtion – 119$
- Primesense Carmine – 199$
- SoftKinetic DepthSense311 – 299$
The speech recognition is based on an offshoot of Julius (Japanese open-source large vocabulary CSR engine) and is capable of recognizing key words necessary for TV, lights and other devices. As with other examples, you "wake" the system up with a trigger command, and follow with an executable command.
The voice input is received via Flowton's 2-microphone array, or in the case of Kinect, utilizing that 4-microphone array. Using an array helps with ambient background reduction and localizing of the sound source, thus better sound capture. There is also an active sound cancelling in place to isolate the sound coming from the TV or any other pre-identified source output. This feature also ensures reliability even with background sound.
Flowton Controller (execute)
Now to actually execute commands, the Controller needs to interact with your electronics and appliances. And it does so through:
Infrared (IR) – this is your TV, DVD, VCR, Stereo systems, and other.
- (1 peripheral comes standard)
Bluetooth – this could link to your Thermostat, Cell phone, Game Consoles, Kitchen appliances, and more.
Wi-Fi – issue commands remotely to your PC, Laptop, Tablet, etc.
ZigBee – aside from using this protocol to interact with other Flowton peripherals, it is also used in various home automation systems, alarm systems, home security, access control (garage doors), and such.
Smart plugs (with dimmer) – this pretty much constitutes everything else, from lamp, to fan, iron, coffee machine, alarm clock, radio, Christmas decorations, and of course the curling iron.
- (1 peripheral with the EXTENDED bundle)
Remotes be gone...
This is it - the final blow to the good ol' remote control. You have served us well...
But let's be honest. The remote controller is still good for SOME things. You just probably will not use it anymore to turn on your TV (it is much simpler now to wave your hand or speak a command). Or switch channels. Or change volume. Etc. The freedom of knowing you can issue any command with only your body is liberating. And once the remote is gone, so are the fights over who has it. Anyone can simply "pull up" the controller at any given time.
So yea, the remote controller is still welcome inside the house. It just won't get much usage in the future...
How easy is the Controller to use?
It takes a bit of practice, yea. But just as with the first mouse, the first touch screen – once you get it, you can never imagine it any other way.
This is a brand new user interface we are talking about here! One without a necessary screen, invisible yet present. Imaginary, yet functional. Without buttons. In essence, we had to fundamentally rethink the way people interact with their devices once at home, or would wish to, had the technology allowed it.
Gestures (Motion) -
Where the sensor can see you, is where you can use your body to execute commands. The Flowton camera unit has ~ 70 degree field of view (horizontal) and ~ 50 degree field (vertical). The practical range is between 0.5 - 3.5 meters away from the sensor.
This is our Flowton Field – a space within your home that is not so empty anymore.
When it comes to an attached sensor, you may refer to the sensor maker for exact details, but typically they all stay in the same operational range, and are close to the numbers stated above.
Everything starts with an initiator command – a certain action that lets the camera start watching you for other commands. This comes preset as the main command – a hand push towards the camera, and also programmable commands, such as you getting up from your couch, and many others.
Just put up your hand, and “push” a virtual button. Done? Ok – you now have a virtual TV controller around your hand… raise your hand a few inches, and the volume starts going up! Flick your hand left, and the channel changes down.
It’s that simple!
Next, push a “button” in the direction of your light stand. This way you directly bring up a “controller” for the light switch/dimmer. Again, move your wrist up to turn the light on, and bring the hand back to center. Now maybe hold your hand left for a few seconds to dim it, and get that movie theater feel.
But wait, no movie is complete without some chips! And as you get up, the Controller recognizes it, and springs into action – pausing the movie until you return.
You get the idea.
Voice (Sound) -
When the sensor can’t see you, is when you may tell the Controller how to behave. Just as with motion, this also requires an “initiator command”.
You can address the device you wish to control, as in the example of: “TV, ON”, or “LIVING ROOM LIGHTS, OFF”.
Intuitive commands such as “ALL APPLIANCES OFF” issued prior to leaving the house will ensure yet again, that you are preserving every ounce of energy.
Want to give your “butler” a name? In the future, you can! I mean, who wouldn’t want their own Jarvis?
Programming the Controller
One of the key important issues is how to program this Controller and connect it with all your current electronic devices. We have ensured that this experience will be swift and effortless.
After receiving the device, you will be required to connect the Controller to your PC, where a quick tutorial and set-up process will take place, which will be available through our website. Here you will create a personal network of all your home electronics and apply it to the Flowton Controller.
From there, you will be directed towards a section where you can assign devices and commands to your gestures. There will be plenty of template controllers available, for the television set, the lights, etc. Equally, you will be able to customize the Controller and personally tailor your brand new natural user interface to the liking.
This is a concept stage portrayal of how the graphic user interface will allow you set up your own controller.
What stage are we in?
We have developed our own motion tracking software with no dependencies. Basically, we had to build everything from the ground up. So there were hurdles along the way, but the result is that we now have a system that can be tweaked and played with, without any restrictions.
Our focus is depth tracking. It is simply unrivaled in object detection and tracking. Color camera tracking on the other hand has an increased rate of false positives in imperfect conditions, does not allow for complex gestures such as directional vectors (when you point to the device you wish to control), and the natural interface is generally less precise. That being said, we are committed to bringing a superb experience with every device, regardless of whether you choose to use the Flowton Controller as is, or supplement it with a depth camera.
Since our vision is to bring this product to the masses in form of a consumer electronic device, we had to make everything run ultra fast so we can have it on a tiny (relative to it's power) ARM board.
Basically, we have developed a proof of concept prototype that functions as intended in its current form. You can see the Flowton Controller V1 in the picture below (as well as in the video during the demo)
We didn't bother fitting a color camera into the V1 prototype because the processor + motion tracking + speech recognition is the crucial combination that we had to spend time on. Also we used a different motherboard and generally had to go through a few different iterations of the circuitry and other hardware inside.
We have finished the hardware designs for the Flowton Controller V2 and worked everything out to allow it to be mass produced. And now we need your help!
We have scoured far and wide seeking the best and brightest. Despite being small in size (under 10 people), we have assembled a well diversified team comprising of software and hardware engineers, designers, business folks and creative thinkers.
The current lineup goes as follows:
Software - Daniel Mohammadi (BASc)
Computer Vision - Ray Lo (PhD)
Hardware - Martin Labrecque (Postdoc)
Computer Science - Amir Hejazi (PhD)
Designer - Faraz Farizi
Product - Alec Gordon
Finance - Kevin Panchalingam
Adviser - Danielle Zhou
Individually, all of our members are over achievers - from running their own hardware companies, to spearheading tech development in fortune 500 firms, to publishing research papers in top journals, and winning scientific innovation awards.
Collectively, the combined expertise and aptitude for building great products is much greater than the sum. Everyone gets to wear many hats and influence the product from every angle. We have every facet of development covered, and are looking forward to building the platform of tomorrow.
After nailing down the core concepts of the interface, we subsequently filed for a patent for this novel method of control. Also we have received a research grant to study the human side of the equation - to find out how well people perform while navigating through this new natural user interface, what can be improved, and so on.
True remote control
Anything you can connect to you can control. The future trend is that of everything going wireless. Thus there is a good chance that with ensured software updates our Controller will age exceptionally well. We would be perfectly happy if all currently owned TV sets had Bluetooth. But they don't, IR is still very much present, and so we have to build around it.
Soon, even your toaster (?) is going to be wireless enabled, it will become smart, and light switches are all going up into the "cloud". With enhanced wireless functionality more and more electronics and appliances can be connected to a personal network.
And we will work on applying that personal network of devices onto an ever present interface that any person can pull up using only what nature gave them. Our bodies are sophisticated enough, we shouldn't need to actually touch anything.
We are working hard on creating an environment where Natural User Interface enthusiasts and developers can all come together and be able to build incredible stuff. There are plenty of ideas already floating around, but we can only build so much, and will need your help!
With depth cameras, think about being able to ID and track any XYZ point in space. You can drag it around. You can issue a trigger command to another. You can place a light switch onto an empty wall. Or if the light stand is within the field of view, you can put the "switch" right onto the appliance, and so you can now touch it and it till turn on/off. This is something we want people to grasp, and then run away with. The possibilities are truly limitless. And we will work very hard on opening up the development to all those who want to build with us.
Follow the updates for when our API will be released. You can expect some time later this year, well before the products ship, so you will have ample time to develop something groundbreaking.
Flowton Controller -
- 1.7 GHz processor
- 2GB of RAM
- 640x480 VGA RGB camera @ 60 fps
- 2 microphones
- 8GB of flash storage
- WiFi 802.11
- Ubuntu OS
Risks and challenges
Just as with any hardware product, there are some inherent challenges that come with working on bringing the best and most functional device to backers.
It is not enough to put together some parts, write some code, and then multiply it in a factory. We had to rethink how to assemble all the powerful hardware into the smallest form possible. There are many aspects that we needed to consider in order to come up with a more efficient and cheaper solutions. A huge amount of time and effort was put into designing, testing and manufacturing the circuitry needed. And in the process we had to design few key parts ourselves.
CHALLENGE 1: Manufacturing
We have developed some key partnerships that will allow us to swiftly transition from building prototypes in the eponymous garage, to mass producing quality consumer electronics devices.
Equally, we took the necessary steps to design a number of our own hardware components along with the enclosures as they will appear in final form when shipping to backers. We have researched the necessary production material costs and validated critical components against required specifications. That said, we had to resort to some strategic component sourcing in order to make the ship date.
However it would be foolish to assume that everything will remain the same once the production process actually begins. Despite the fact, we stand prepared to face whatever challenges come our way, and are committed to delivering an exceptional product experience in the allotted time.
THE NEXT 7 MONTHS
The time after Kickstarter will arguably be the most critical time yet. We have been actively working on the Controller for over a year and a half now, and we utilize research from our team that extends yet further back. We are on track to finish the hardware development in the proceeding months, in order to deliver this product to you early next year.
So our revised timeline will look as follows:
September 2013 - Finalize all hardware designs and ready the parts for manufacture
October 2013 - Obtain the necessary safety/electronics certifications
November 2013 - Pass on the product designs and ramp up production. Backer portal
December 2013 - Complete the SDK and establish smart home integration
January 2014 - Testing and revision. Ship out the Dev Kits
February 2014 - Ready the GUI and finalize the NUI of the Controller
March 2014 - Package everything neatly together and start shipping
It is a tall order to fulfill the obligations in a timely manner, however most of the design choices have already been polished out, and our own brand of user interface has long been in the making. The task remains to deal with whatever setbacks as they come (manufacturing is notoriously tricky), and make sure we do not deviate from the plan at hand.
CHALLENGE 2: Natural User Interface
Transforming code into a usable user interface that people will actually get utility out of requires plenty of abstract thinking, and is the other difficult half to take care of. There have been a few examples of natural user interfaces to control the television set, however since they generally replicate the pointer on screen, and thus rely on the presence of the screen, they cannot be ported well onto other everyday devices without serious restructuring. Not only that, it is important to create something that will be easy and intuitive, and devoid of minor imperfections that can take away from the seamless experience.
We have invested considerable resources to try and design a functional interface that people will have a fun time with. We want to infuse new life into mundane interactions that people often avoid if they could. That is why we are keeping a close eye on our beta testers and are extracting valuable data of user interactions.
For more information and insight please visit our website.Learn about accountability on Kickstarter
Support this project
- (35 days)