Share this project


Share this project

imitone lets you play any instrument with your voice.
imitone lets you play any instrument with your voice.
2,433 backers pledged $90,517 to help bring this project to life.

Recent updates


76ffc359a074bea1121a154c356a7172 original.jpg?ixlib=rb 1.1
B36540b165bc4a852f8168860e9d3c81 original.jpg?ixlib=rb 1.1
C9f572ce40e0b92fcb28735dcb4568a9 original.png?ixlib=rb 1.1
C9cdc0d5deb4ffb3544dc96dd7bfdcda original.png?ixlib=rb 1.1
20203631b45c799a1aad258f0b1dd0e6 original.png?ixlib=rb 1.1
40c0dad05dd50841f35f65bf2078a23d original.png?ixlib=rb 1.1
00f402775238190c123384a43ebcc1f9 original.bmp?ixlib=rb 1.1
25ae57cfa80584cee79eb1482a7e5d71 original.jpg?ixlib=rb 1.1
0c95754c7a7c064add930dc818305492 original.png?ixlib=rb 1.1
85d7d9e6d2c59bec4bf7ec53e6253691 original.png?ixlib=rb 1.1
429eec305b6dd18ea560b1549d2872b0 original.png?ixlib=rb 1.1
4f034c7411fa38cbcf2d0b4354141023 original.png?ixlib=rb 1.1

Crossing the Gulf

Posted by Evan Balster (Creator)

Lately I've been thinking about the story of the tortoise and the hare.


I've been working on imitone for more than six years, and there is still a lot left to do.  Since 2014, I have seen many things like imitone race right past us, into app stores and news stories.  I remember feeling nervous a few years ago, when an imtone backer told me about another voice-to-MIDI plugin premiering at NAMM.  I went to see it, but by the time I got there, it had been pulled from the showfloor — and the internet, too.  It hasn't come back.  Why?

When people get excited about my work on imitone, they always talk about the potential.  I've shown it to beginners and musical veterans alike, always raising eyebrows with the technology.  Most comparisons have concluded that imitone responds to voice better and faster than anything else available.  But when someone sits down to actually make something with it, the magic starts to fade.  The tech is working beyond all expectations, but somehow it isn't... working.  Not without a lot of skill and patience.  Not "like magic".  Why?

I think there's one answer for both questions, but it's subtle.  Psychologists have a term, "gulf of execution", for when a tool doesn't do what we expect.  imitone works well when our expectations are loose — say, when improvising.  But when we expect something specific — like a song we've had in mind — the results are confusing.  We get notes that are faithful to the pitch, but not the music.

After years of work, imitone is not yet the answer to the first question we asked with our Kickstarter campaign:  "Have you ever woken up with a melody in your head?"

The Gulf

Sometimes, the best tool we can imagine isn't good enough to do what we need.  What if I need to hit a nail, and I've never seen a hammer?  I could close my eyes and picture a tool for the job, but it probably won't be as good as the hammer at the hardware store.  To me, "The Gulf" has started to mean the space between that first, blind imagination and the real tool — which took a lot of work by a lot of people to get to where it is today.

imitone's job seems simple:  Play the notes we sing, as we sing them.  A simple job makes it easy to imagine a simple tool...  But I haven't ever seen a tool that does this job well, and I have spent a very long time trying to imagine and build one.  The Gulf was much wider than I thought...

Surprise Valley, US Geological Survey
Surprise Valley, US Geological Survey

The longer I spend in this place, the more I learn about its history...  It's like I've passed by the camps (or the graves) of many other explorers, going back decades.  I've met people who left their footprints here before I was born.  Almost everyone has had to stop somewhere — to wrap up their journey in this realm of invention.  But there are a few others who are still searching, after many years...

Maybe it's silly, this idea of an imaginary desert testing my endurance.  Maybe I'm making excuses for spending too long, being a perfectionist... but I can't imagine doing it any other way.  I've chosen to gamble on a faint sense that the breakthrough we need is out here.  Right now, I'm moving towards it in the best way I know.  That means taking my time, following my compass and refusing to call imitone finished just yet.  I have the feeling that very soon, I will find some green and undiscovered place.

Know that I do all of this in the name of making silly noises at computers.  :)

The Research

So, about the work.

imitone's fourth phase of research and development began in October.  Originally planned for mid-2017, this dive into imitone's tech is some of the most important work I'll do in the project — and some of the most exciting.  It has me revisiting all the theories I've developed, all the discoveries I've made and a long list of loose ends and wild ideas.  I'm rewriting imitone's blueprints and building the tools I'll use to finish it.

imitone only sees the past, and often needs to guess before the sound can be seen clearly.
imitone only sees the past, and often needs to guess before the sound can be seen clearly.

Crucially, I have learned that even with the best possible sense of pitch, imitone can't always be sure about what it's hearing.  In the split-second when a sound begins, the tone might be weak, or there could be breath on the mic, or an echo in the room.  Each of these will blur imitone's picture of your voice.  On top of that, your pitch could be moving around, or it might be halfway between two possible notes.

If we make our "best guess" based on a blurry picture of your voice, or an out-of-tune sound, that guess could be wrong.  In these cases, imitone should use a sense of the music to choose which note to play.  To have that sense, it should know as much as it can about the song — and the way you sing it.  The more it knows, the better it can understand your music when it doesn't have a clear picture.

Right now, imitone only knows the basic key and scale you choose for it — and it doesn't use that knowledge very wisely.  In the future, imitone will listen and learn about key, scale, rhythm and musical style as you sing.  It will also gather clues by looking through MIDI 2.0 connections into the projects and instruments on the other side.

Future versions of imitone will be able to handle key, scale and range automatically.
Future versions of imitone will be able to handle key, scale and range automatically.

A truly educated guess should consider not just the evidence, but the strength of each piece of evidence.  This means imitone should think in terms of probability to pick the right notes.  My 2016 research left me believing that this was the key to crossing The Gulf.  Probability led me to statistics, which led me to "machine learning".

Importantly, while imitone needs to listen, learn and play like a musician, it isn't one — you are!  Any high-tech "music sense" we build only needs to kick in when there's a tough decision to make.  This will happen less often for experienced singers.  For beginners, imitone can take a more active role, helping to play notes that fit into the song.

Pitch Tracking & Back-Tracking

The first step in our new work is to do something over again:  Learn how to instantly recognize pitch.  imitone is already one of the best tools for that!  But it does something a bit foolish that keeps us from moving forward.

imitone's pitch-tracker listens to your voice and makes a "best guess" about the pitch (or non-pitch).  We send this guess to the transcriber — the part that picks notes to play.  But we don't say whether it's a good guess.  It might be pretty bad if we only have a blurry picture of the sound!  This means that imitone can't consider the strength of the pitch evidence when choosing a note.

imitone tries (and fails) to understand a trill.
imitone tries (and fails) to understand a trill.

For example, you can sing a trill (a high, rolling "rrrr" sound) and imitone will imitate it with hundreds of little notes.  The sounds making up the trill are very short and your tongue is a bit noisy, so imitone doesn't get much evidence about each one.  A tiny sound makes for a blurry picture.  That turns into a rough guess, which becomes a messy note.  imitone doesn't consider how literally hundreds of notes with the same pitch have happened in the last second.  Nice one, imitone.

It turns out that we even use "best guesses" like this inside pitch recognition, creating even more problems!  To make imitone better, we need to use probability all the way down to the sound wave and all the way up to the musical score.  That involves some serious work on the math — but it also means there is enough room to make a big improvement, by fixing all of these smaller problems.  If imitone could do a really good job of recognizing pitch before, it should be able to do an amazing job with probability!

Good news:  I've worked out the theory, most of this redesign is done, and I am beginning to "train" imitone's new pitch tracker.  This works a bit like the holodeck in Star Trek — I make a training program, and put the tracker in it.  It listens to hundreds of thousands of simulated sounds, and learns to understand them through practice.  Eventually, it comes out fine-tuned and ready to do its job in the real world.  I expect this machine-learning approach to take imitone much further than the hand-adjustments I used before. 

An early exercise to train imitone's "self-attunement" math.  Looks like an eyeball!
An early exercise to train imitone's "self-attunement" math. Looks like an eyeball!

Going further, it's possible that imitone could learn the sound of your voice (or instrument) over time, forming a special "matrix" that represents all its experience with that sound.  imitone would learn to respond better and faster to familiar sounds.  While a feature like that is science-fiction for now, it's a way we could advance imitone even more in the future.

Scaling Up

Once I get imitone's new pitch tracker ready, I will begin work on its new, probability-based sense of music.  This will tie into long-planned features like automatic scale detection, and also some unexpected features, like rhythm.  I have a lot to figure out about how to use this knowledge for split-second decisions, and how imitone's "listening and learning" will work.  I expect these projects to take time. 

As part of this work, I have decided that every version of imitone should include a selection of musical scales and tunings from around the world — representing not just Western scales like major, minor, jazz and blues, but also Arabic, Chinese, Indian and Indonesian traditions (and more).  Your voice is not limited to the notes that can be played on a piano, and imitone shouldn't be either!

Traditional Indonesian music uses notes that don't exist on the piano.
Traditional Indonesian music uses notes that don't exist on the piano.

I am looking for input on musical traditions to include with imitone! 

Our Work Continues

Research work has slowed down temporarily as I prepare for The NAMM Show next week, where imitone will be part of the MIDI Manufacturers Association booth.  My work there has been a big investment in the future of digital music-making, and I'm investing now more than ever:  The Association is running a special event to test the first batch of MIDI 2.0 prototypes, and imitone will be one of them.

I'm also working on an app for Android, which has become a much better platform for audio software since 2014.  Thanks to the efforts of Phil Burk and the Android Audio team, the latest versions (Oreo and Nougat) support low-latency sound — this will help imitone to feel like a pocket-sized musical instrument.  Unlike our secret MIDI 2.0 prototype, I'll be able to show this early Android app to the public at NAMM.  A mobile beta is on the horizon...

Step by step, I'm getting closer to the tool we need.  imitone has been growing, and soon it will flower, becoming something we haven't seen before.  Exciting things are coming.

— Evan

Frenchman Flat, US Department of Energy
Frenchman Flat, US Department of Energy



P.S.  Wait, MIDI 2.0??

Oh, yeah.  This is a real thing that is happening.

MIDI is the language of digital music.  When you sing, and imitone plays a note, MIDI is what takes that note to the digital instrument you're playing, or the song you're recording.  Because of MIDI, imitone can connect to thousands of other music tools.  But MIDI was created 35 years ago.  It's old-fashioned, and there are lots things it can't do, or does in an awkward way.

MIDI 2.0 was announced yesterday, and will improve and modernize MIDI without replacing it.  I have been involved for over two years.  With it, imitone will be able to "scan" the songs and instruments on the other side to automatically pick the best settings.  For example, imitone might learn it is controlling a digital violin and automatically go into "strings" mode.  Other settings like pitch bend and controller mappings can be automated, too.

For more information, I suggest searching for "MIDI 2.0", "MIDI-CI" and "MIDI Polyphonic Expression" (which influenced MIDI 2.0).  This is a deep subject, and I'll have more to say in a future post!

Looking Back

Posted by Evan Balster (Creator)

Hello, all —

It has been a while since my last update here.  About 18 months.  Where do I begin..?  Despite the quiet, a lot has happened in this time.

Most of the work involved in late-stage development is boring and tedious, with some highlights.  I can't do justice to all of these without writing a very long post.  I'd like to get in the habit of writing more often, though — and this project still has a lot of untold stories.  Let me know in the comments if there's something you'd like to know more about!

● January 2017Improved our error reporting, part of an effort to make imitone more reliable.  Prepared for NAMM 2017, where I worked with Jim Crawford to design some of imitone's built-in instruments and to demonstrate a futuristic "perceptual" technique to MMA.

● February 2017Added a sampler for imitone's more "realistic" instruments and made preparations for work on a mobile prototype.  Worked on an imitone research tool.  Participated in protocol design with MMA.

 ...Wondering what this "MMA" thing is about?  No, I'm not wrestling anybody.  The MIDI Manufacturers Association (MMA) is responsible for developing MIDI, the universal language of digital music.  Because imitone uses MIDI to send notes to instruments and other music apps, it's a pretty big deal for us!  There's also a MIDI Association (MA) which is open to creators of all kinds — you can join here.

I happened to join MMA at a very special time.  Since 2017 there has been a fresh effort to design a "Next-Generation Protocol" to improve MIDI, and I have been an active contributor.  Features under consideration include higher resolution, per-note effects, and (most importantly for imitone) the ability for the controller to learn about and adapt to the instruments, apps and projects it's connected to.

imitone is the smallest and newest company involved in these efforts.  Participating has been a tough choice — it's a huge investment of time and it is undeniably stretching out imitone's schedule — but by chiming into these conversations I can ensure the Next Generation of MIDI will work for us, and make exciting new things possible for beginners and experienced musicians alike.  Great things are coming...

● March 2017Began developing a mobile imitone prototype.  Designed a mobile UI, including pop-out menus which now appear in the PC/Mac version.  Had a long conversation with imitone's designer Richard Hogg about how our design will evolve in the future as we strive to make it simpler and more accessible.  Attended the Game Developers' Conference.

● April 2017Completed imitone's first mobile prototype.  Aided in some Next-Generation Protocol prototyping.  Ported my other project, SoundSelf, to a game console.

...I know some of you have been eager to use a mobile version of imitone!  Unfortunately a few hard problems need solving before our iOS prototype can turn into beta.  Apple's requirements have changed in the time since our Kickstarter campaign, and we're exploring our options with the advice of fellow developers.  (We're also excited about working with the new low-latency audio technology in Android Oreo!)

Around April I started keeping detailed train-of-thought journals for each day of my work — one of many habits I've built over the years to deal with the "tunnel vision" of working alone on these long projects.  I notice I spent a few evenings in April on a math formula that creates huge, disorienting labyrinths...

● May 2017Collaborating with music teachers, I designed a special synthesizer that enhances the singer's sense of deviations in their voice pitch — only to discover that this invention makes it feel absolutely awful to sing.  Shelved it.  Proposed some compatibility tricks for MMA's protocol.  Improved SoundSelf's VR compatibility.  Upgraded my development tools.

● June 2017Streamlined my process for shipping updates to imitone and SoundSelf.  Worked on SoundSelf's VR and console ports.  Researched the "T-Product", a statistical technique I'm developing for the next generation of imitone technology.  Developed some research tools for imitone.  Continued to participate in MMA's protocol work.

...I spent several scattered weeks in the summer with a family member in declining health.  In the past my busy schedule has sometimes led me to neglect family matters, but I've learned it's important not to let life pass you by...

● July 2017Simplified imitone's setup menu with shorter device names and an illustration of the audio/MIDI flow.  Improved the app's visual fidelity.  Released imitone 0.9.1 and 0.9.2.  Prototyped a feature for MMA that allows imitone to control digital organs' drawbars, allowing the singer to make the organ "speak".

● August 2017Took imitone to the PixelPop Festival in St. Louis, where I tried a new type of installation with imitone operating alongside a DJ.  This experiment was an educational failure.  (It's hard to keep imitone in sync with key changes.)  Watched the solar eclipse in Wyoming.  Fixed several bugs in imitone (0.9.2a-0.9.2d), including an issue with international usernames.

...In late August I began planning imitone's plugins, at the insistence of some voices in the pro-audio community.  I've noticed I can be eager to please my more vocal users, but this can lead to long detours in my work that take time away from "core" efforts.  For example, I've logged a lot of time chasing down bugs that affect only a small handful of users or machines.

Plugins turned out to take far longer than expected, becoming another sub-project I need to juggle.  While one of these moves forward, the others stand still, and I've been pushing crucial research down the road.  While I won't say I regret my choice, this gives me more understanding for developers who choose the "quick and dirty" path toward a rougher, buggier or less-complete final product...

● September 2017Began reworking imitone's code to lay a groundwork for developing plugins in multiple formats.  Developed some initial proposals for standardizing the "perceptual" magic trick I had demonstrated to MMA in January.

● October 2017Continued intensive work on plugin development.  (I also spent a few weekends building a peculiar device: a special electrolarynx that turns the wearer's head into a powered speaker and lets them shape the music with their mouth.  The user must hold their breath in order to "sing", making for an odd performance.)

● November 2017Continued plugin work.  Attended a convention in Austin where I did some more work on SoundSelf's console port.  During this time I also consulted with accessibility expert Adriane Kuzminski, designing imitone's VST to be blind-accessible with screen reader extensions for Reaper or Pro Tools.

● December 2017.

...This month, my anxiety about imitone's schedule came to a head.  Somehow a year had passed with only minor updates to the app, and after several months of intensive development the VST plugin wasn't ready yet.  I was working myself into exhaustion with only a vague sense of where the project was heading.

I'm not sure how I can explain the feeling.  This article makes a neat summary of what I did each month — but in a state of "tunnel vision" there is no such awareness.  When something gets done, there's no need to think about it any further.  Any short-lived sense of satisfaction vanishes, leaving only the expanding to-do list.  Looking back, I could only think "no updates since July!"  Looking forward, I could only think about how much work still remained.

All of this, I think, was a symptom of spending too much attention on work and too little on management.  Even as a "solo" developer this is a surprisingly costly mistake.  Contrary to the myths about how things like this are done, simply "working harder" can lead to catastrophically poor choices about time and strategy.  While I know by now how to avoid burnout, I still have much to learn about production planning.

My journal paints a picture of where I've spent my time these last 18 months. Each square is a timed workblock. imitone is orange (and some of the grey).
My journal paints a picture of where I've spent my time these last 18 months. Each square is a timed workblock. imitone is orange (and some of the grey).

As the holidays approached I decided to take some time to improve my tools for keeping organized, reviewing my past work, and assessing my priorities with this project.  Centrally, I revived my old habit of keeping a visual journal of how much work I'm doing and where it's going day-to-day and month-to-month.  I customized the excellent note-taking app TiddlyWiki to integrate my daily journals with my work logs, painting a color-coded picture of my work over the prior year (pictured above).

This was hard work, but deeply rewarding.  With this picture painting itself week by week, month by month, I could remember my place in these long and ongoing efforts.  Stopping to improve now and then is important, and that requires looking back.  Taking time to reflect on recent successes and failures so I can improve...

● January 2018Used my new tool to catalogue and color-code my 2017 work logs.  Continued plugin work.  Discussed plugin-related concerns with MMA.  Attended NAMM 2018 where I spoke with various engineers, educators and musicians.  Attended MMA's annual meeting and some special events related to the "Next-Generation" initiative.  Spent my free time improving a math extension I created for TiddlyWiki in December.

● February 2018Developed imitone's VST3 plugin into an "alpha" plugin and quietly released this to our customers.  Wrote some MMA proposals about MIDI compatibility.  Met up with SoundSelf's creator for a coworking period focused on our console port and other development goals.

● March 2018Further coworking on SoundSelf.  Spent a few weekends writing a computer program to investigate a hard math problem related to the "T-Product" technique.  Eventually solved it in an exciting breakthrough!  Attended the Game Developer's Conference and participated in Train Jam where I collaborated on a small voice-controlled game.

The VST alpha as it appears in Reaper. It's pretty bare-bones for now! We'll be adding imitone's signature look-and-feel soon.
The VST alpha as it appears in Reaper. It's pretty bare-bones for now! We'll be adding imitone's signature look-and-feel soon.

 ● April 2018Drafted a large proposal related to MMA's Next-Generation Protocol.  Developed and released a VST version of the imitone plugin.  Improved the way VST and VST3 plugins handle presets.

● May 2018Spent the month on a new graphics system for SoundSelf, which is approaching completion.  This technology will also help us add imitone's UI to the plugins, which are currently a bit hard to use.

● June 2018Continued working on the new graphics system (which ran over-schedule).  Began participating in an MMA prototyping initiative — as you might expect, I want imitone to support MIDI's Next-Generation Protocol from day one.

● July 2018.

This month I'm continuing to prototype with MMA, planning out the next few months of work and preparing for a new adventure:  I'm moving out of Iowa and will be living (and working) on the road for a while.

There are still some difficult, demanding tasks involved in getting my other project SoundSelf across the finish line, but most of that (along with the tough parts of plugin development) are behind me.  As I prepare to leave my home of six years, I am finally finding room to work on the most rewarding parts of imitone:  Research and user interface.

"Phase 4" research has been pushed back for more than a year, but this is where imitone's most important improvements will be made.  It will take two to three months, but I don't want to delay it much further.  Phase 3, back in 2016, left me with a slew of promising leads, and clear direction for improving how imitone detects pitch and transcribes it into music.  (The "T-Product" technique, first theorized in 2016 and finally solved this year, promises to be an essential tool, and might have applications in the broader field of signal processing.)

Aside from that I'm also interested to bring our plugins to Mac, add imitone's interface to them and make a small update to the imitone app, improving MIDI output, fixing some bugs and tweaking the UI.  A Linux port is also overdue (my thanks for the friendly reminders).  Lastly, I'd like to produce some tutorial videos to help new users get started with imitone, and add some instructions for using imitone in live performances.

Looking back, I can see that most of this project — and most of the really tedious work — is now behind me.  More is done, substantially, than is left to do.  Acknowledging that gives me a sense of relief and accomplishment.  Not too far ahead, I think, lies the tool I've been looking to build all these years.

I'll continue my quiet work to make that tool a reality, for as long as it takes.

— Evan

imitone 0.9 — a big step forward

Posted by Evan Balster (Creator)

On December 1, we released imitone's biggest and best update ever.  Go get it!

imitone 0.9 is much better at following your song, thanks to a smarter note interpreter.  It has also been re-designed, comes with a shiny new manual, and introduces three big features that have been a long time in the making:

Hold Mode

Previous versions of imitone have always been eager to change notes, playing near your voice's pitch at all times.  This can be a problem when trying to sing long, steady tones: if you lose control of your pitch for even a split second, it could play another note.  It can also cause trouble when singing a low note, followed by a high note — often imitone would pick up on a quiet pitch slide that you weren't trying to make.

Too many notes!  Left: Sweep Mode.  Right: Hold Mode.
Too many notes! Left: Sweep Mode. Right: Hold Mode.

When your pitch is sliding, 0.9's Hold Mode will keep playing the original note until you settle on a new one.  This produces a tidier score and is especially helpful for composing.  We've kept Lock Mode around as an even more careful variation.

Another classic problem.  Left: Sweep Mode.  Right: Lock Mode.
Another classic problem. Left: Sweep Mode. Right: Lock Mode.

While I plan on fine-tuning it, this first take at hold mode works very well!  I also had a few new insights into how to make the other slide modes smarter — you'll notice that even the old modes are less jittery than before.

Pitch Guide

If you're not a trained singer, odds are you drift out of key now and then.  Until now, imitone would always try to choose the note closest to your real pitch.  This caused a problem when singing out-of-tune, near the half-way points between notes — imitone would round some notes up and others down, making it feel random.  I described this problem in detail earlier in the project.

I'm singing the same note, out of tune.  Tiny changes in my pitch change imitone's note.
I'm singing the same note, out of tune. Tiny changes in my pitch change imitone's note.

The key is in the way people hear music:  We have a much better sense for difference in pitch than for exact pitch.  Most of us can't sing a perfect "C" without some help, but we can usually get the distance between C and G right.  When we sing out of tune, Pitch Guide appears as a white fork, and adjusts imitone's "guess" until it matches up with the scale we've set.  It carries over between notes, so that the movement (C-G) will be the same in our song.  This way imitone can understand what we're trying to sing, even if it's out of tune.

On the right, pitch guide kicks in and saves the day.
On the right, pitch guide kicks in and saves the day.

Highly skilled singers may not need pitch guide, but most of us will want to leave it on at all times when making music.  The effect is subtle, but important:  It makes it less of a problem to drift out of tune, which can happen easily when we can't hear our own voice.  It also makes imitone work automatically with unusual tunings.

Instrument Presets

As we add settings to imitone, it gets harder to understand what they all do.  Presets help by providing suggested settings for controlling different groups of instruments.

Daah!  A piano's energy comes from the beginning of the note, and we usually don't want these notes to slide around.  The keys preset, also good for bells and xylophones, combines attack triggering with hold mode so that we can land notes one-by-one.

Haah!  A violin, on the other hand, has continuous energy, and can lilt from note to note without a break in the sound — so we use sustain and portamento mode, with vibrato control switched on for good measure.

Biddly-diddly-doo!  A saxophone can dance all over the scale, playing many brief notes in between longer ones.  For this we use imitone's old defaults — sustain and sweep.  This also works great for electric guitar.

We chose to give the presets vague names because they're just guidelines.  Some saxophone performances might sound better under the "synth" preset, while some synthesizers might play better with "keys".  "winds" and "strings" are pretty interchangeable.  So use them as a compass for understanding imitone, and experiment freely!

The Other Stuff

Keys up top!  Pretty rainbow lines!  New manual!  We've put some serious effort into making imitone look and feel nicer with this update.  The internal technology has learned new tricks that make it resistant to room echoes, smarter about pitch and volume, and far more flexible with CPU power.  "Quick-Bend" (hold the spacebar) lets you temporarily change the slide mode while performing.  Blues scales are delightful!  The change list goes on...

The one other addition that's worth special attention out is our new error reporting system.  When imitone crashes, or has a problem, you'll see a window like this:

If you see one of these, please hit send!  This way, we'll get a pile of technical information about the problem, along with an optional message from you.  Thanks to this tool and the users who sent us reports, we able to fix five major bugs with three patches in the first week after release and a fourth on the way.  (Developers:  Our report uploader, Tattle, is open-source.)

In summary: it's a really good update.

People are starting to put imitone to more serious use for making music, because it's more expressive and reliable than ever before.  It's starting to create the sense of proficiency I wrote about in 2014.  Seeing users succeed with 0.9 has me feeling comfortable with the idea of bringing it to a wider audience.  While it's likely there will be an 0.10 before release (good luck alphabetizing that), this is a big milestone in the journey toward our final product, and enabling more people to be musical.

The funny thing is, this update didn't really go according to plan.  Contrary to what I said in July, I've done very little new research for 0.9.  Instead, the process behind these improvements was a happy accident, born from a question that bothered me back then:  How much potential does the current technology have?

I'll be telling that story in another post, along with our plans for the next stage of development — but for now, go give the new imitone a spin; I think you'll be pleasantly surprised.  Don't hesitate to share your thoughts or questions via the in-app feedback button.

Have fun, learn things, make music.  :)
— Evan

imitone 0.8.2, Research and Accountability

Posted by Evan Balster (Creator)

In this post I’ll discuss my challenges in demonstrating progress and choosing the right path foward as I work through research and development on imitone.

imitone 0.8.2

At 1:30 AM last Thursday, I quietly released imitone 0.8.2.  I did this from a bench outside a canoe rental store in the remote town of Bluffton, Iowa, on the night before a family trip. Both the trip and the update had been many months in the making—so I chose to wait until returning home to announce the changes.

You can update to imitone 0.8.2 here.
(Note: This version may be incompatible with Mac OS 10.5)

Version 0.8.2 will not, at first glance, look very different from 0.8.1.  Play for a few minutes, though, and you might see a substantial improvement in the feel of the software.  This is because, in addition to other enhancements, I’ve built the rough beginnings of a new stage in imitone’s pipeline, called the interpreter.  Think of it as imitone's musical intuition:  its job is to handle the tricky decisions involved in interpreting pitch into musical notes, which has long been the “weakest link” in imitone’s chain.  The one here is temporary—a stand-in for a much more powerful algorithm that has been at the center of my research efforts.  That algorithm, along with related changes in the pitch tracking system, will form the technology used in imitone 1.0.

In the four months following my last update, almost all of my working time was spent on this process of research and development.  I chose to do this in spite of my earlier announcement that a VST plugin was underway, for two reasons: First, VST is a standard for professional tools, and I did not judge the technology to be up to that level.  Second and more importantly, finishing imitone is going to require a rigorous schedule, and I cannot make one until I have a complete plan for the technology.  That will require research to be near-complete.


At this point I’ll stop to address the elephant in the room: The Kickstarter campaign was more than two years ago.  I’ve spent many months on research lately, with only a slow trickle of tangible progress.  I’ve logged long hours, built specialized tools and taken hundreds of pages of notes, but as far as my backers are concerned, these are invisible. For all they know, I could be living in comfort on their pledges, doing scarcely any work at all—and as time has gone by, some have voiced their concerns.

I have strong feelings about my professional integrity, and lately this problem has weighed heavy on my mind.  As crowdfunding gets bigger, we see more glaring examples of irresponsible, shady or outright fraudulent behavior on the part of project creators—this damages the reputation of crowdfunding as a whole.  On a more personal level, let’s not forget that I myself have the disgrace of a failed project.

In trying to show diligence, I’ve issued a few dozen prompt refunds over the course of the project, each time asking my unsatisfied customers to let me know how I could do better.  I’ve put best efforts toward technical support, endeavoring to respond to each request within a few days and rethinking my system when I have fallen behind.  And, more than anything, I’ve worked long and hard on the technology, with careful efforts toward making sure my work doesn’t become sloppy or deviate from the long-term vision.

When it comes down to it, though, none of these are the best ways to assure my backers I’m fulfilling my promises—and that assurance is my responsibility as a project creator.  To earn that, I need to show you proof of progress, and transparency about my process.

Proof of Progress

To make a gesture of good faith and avoid a repeat of last year’s awful nine-month lull, I put a hold on research to build last week's update.  My goal was to pick the "low-hanging fruit" from my research and make short-term improvements.  I initially expected to complete this two weeks earlier, but because one of the new technologies I tried to add was not ready, the differences (aside from a big reduction in CPU usage) were not as obvious as I hoped: For tracking, it was two steps forward and one step back.  For this reason I decided to hold back the immature CPU-saving technology and build a basic interpreter instead.  This time-consuming job, which was originally planned for a future update, paid off—and the update exceeded my modest expectations.

This takes a little pressure off, but I don’t expect that sensation to last long! Whenever I’m building an update, I’m always worried it will be too small or seem like a step backward.  (This was definitely the case with 0.8.2!) Whenever I actually put one out, though, I feel the opposite:  That I should be making smaller, more frequent updates in order to get more feedback and appear more active.  In the approach to completing imitone, I’d like to adopt the small-update strategy, but for now there’s a big problem stopping me.

Process Transparency

Research is a strange process.  I’ve written about it before, but I know more now than I did then.  And here’s what I’ve learned: It’s the process of discovering new things and making absolutely sure I understand them.  It's almost impossible to predict how long these discoveries will take, so I can't schedule around them.  The best I can do is to make the process as efficient as I can.

I can’t speak for others, but to do good research and make progress, I need to dedicate as much of my time and energy to it as possible.  I need to become obsessive, leaving part of myself at work at all times.  When I’m in that mode I fall asleep slowly, lingering on the problems I face.  I wake up in the morning realizing the technical terms I’ve invented the day before have been repeated over and over in my dreams.  I make breakthroughs when showering or brushing my teeth, and feel the rush of exhilaration (or disappointment) when I put these ideas to the test.  I spend hours or days trying to codify, expand and generalize the theory behind them.  This is science.

Once, I fumbled in the dark.
Now I can find my way to the lightswitch, but it takes everything I have.

…Flowery language aside, I’ve learned that I make the most progress when I work like this for weeks or months at a time.  It’s a completely different process than the creative programming that led to the original imitone—more time-consuming but far more insightful.  And, as discussed above, the mysteries need solving before I can fulfill my promise.

This leads us to the present: Before developing this update, I spent months working on imitone without touching the code once.* When I returned, I took a few of my smallest and most complete discoveries and quickly turned them into improvements.  With the fruits of my study, it was surprisingly easy to make that happen.  Imagine what I could do with a complete blueprint.


I know that some of my supporters and observers are deeply frustrated by my choice of priorities.  My most vocal critics seem to be those experienced in music technology, and overwhelmingly demand the VST port that will allow them to integrate what I’ve built into their workflow.  Many of these musicians have been around for a while; they’ve seen other audio-to-MIDI systems in years past, and they see in my project an improvement over those.  In that light, nothing about what they’ve asked of me is unreasonable.

But I’m not looking to improve on the broken promises of past technology—I’m looking to invent something completely new.  I set out, years ago, with the dream of building a tool that could give the feeling of musical ability to someone taking their first steps into it. Something that would work like magic.  I’ve been chasing that star for years, and more than ever before I feel like I know what I’m doing.

Now, the grand design is taking shape: It will lower the floor, raise the ceiling and, perhaps, finally solve this laughably simple problem to which I’ve dedicated years of my life.  And when I achieve that, it will be clear that each stage of the work in progress was just an imitation of the work itself.

The Way Ahead

From here, I believe I see the most effective path forward—but it could be dangerous if I don’t have the confidence of my supporters.  I cannot say for sure, but I believe most of the research is done: I have a detailed plan for the backbone of the final technology, but filling in the gaps requires me to solve a few more problems.  I want to dedicate as much of my time as possible to completing it.  Again, I can't predict how long that will take; it could be one month as easily as five.  My blind guess is about two.

When I do finish research, I’ll be able to make the kind of rapid progress I’ve made with 0.8.2—but on a much larger scale.  In addition, I’ll be able to transition completely to a software development mode, making tech improvements in tandem with long-demanded ports and features.  Continuous work in that area should build momentum and yield frequent updates.  imitone is, after all, a relatively small piece of software—it's the technology that takes time.

I’ll be taking some time now to plan my next steps** and do some long-overdue housekeeping on the website.  While I’m not asking your permission to go ahead with the next stage of research, I want to hear your thoughts about it.  I’m also open to feedback about the latest update.  I realize my vision for imitone is only one of many hopes people have for the project, and I couldn’t have come this far without your support.

So ask, comment, request, disagree, criticize; you have nothing to apologize for and neither do I.

And now, it’s time to get back to work.
— Evan


* One exciting discovery I’ve made is a new way to perform statistical measurements on filtered signals, which has applications beyond imitone in the field of digital and analog signal processing.  I may investigate publishing it as open-access research.  To fully generalize the technique, though, I need to solve a complicated frequency-domain integral using computational algebra software. I may hire a mathematician to assist with this work.

** In case you’re curious about the status of my other project, SoundSelf, it is approaching completion but has demanded only a few weeks of my time since February.  In the near future I’ll be fulfilling a few final technical needs, some of which overlap with optimizing imitone and building the VST/AU port. I expect shipping that project to involve about 4 more weeks of work on my end, which I’ll schedule thoughtfully.

*** Poison oak is not fun.  If you find yourself running through poison ivy or poison oak, wash your skin with soap and water within five to ten minutes.  Owww.

0.8.1, The Floor and The Ceiling

Posted by Evan Balster (Creator)

Time passes, progress is made.

imitone 0.8.1 is now available, and introduces a major feature:  hard triggering, which listens for "attacking" (sudden) sounds, like "dah", and makes playing various attacking instruments -- guitar, piano, mallets, or anything that begins with a "bang" -- much more natural.  When power control is used with hard triggering, the initial loudness of the note is used.

Useful settings for playing piano with imitone's new Trigger Mode. The note's attack power appears as a spot on the volume bar.
Useful settings for playing piano with imitone's new Trigger Mode. The note's attack power appears as a spot on the volume bar.

In addition:

  • The vibrato stabilizer is greatly improved.
  • There is a new note mode called "Lock", which prevents all sliding.
  • Pitch tracking has been further improved.
  • CPU usage is reduced by about 33%.

Lastly, imitone now automatically detects certain problems which might make it work poorly, displaying warnings and guidance when these happen.  The most common of these messages, "unclear tone", will appear when your voice is difficult to understand.  It helps you to develop a singing technique for playing imitone.

Here, I'm slipping into "vocal fry", making my tone unclear.
Here, I'm slipping into "vocal fry", making my tone unclear.

This is something I had mixed feelings about:  I've done a lot of work engineering imitone to track most types of vocal sound, in the hopes of making technique unnecessary.  At the same time, I want to make tool that can be learned and mastered.  As it turns out, this tension is well-understood in the world of music.

"The Floor"

This is a term I learned recently, which some musicians use to describe how difficult it is to play an instrument at a basic level.  Consider a piano and a violin:  to play a few notes on the piano, we need only to press the right keys.  But to play those notes on the violin, we need to hold the bow just right, with our fingers pressed against precise, un-marked locations on the neck -- or we could get a horrible, out-of-tune sound.  Unsurprisingly, it takes much longer to get started with a violin than with a piano.

My goal with imitone is, in a sense, to create a new instrument:  one with the lowest floor possible.  An instrument which can be played by someone with no musical experience, and which can be learned so quickly that the player can comfortably improvise or compose after minutes or hours of practice (rather than weeks or months).

To create that instrument, we start with the voice.  We use this sound every day, and it comes naturally to us.  We think about what makes musical singing difficult -- control over pitch and timbre -- and we try to simplify them.  So far, I've worked to make imitone track the pitch of the singer's voice very quickly and accurately, regardless* of timbre.  This is only part of the solution, however:  Those of us who don't have excellent pitch control will struggle.

I realize that even perfect tracking won't fix this problem.  Some of the biggest challenges ahead involve tricky decisions about how notes are chosen as the actual pitch changes.  I described some of these in this update over a year ago, and I've had some interesting new ideas recently.  In any case, today's imitone is far too eager to slip from one pitch to another -- and the "easy" solutions tend to create other problems.

Brief variations in pitch can cause undesired notes to play.  Using a scale helps, but doesn't eliminate the problem.
Brief variations in pitch can cause undesired notes to play. Using a scale helps, but doesn't eliminate the problem.

Solving these problems, and creating a better "sense of control" generally, will be the focus of my work for version 0.8.2.  With these changes, I hope to lower imitone's "floor" substantially.

The Ceiling

This term describes the potential of an instrument in the hands of a player who has mastered it.  Let's compare the violin with the humble recorder.  The violin's nuances give it a high floor, as we discussed before, but also a very high ceiling -- the world's best violin players are famed as "virtuosos".  The recorder, by comparison, is an easy instrument to learn and its low floor makes it a popular tool for introducing children to music.  It is also considered to have a low ceiling due to its "toy" sound -- master players are fewer, and far less popular than great violinists.

The piano is an interesting example:  it has both a low floor and a high ceiling.  This is a rare and desirable quality, and it's one reason that pianos have, in spite of their great size, remained popular over the centuries.

I haven't given a lot of thought to imitone's ceiling, so far.  I've been focused on the floor:  any situation where imitone might misbehave in the hands of an amateur has been, in my eyes, a problem to be fixed.  (A violin-maker would think differently.)  It continues to surprise me when professional musicians -- especially those who are perfectly able to play a keyboard -- show a strong interest in what I'm doing.

It is my sincere hope that some will find imitone, use it to learn, and outgrow it.

That said, it is not my place to limit the potential of this technology.  Perhaps, though I set out to create something like a recorder, the end product might be closer to a piano.  I can't know until I've built it and put it in the hands of people who will explore its possibilities.  To that end, I'll be starting work this week on VST and AU plug-in support for imitone, to make it a more suitable tool for audio professionals.  I'm also eager to explore some new, advanced controls (timbre) which could allow skilled singers to "shape" instrument sounds more deeply.


Getting all this right is difficult.  The idea behind imitone -- controlling music with the voice -- is obvious.  Engineers and scientists far more knowledgeable than I have tried and failed to make it happen, often with the resources of huge companies behind them.  Others seem to have stopped halfway, calling their creations finished simply because they were better than what came before.

For one reason or another, working quietly in my small office in Iowa, I have come further.  Getting here has taken more work than I could have imagined, but I'm not going to stop halfway.  I'm working toward something that needs to exist, and I'll know it when I see it.

As I make my way there, your thoughts and feedback will keep me on track.  I couldn't have taken this path without support from thousands of early adopters, help identifying bugs, and detailed feedback from those of you who have shared your thoughts so far.  (You can contribute at the beta forum, as always.)

It has been a long road, but the tool we've been dreaming about is getting close.  I thank you for your patience, and I give you my assurance:  Something great lies around the corner.

-- Evan