Share this project

Done

Share this project

Done
Compose music with your voice, or any instrument, live.  Works with all your favorite music software.
imitone lets you play any instrument with your voice.
imitone lets you play any instrument with your voice.
2,433 backers pledged $90,517 to help bring this project to life.

imitone prime

18 likes

65 hours remain!

Given the lack of specific information available about imitone prime, its popularity during this campaign has been surprising -- it accounts for almost half the funding raised for the project!  Today, I'll be addressing that informational gap as best I can.

First, its purpose:  "imitone" is designed to be simple, accessible and powerful. Notice I don't call it "imitone basic" or "imitone lite".  That's because it's my core product.  For a majority of people, it will be everything they need and nothing they don't to bring their ideas to life.  However, the technology offers a depth of possibilities that goes further than that, and a significant minority of users will be inclined to explore and utilize those.  imitone prime is designed with professionals, performers and tinkerers in mind:  The proverbial toolbox to imitone's swiss army knife.

This article will get somewhat technical from here on in -- imitone prime is an advanced piece of software, with many esoteric features.  So!  Let's examine each in detail:

Timbre control

"Timbre" is an umbrella term for all the features of a sound aside from its pitch and intensity.  These are typically difficult to quantify and easy to qualify:  The difference between an "aah" and an "ooh" is easy to recognize, but hard to express in mathematical terms.

Thus far I have implemented one timbre metric in imitome prime:  brightness, meaning the "harshness" or "openness" of a sound for a given pitch.  As a general rule, "hmmm" is a dark sound and "aah" is a bright sound.  Your throat produces a bright sound -- aah -- when you vocalize, and your mouth and lips filter that sound and darken it based on their shape.  Nasal, gritty or harsh vocal sounds are naturally brighter than others.

prime's brightness control lets you map this metric (the tone's "spectral centroid") to a MIDI message of your choice.  By default, it outputs to CC74, which most often controls a filter on the receiving synth.  By shifting your voice between vowel sounds, you can control this filter as you sing.

A quick demonstration:

Many other timbre metrics are possible, and if I should have any good ideas you can expect to see them in prime.  I'm currently contemplating a harmonicity control -- it would measure transitions between "breathy" and "clear" sounds, which could be used as a dry/wet fader.  Since harmonicity is used to select notes, such a controller might continue to change even when no note is active.

Multi-channel control

MIDI allows for transmission on up to 16 channels.  Whereas imitone is designed to control a single instrument, prime will be able to control up to 16 per output device, with each of these mapped to a different audio input.

Each active channel is set to a "mode" -- tone or beat.  Each may have different settings for its detection and MIDI sequencing, as described in "advanced configuration" below.  Sharing a single audio input between channels will be supported.

A few examples:  You might have a single microphone set to detect tones on channel 1 and beats on channel 2, with the intention of alternately humming and beat-boxing.  Or you might get a few friends together and use several microphones, each controlling a different instrument, to form a string quartet or a ska band.  (This can be done as a performance if you use contact microphones.)

Advanced Configuration

imitone prime will offer many, many more adjustable settings than imitone, most of which will be individually manipulable for each MIDI output channel.  These break down into two categories:

Sequencing settings build on the switches offered in imitone.  You will be able to configure its continuous controls (expression, vibrato and brightness) to different CC numbers or aftertouch controls.  CCs may be coarse or fine, and aftertouch may be monophonic or polyphonic (in which case controls may be sent separately for each note).  Pitch bend and glide will be somewhat more customizable, permitting a choice between various legato / portamento methods and the adjustment of the pitch-bend range.

A feature which has been requested several times is the ability to turn off certain MIDI features -- for instance, to turn off musical notes and use imitone exclusively as a way to control expression and vibrato while playing a conventional keyboard.  I have decided to add support for this, and will likely add further settings based on user suggestions.

Analysis settings allow fine-tuning of imitone's internal algorithms, allowing them to be "tweaked" for certain uses.  These include detection range, tuning scheme, resonator cell parameters, and harmonicity thresholds for note selection.  Higher-level algorithms like vibrato stabilization will also be adjustable.

Some settings (in both categories) may need to be configured by editing a text-file, especially in early and beta versions of imitone prime.  It will be possible to save and load configurations per-channel.

Polyphonics

imitone prime will have an experimental "poly mode" where it will be capable of detecting more than one note simultaneously from a single input.  This means you could have multiple people harmonizing into one microphone, or detect a double-stop on a string instrument.

An early proof-of-concept.
An early proof-of-concept.

While my technology lends itself well to this problem, it's a notoriously difficult one to solve well (especially for larger numbers of notes) and I need to do more research before I can make claims about its effectiveness.

Noise Cancellation

This feature was originally designed to enable performance with imitone as part of a live band.  It requires two microphones -- a signal and a reference.  The signal mic is placed near the tone source and the reference mic is placed in the environment where it gets similar levels of ambient sound but much less of the controlling voice.  By comparing the two, the desired tone can be isolated.

A throat microphone designed for use with two-way radios.
A throat microphone designed for use with two-way radios.

However, since beginning work on this feature, I learned about a much more efficient alternative:  the throat microphone.  Wearable on the neck and costing as little as $15, this device isolates far better than any algorithm and requires less set-up and equipment to use.  For this reason, I'm considering cancelling this feature or otherwise making it a very low priority.  Let me know what you think in the comments.

Data Visualization

While working on this technology, I've developed various charts and graphs to give me insight into its internal algorithms and processes as they operate on my voice.  Removing these from imitone for the sake of simplicity was the right choice, but I was a little heartbroken about it.  I plan to sneak my favorites back into imitone prime as helpful displays -- but not before letting imitone's designer Richard give them a face-lift.

Waveform (below left) lets you see the wave-shape of your voice, which holds stable for any given sound and changes as you alter your timbre, pitch or loudness.  This information is used to compute brightness and harmonicity.

Fourier (above right) shows you an FFT spectrum of the waveform, letting you view the distribution of your overtones.  Yes, I know I was laying the hate on FFTs last week, but remember my remark:  The fourier transform takes a cyclic signal, and a periodic waveform is one.  Furthermore, I still don't use any transforms in the actual detection algorithms -- this is just a handy way to examine the data.  For instance, the sound above is throat singing, which produces a characteristic "lump" of overtones high in the spectrum.

Lastly, the spectrum is close to my heart.  This colorful artifact offers an insight into the core logic of imitone, evolving in tandem with the technology.  I can't really tell you what it will look like, because the data will continue to change.  But darn if it isn't a mesmerizing thing -- especially in motion.

--

And that's imitone prime in a nutshell.  At least, as far as I've planned -- I might make additional features or discoveries that find their way in.  Perhaps I've convinced you that you need it, or that you don't need it!  In any case, I hope you come away from this feeling a bit more informed in your pledge decision.  imitone prime's final cost will likely be around $99, and the $60 pledge level includes both versions of the program.

That's all for now --
-- Evan

[tweet this update]  [share on facebook]

Cookie Gawain, Craig Wood, and 16 more people like this update.

Comments

    1. Evan Balster 2-time creator on April 8, 2014

      Guy -- Auto harmonization, if I'm not mistaken, is already available in several DAWs and plug-ins. Because imitone pretends to be a MIDI controller, it can talk to all of these and your request is already possible!

      Bruno -- The VST stretch goal was already passed, as per update 11 -- so there will be a plug-in version!

      James -- throat mics offer excellent sound isolation, but tend to result in strange-sounding vocal recordings. Imagine putting your ear against someone's neck as they talk -- that's what it's like. There are also lots of poor-quality throat mics out there -- imitone does a great job with shawdy microphones, but most other uses don't.

    2. Missing avatar

      Guy Funnell on April 8, 2014

      I know this is not your current concern but would it be possible one day to add Auto harmonisation abilities to the software so that it harmonises as you sing?
      this would be fantastic.

      Guy Funnell Uk

    3. Bruno Spoerri on April 8, 2014

      I will use imitate mostly with my saxophone and in conjunction with Max/MSP-patches. So my question: I hate to have 2 applications open at the same time - could you provide either a MAX object or a VST, that could be used in Max? Thanks Bruno Spoerri

    4. Missing avatar

      Lumir Hrma on April 8, 2014

      Thank you Evan, pledge adjusted accordingly.

    5. Missing avatar

      James Wolk on April 8, 2014

      Hope this isn't too off topic...just wondering if a throat mic might be used effectively for recording vocals in general as a work-around for recording in a non-soundproof room...(with or without imitone)...
      I concur with the writing style. Excellent. Only found one mistake so far.

    6. Missing avatar

      steve pratt on April 8, 2014

      This all sounds great and I must compliment you on your written style. You communicate your ideas really well. I already pledged to get prime and im really excited about its possibilities. I cant wait to get using it.

    7. Evan Balster 2-time creator on April 8, 2014

      Lumir -- Too early to say about an upgrade path -- you can adjust a Kickstarter pledge before the project ends, or contact me about adjusting a PayPal pledge. I might consider custom CC mapping as a config-file feature in imitone.

      On noise cancellation: My reasoning is that it already would have required a special equipment set-up -- two identical microphones, which might need to be hooked up through the same audio interface to avoid a latency hit. A contact mic probably works out to be a cheaper and easier set-up by comparison.

      梁德华: In the long run I would be very interested in bundling imitone with some kind of microphone, or even make a hardware version, but I would need to pair with a really solid product and I haven't yet found one I'm willing to endorse.

    8. Franklin Webber on April 7, 2014

      I backed Prime because I wanted to make sure that the project reached its higher level stretch goals. I also am sure that I will want to play with some of the advanced features.

    9. 梁德华 on April 7, 2014

      If you are going to drop the noise cancellation feature, I would probably suggest you might consider providing a throat mic with it? I'm pretty sure there are people out there whom might still be interested in the noise cancellation part.

    10. Missing avatar

      James Wolk on April 7, 2014

      I stumbled on prime via KVR topic and sure am glad I did. Looking forward to seeing/hearing any and all features that come about during the development cycle(s). Thanks for including "... turn off musical notes and use imitone exclusively as a way to control expression and vibrato while playing a conventional keyboard." That feature has me intrigued further.

    11. Missing avatar

      Lumir Hrma on April 7, 2014

      Ok, after reading this - will there be an upgrade path? I thought custom CC mapping would be in the basic version - thus backed that one, but now I want prime - what can I do?
      And again, cograts Evan to this masterpiece!

    12. Missing avatar

      Jin
      Superbacker
      on April 7, 2014

      Noise cancelation doesn't seem like a particularly interesting feature to me. When using non-voice inputs, the use of a purpose-built controller or contact mic/pickup seems like a more than adequate solution. Rather than attempting to broadly apply imitone to the spectrum of instruments out there, I'd like to see more focus on the human voice, which aside from being the most expressive instrument we have, is also available to the overwhelming majority of the population. I thinksuch a focus would also enhance the appeal of imitone.

      In a studio environment with proper gain staging, I don't see extraneous noise as an issue. For live performances, I think the throat mic is the simplest and best solution. However, I'm worried about non-ideal record situations, such as multiple singers on one mic (the polyphony issue above) or multiple singers on multiple mics in a less-than-ideal recording environment. This is a low priority and I'm not completely familiar with how imitone works, but would it make any sense to optimize pitch detection to cancel out bleed from other inputs? Similar, although not quite the same problem as before.

      As you might gather from my above comments, sequencing and multi-channel options are pretty important to me. The analysis certainly looks interesting, although I'm not sure how I'd use it myself. I'm pretty used to FFTs and other spectral mappers like iZotope makes. Either way, I'm excited to see where this goes!

    13. Missing avatar

      Adam S. Crane on April 7, 2014

      Noise cancellation and polyphony are the major prime features that interest me. Noise cancellation would honestly be most useful for things that *aren't* a human voice. You decide you want to integrate with another band and you're not sure what you're going to need to mic, no problem. With noise cancellation you are ready for any environment.

      Also I think people like prime because it's a way to donate more money while still getting something out of it — most of us, I'm assuming, just want to hit the mobile app stretch goal (while also being generous). That is my #1 concern with this campaign.