Project image
)}
$707
pledged of $5,000pledged of $5,000 goal
30
backers
Funding Unsuccessful
The project's funding goal was not reached on Tue, August 7 2018 3:17 PM UTC +00:00
$707
pledged of $5,000pledged of $5,000 goal
30
backers
Funding Unsuccessful
The project's funding goal was not reached on Tue, August 7 2018 3:17 PM UTC +00:00

What is a prototype?

A prototype is a preliminary model of something. Projects that offer physical products need to show backers documentation of a working prototype. This gallery features photos, videos, and other visual documentation that will give backers a sense of what’s been accomplished so far and what’s left to do. Though the development process can vary for each project, these are the stages we typically see:

Proof of Concept

Explorations that test ideas and functionality.

Functional Prototype

Demonstrates the functionality of the final product, but looks different.

Appearance Prototype

Looks like the final product, but is not functional.

Design Prototype

Appearance and function match the final product, but is made with different manufacturing methods.

Production Prototype

Appearance, function, and manufacturing methods match the final product.

62574af69ba224da7fdf4598f0ac7ba2 original.png?ixlib=rb 2.0

Prototype Gallery

These photos and videos provide a detailed look at this project’s development.

About

Hi, thank you so much for looking into my Kickstarter! 

Who Am I: 

I'm a Neuroscience Graduate student at Brandeis University who has recently become extremely fascinated by the field of Machine-Learning, especially Artificial Neural Networks, the most promising form of A.I. for many tasks. I've become passionate about the idea of applying principles of real neurons (such as the ability to switch between firing modes or the ability to adjust sensitivity to input) to the nodes in Long-Short-Term-Memory cells. Long-Short-Term-Memory cells are a form of recurrent neural networks that was, itself, inspired by neuroscience (neurons in the Hippocampus to be precise), so I believe that's the perfect type of network to test my hypothesis on. 

I'm hoping that by applying certain principles of real neurons, to Artificial Neural Networks, I will be able to improve these Networks' accuracy and generalization, thereby demonstrating the need for more research applying the cellular mechanisms of real neurons to Artificial Neural Networks. By the end of the upcoming year of University, with your help, I'm hoping to have completed a Master's Thesis, using this data.

What Am I Doing: 

I'm going to be running a tremendous amount of experiments to ensure that any positive results I get are reliable, and have the proper controls. Preliminary data looks good, with statistical significance all around, but the sample size is small (N=40), and not every necessary control has been tested. It took approximately 2 months of continuous code-running to collect the preliminary data on my laptop, and the full dataset I need to collect is about 150 times as large (8x the sample size, 20x the number of conditions).To add some perspective, it took approximately an hour to train a network once/sample size of 1, and I am aiming now for a sample size of 200, per condition.)

What Is The Exact Hypothesis:  

Akin to how real neurons have the ability to switch between firing mode, and neurotransmitter release, I have coded in a layer of nodes (which could be seen as analogous to epigenetic expression) which choose between (the two most commonly used activation functions), Tanh and ReLU, at each time-step, for each node of the core layer of the LSTM cell. (An activation function tells the neuron the relationship between its input and output, Tanh is a sigmoid activation which translates any input into a value between -1 and 1, ReLU is a linear activation function which translates input into an output between 0 and infinity). Thereby my modifications provide these nodes the capability to switch between two modes of activation. Similar to a neuron switching between a tonic and bursting firing mode.

Additionally, akin to how real neurons can release more or less transmitter in response to the same stimuli, due to factors such as facilitation and depression, I have also added a layer that scales the slope of the activation function of the core layer nodes, at each time-step, for each node. This enables a dynamic sensitivity to input that can be envisioned as the intrinsic excitability of the neuron or axon terminal modulation (facilitation & depression). This alone, and this in combination with the previous condition, both yielded positive preliminary results, and have the potential to become standard practice if further data-collection demonstrates that the preliminary results are reliable.

Why Do I Need The Money:

Because there is so much data collect, the only feasible option appears to be renting a bunch of GPUs on the Amazon or Google Cloud. This happens to be extremely expensive, and since I am not a PhD student, my research is not funded by my University. Therefore, in order to continue my research, my options are to either pay out of pocket, get a grant from Amazon (fingers crossed), or hope that the internet gods smile upon this Kickstarter. 

When running the preliminary data on my laptop with 1 GPU and 4 CPU's, training the network once took about an hour, therefore, a sample size of 1 requires an hour, given 1 GPU. For each experimental condition (of which there are 2), there are about 16 controls, all needing a sample size of 200. I estimated, that in order to shrink the estimated run-time for code down from 3,200 hours (132 days), to 200 hours (9 days), it would require 16 GPU's, which is available for about 500$ a day (4,166$ total). Since I have 2 main conditions I would like to test, this will likely require 8,333$, and then if either condition gets glowing results, I must repeat the process with 2 more sets of training data, in order to ensure reliability across data-sets. This lands me at about 25,000$ for every experimental condition to be fully tested. 

But the amount I set for this Kickstarter should be enough to test the first experimental condition. However if, by way of magic, 25,000$ is actually raised, it will go towards running the full set of experimental conditions on the Cloud. If Elon Musk stumbles upon this Kickstarter and decides to donate more than 25,000$, every extra thousand will go towards reserving more GPU's, such that I can increase the sample size even further, with the ultimate goal of reaching a sample size of 1,000 per condition. Currently, my preliminary sample size of 40 is laughable, with a sample size of 200, my results will be noteworthy, but if 100,000$ is raised, the resulting sample size of 1,000 could make my research actually publishable. 

Future work: 

After this data is collected, I plan to write a master's thesis on it, or ideally, publish this data in a reputable Journal. Additionally, I plan to continue on with another hypothesis, regarding the connection between Artificial Neural Networks' Learning Rate, and the Plasticity of synapses. I believe that the current standard practice of having a set Learning Rate for every node at each Epoch may be analogous to Critical Periods of high plasticity. However, after the initial period of development, the accuracy of Artificial Neural Networks tapers off sharply and requires dramatic decreases to the Learning Rate of the nodes, which too, mimics neural development. What seems, to me, to be missing, is the fact that after early human development, while the human brain is not nearly as plastic as it was, there is still an enormous amount of plasticity which occurs conditionally on a synapse-by-synapse basis due to dynamic factors such as Activity-Dependent Plasticity. If my current project is completed within the first Semester, I would like to expand my model of Neuro-realistic nodes, by enabling node-specific learning rates which follow a set of rules based off certain mechanisms by which synaptic plasticity works in the human brain. 

Risks and challenges

The first risk regards the Cloud computing speed. If the Cloud computing speed is slowly or faster than estimated, than this could cause drastic changes to the what can be achieved with the raised funds. On the optimistic side, I may have underestimated their power, and will therefore be able to increase my sample size. On the pessimistic side, the raised funds may only be able to pay for a smaller sample size than I hoped for.

Additionally, there is always the risk in research, that the results simply come out non-significant. I want this money to be able to test my hypothesis by collecting a tremendous amount of data using many GPU's. But even after collecting the data, there is a significant possibility that my hypothesis will not be supported.
Preliminary data looks very good, but it's only preliminary, and I don't want anyone funding me with the idea that this is a guaranteed way to revolutionize Machine Learning.

If my research comes back positive, than this will be a small step forward in demonstrating the potential of this path in Machine-Learning research. But if the research comes back with poor results, I will have to move on to subsequent hypotheses. While I already have a hypothesis to test next (applying the principle of synpatic plasticity to the learning rate of nodes), it has not yet been coded and it too could fail.

I heard an amazing quote about research once. Research is about searching, and you're going to fail, alot, that is why it's called re-search.

Learn about accountability on Kickstarter

Questions about this project? Check out the FAQ

Support

  1. Select this reward

    Pledge US$ 20 or more About US$ 20

    Pity Package

    You pity my sad situation of having a scientific hypothesis, but a lack of resources to test it.

    Thank you!

    Less
    Estimated delivery
    2 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  2. Select this reward

    Pledge US$ 50 or more About US$ 50

    Glad To Help

    This was a triumph, I'm making a note here, huge success.

    Less
    Estimated delivery
    2 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  3. Select this reward

    Pledge US$ 200 or more About US$ 200

    Direct email updates on research results

    If interested, and you can provide an email, I will send you details about whether the experimental results are coming back statistically significant or not.

    Less
    Estimated delivery
    0 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  4. Select this reward

    Pledge US$ 500 or more About US$ 500

    The Enlightenment

    If interested, I will send you via email, constant updates on every step of the process. This includes both the good news, and the unexpected obstacles.

    If this gets annoying, it will cost another $500 to get me to stop.

    Less
    Estimated delivery
    1 backer
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  5. Select this reward

    Pledge US$ 2,000 or more About US$ 2,000

    Elon Musk

    If interested, I will personally tweet at Elon Musk and tell him that an amazing person pledged 2,000 dollars!

    Less
    Estimated delivery
    0 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.
  6. Select this reward

    Pledge US$ 5,000 or more About US$ 5,000

    God-Mode

    If interested, I will unlock the God-Mode cheat and you will become invincible.

    Less
    Estimated delivery
    0 backers
    $
    Kickstarter is not a store.

    It's a way to bring creative projects to life.

    Learn more about accountability.

Funding period

- (30 days)