Neural Nets
I've been doing a lot of reading lately about Neural Nets and the different learning processes they use. Neural Nets are rather crude artificial representations of the human brain that try to simulate the learning process. Our brains are made up of billions of tiny neurons that can connect to thousands of other neurons by emitting electrochemical signals. Signals are received along synapses, which can be attached or detached to the dendrites of other neurons. When a neuron receives an input electrical signal, the neuron processes the information and determines whether to move it forward as an output to other neurons. There are a couple great tutorials on the concepts behind neural nets at AI-Junkie.com and Jochen Frohlich.
I have a couple problems with Neural Nets. The first being that they are a serial simulation of parallel processing. Our brains are not very fast in the electrical, computational sense, but they are very efficient parallel processors. Each cycle of our brain function processes millions of instructions, whereas today's CPU's only process a single instruction per cycle. So trying to equate the processing architecture of our brains to a CPU is a stretch. IBM has taken the approach of actually trying to replicate the parallel processing capabilities of a rat's brain using the Blue Gene L supercomputer. Mouse brain simulated on computer
Second, neural nets are restricted to binary inputs, whereas our brains are capable of taking many types of inputs such as the senses, touch, sight, smell, sound, and taste. Our brains also act differently depending on hormone balances, nutrition, and what mood we happen to be in. All of these factors alter how the neurons in our brain fire and transmit information.
Third, and most important is memory. It is still a mystery how we remember certain events, people that we meet, or experiences. It is theorized that our memories are nothing more than dynamic connections between neurons that occur during a particular experience. As the memory fades and becomes less used, the connections across neurons degenerate, and eventually break, causing the memory to lapse. It is also theorized that certain parts of our brain are highly tuned to manage these connections between neurons and index these memories using other neurons, memory neurons.
Neural nets simply do not have the ability to record a pleasant experience, or recognize an old friend in a crowd who has aged or changed their hair style. Our memories and sub conscience are truly unique in this regard. We have movie memory, or what some refer to as holographic memory. We can actually re-run events in our brain as if they were occurring for a second time. We can even make up events in our brains that never occurred, and we can dream.
I have an idea for improving the architecture of neural nets. The parallel processing aspects of our brains are not going to be equaled by CPU's anytime soon, if ever. We need a way then to leverage the serial nature of a CPU, and still perform the mass of computations required to simulate billions of neurons. My thought for this approach is to make smarter neurons. Instead of having neurons just take inputs, do some magic, and provide an output, what the neurons were given more options, say to record memory, dynamically associated or disassociate with other neurons, and to provide negative as well as positive responses to input. Neurons should also have a weight for application to the senses, and the ability to prioritize work based on the state of the senses, say the 'hunger' sense is abnormally high, other senses would be diminished so that the organism could focus.
