On Intelligence - A discussion

I recently had the pleasure of spending some vacation time with my very good friend Dr. Michael Nussdorfer.  Mike is the Medical Director for Radiology at Barton Memorial Hospital in South Lake Tahoe, California.  He is an expert on the anatomy of the human brain, diagnosing brain injuries, sleep disorders, and neurology in general.  He is also a technology buff like myself, who shares some of my interests in expanding our understanding of neural processes.

Mike and I typically measure the quality of our vacations by the number of books we read.  We often bring each other books we have recently read so we can discuss the ideas and predictions.  Fiction is rare, as we usually stick to science or history subjects.  For this trip I brought along Jeff Hawkins' On Intelligence to share and get his opinion on some of the ideas presented.  The discussion made me realize some of the underlying challenges in trying to mimic the neural processes of the human brain.

I think anybody who reads this book will quickly get caught up in Hawkins' posit regarding neural feedback.  The book claims that the majority of our neural pathways are actually wired for feedback, as opposed to pathways from inputs leading to deeper neural layers.  Mike was caught off guard for this one as well.  And we both expressed wonder for how this must work.  I was left with the curiosity of how feedback was used in neural cell. 

In simulated neural cells, such as those used on back propagation neural networks, feedback values are used to adjust connection weights and retest possible outputs.  But I dont think its necessarily clear how this information is used in live neural cells.  My thinking is related to another idea discussed by Hawkins, neural cell proximity strength.

Neural cells fire stronger electrical signals to neural cells that are closer to them.  In other words, the longer the dendrite, the weaker the signal.  So neural cells have more influence on the cells in their immediate area, which strengthens the concepts of neural region organization.  But if this is true for inputs signal strength, then it must also be true for feedback signals.  Therefore feedback has increased influence based on proximity, todays neural networks typically do not accommodate this behavior.

Another area we discussed was synchronization and neural cycles.  The brain is amazing in the way it handles so many inputs and feedback at once.  This is an incredible amount of information flow, so much so that we were both left wondering how this works.  I read while listening to my iPod.  My feet beat to the music, while my eyes scan the words.  Sometimes I find myself reading in time with the music.  So this begs the question about our neural cells.  Are neural cells serial, or do they maintain a level of parallelism?  Can a neural cell work on more than one thought at once?

Todays neural networks are serial of course, meaning that a simulated neural cells can only handle one set of inputs, and the corresponding feedback, one value at a time.  But think about how live neural cells may work.  Inputs come in while feedback is coming back.  At some point there is some crossover.  How do neural cells distinguish between cycles, and associate its information with a particular request?  And do they reset after use? Or do they retain information from previous requests?

For the questions the book answers, its raises many more.  The synchronization problem is a big one.  And I believe our simplistic view of neural cell design will prove to be an inhibitor to designing intelligent systems.  Hawkins and his cohorts have produced a development environment around some of these ideas.  Numenta is the platform for intelligent computing, give it a try if you'd like to take some of Hawkins ideas to the next level.