A Platform Technology For Brain Emulaton Synthetic Neuroanatomy Peter AJ van der Made M.Sc. vWISP Pty Ltd. Technology Park E3, suite 11, Bentley pmade@vwisp.net.au Abstract—A computer is a great tool for statistical analysis, in recognition engines of every biomorphic variety, in simulation and number crunching, but their usefulness is autonomous robots and in intelligent toys. It can also be used limited in Artificial Intelligence applications and in the in conjunction with a PC to enable the emulation of brain simulation of biologically accurate neural networks. This is due phenomena that currently require a supercomputer. to the sequential nature of these machines whereby all data has to pass through the central processor in chunks of 16, 32 or 64 Keywords-component; Artificial Intelligence, Neuromorphic bits, depending on the width of the device’s data bus. In systems, Neuromimic systems, Synthetic Neuroanatomy contrast, the brain’s network is massively parallel and processes the equivalent of millions of data bits simultaneously. I. INTRODUCTION Sizable networks of biologically accurate artificial neurons Artificial Intelligence is a division of computer science. require the resources of huge supercomputer systems Compared to the rest of the field, it has seen little success consisting of tens of thousands of processors. Even so, attempts over its 65 year history. All attempts to emulate human level to emulate the entire human brain are far removed from their intelligence, or to recreate it in a digital computer, have goal. They typically emulate several hundreds of thousands failed. Even the question “what is intelligence” cannot be mammalian neurons of varying complexity, verses at least 100 billion in the brain. answered with absolute certainty. A bush man has skills that enable him to survive off the land in a harsh climate. A So called “Artificial Neural Networks (ANNs) have little to business man has skills that allow him to survive despite a do with how the brain actually works. They are no more than downturn in the economy. Both are intelligent, and even data classifiers. The learning time of Multi-layer ANNs with stand out from their peers, but they will struggle to survive in back propagation networks increases exponentially due to the each others‟ world. Intelligence is relative to the method of learning, even when simplified training sets are environment from which it has been formed. appliedi. The human brain is a three pound pinkish organ that is supported in the clear, salty Cerebral Spinal Fluid (CSF). Its A new approach is proposed here, whereby the synthetic wrinkly outer appearance does not convey its complexity. brain is constructed as an information carrier that inserts new The brain consists of an estimated 100 to 200 billion neural information into its network through autonomous learning cells, at least ten times as many glial cells and 100 to 200 from sensory input through feedback and Synaptic Time trillion synapses. These are approximate figures based on the Dependent Plasticity (STDP). The information that is learned neuron count in small parts, and multiplied by the entire in this way is applied to later recognize the recurrence of the volume of the brain, compensating for „white matter‟ that same or similar events. It has been proved that synapses are contains very few neurons. Its performance is poor to memoryii that acquire information through learning. Synapses average in tasks that require the recording and recall of are everywhere in the brain, Therefore memory is everywhere. Every movement, vowel, and shape etcetera has been learned precise details. The density of active components per square and is stored as a set of variables somewhere in synaptic millimeter is greater than that of a microprocessor chip. memory. Learned motor functions are stored in the motor Neural cells can have up to 200,000 inputs, with an average cortex, while learned speech patterns are stored in Broca’s and of 7000 inputs. The interconnections between these cells, the Wernicke’s areas. Each module of the brain has been trained „connectome‟ forms a complex network that is unique to early in life and has formed connections that are appropriate each human being. It is different between identical twins, to its function. A new device has been created that mimics the which is an indication that it is not formed from DNA, but as learning, association and processing techniques of the brain. a result of real-world learning and experiences. Inputs Learned ‘training models’ can be stored and later reused as directly address local memory, but they do not simply recall innate knowledge in new devices and they can be combined to values to be summed. Synaptic memory consists of a form a platform for further learning. This approach has chemical junction that contains a neurotransmitter. At least required a new processing architecture that consists of a fifty different neuro-tranmitters have been identified that massive interconnected network of autonomous learning have persistence values ranging from about a millisecond to nodes, each containing memory, feedback and an integrator several hundred milliseconds. Neurotransmitters can be capable of learning and matching incoming temporal spatial either excitatory or inhibiting. In addition to these there are waveform patterns. The platform technology can be applied in neuro-modulators that affect large groups of neurons across prostheses that communicate with the brain, in safety devices, the entire brain. Neuro-modulators are released into the Neuro-Spinal Fluid. Neurons match patterns of waveforms persistence from a millisecond to several hundred that have previously been learned. The learning process milliseconds. Many different neurotransmitters exhibit causes physiological changes in the connectome. Therefore similar characteristics. Neuromodulators affect large the boundaries between structure acquired through learning groups of neurons simultaneously and are secreted into and innate structure are not clearly defined. The brain is an the cerebrospinal fluid. The nodes are organized in entwined mesh of information carrier and stored knowledge. minicolumns and hypercolumns. Vernon Mountcastle Mathematical ability is poor. Cognitive and recognition described the columnar organization iii of the abilities are advanced and far exceed the capabilities of any somatosensory cortex in his 1957 paper. In his 1978 supercomputer. Copying its processes is a challenge, but the paperiv he elaborated that this columnar organization is realization of this goal appears to be well within our grasp. found everywhere in the cortex. A hypercolumn consists Neural cells have two prime functions; to learn from input of up to 100 minicolumns. Each minicolumn consists of and to match temporal and spatial waveform patterns. In around 80 neurons. A hypercolumn comprises a computer simulations, much effort has been spent on computational unit, able to learn and to match complex matching of incoming data, to the detriment of learning temporal patterns. At least 10,000 digital nodes can be abilities. Any brain emulation that is not capable of learning mapped onto a full-custom ASIC, representing one cannot be realistic. Due to the brain‟s structure and complete hypercolumn. Larger systems will require distribution of neural cells which is determined by DNA, multiple devices, which can be stacked. Brain size alone conditions at birth and nurturing, there is a natural inclination is no indication of intelligence. Parrot brains are to learn certain skills in preference to other skills. The brain approximately 40 grams, compared to a human brain of is an information carrier which creates a mind through 1500 grams and a bull elephant brain of around 5000 learning at many different levels. The brain consists of grams. The abilities of Alex the parrotv were astonishing. nothing but cells and synapses. Every action that takes place Intelligence is related to brain structure and density of in the synthetic brain must therefore be explainable from neurons. neuron physiology. A realistic hardware emulation of the brain, the Synthetic Neuro-anatomy is capable of learning B. Modular Training and matching learned waveform patterns in time (temporal) and in spatial distribution. An intelligent machine will consist of a network of many devices. Rather than programming the machine, the machine learns from sensory input. Training a machine that consists of 100-200 billion neurons would be an impossible task. A. Information carrier Therefore it is possible to train individual hypercolumns for This new approach to Artificial Intelligence is to create a specific task, such as the recognition of sounds, syllables an electronic information carrier that has structure allowing it and words, or visual image recognition consisting of the to be trained, and is capable of acquiring complexity through identification of line segments. This task is then copied into a learning rather than attempting to recreate the complexity of function library, where it can be used to upload the function a fully developed brain and mind by programming. The to larger networks consisting of many hypercolumns. This principal unit of each node is the neuron, which has gives the machine sufficient innate knowledge to proceed to feedback, synaptic memory and a single output that branches learn from subsequent sensory input streams. Increasingly out to connect to thousands of synapses on other neurons. complex learned functionality is copied into the function The function of Glial cells is to synchronize the neural core, library over time. and to clear it when necessary. The synthetic digital node, consisting of the digital equivalent circuit of a neuron, glia and synapses, is structured to; a. Learn from input pulse patterns II. THE PRESENT STATUS OF ARTIFICIAL INTELLIGENCE b. Match temporal / spatial input waveforms to Artificial Intelligence does not exist at this time. The previously learned parameters term is used loosely to describe machines that respond to c. Produce an output pulse when a waveform set has inputs by logic inference. These are cause and effect been recognized machines, which repeat the same algorithms for each set of d. Provide feedback to modify synaptic memory to input variables. They have a synthetic expression of strengthen the response to future occurrences of the knowledge, not knowledge itself, and are unaware of the same or similar temporal / spatial waveform sets. tasks that they are performing. The algorithm fails when an The learning algorithm is a modified Synaptic Time- unexpected event occurs. Strong AI, also called „Artificial Dependent Plasticity model, adapted to respond to General Intelligence‟ or AGI has as an objective to produce a the repetition and intensity of input waveforms. machine that equals or exceeds human intelligence. How this will be accomplished is not clear. The expectation is that this Since neurotransmitters are synthesized in the will be accomplished through the programming of a presynaptic neuron, all connections inherit the same massively fast computer using a as-yet-to-be defined neurotransmitter. Neurotransmitters can be either technology. No mention is made of sensory organs. Without inhibiting or excitatory, and their effects vary in an actual hardware platform, this goal remains as yet unreachable and the subject of intense academic debate. parallel by each node. We will consider the implications of Consider that intelligence is definedvi as: these technological differences one by one; a. The capacity to autonomously acquire knowledge DRAWBACKS OF PROGRAMMING and skills b. To be aware of the self and the environment, to learn Within a programmed system, the programmer needs to from it and to be able to interact with it be aware of all possible conditions that the system will be c. The ability for autonomous adaptability to a new exposed to. This has major disadvantages. Failing to environment recognize one or more situations is likely to lead to a system d. The ability to think, reason and combine knowledge crash or unexpected behavior. Programmed systems that to form new solutions learn are severely limited in the scope of tasks that can be e. The ability to comprehend relationships learned. By eliminating programming altogether, the f. A capacity for abstract thought machine is freed from these limitations. Learning within the g. A capacity to create new ideas, philosophy and art Synthetic Neuro-anatomy occurs in small steps. The first level of learning indoctrinates the machine with basic A Synthetic Intelligence must include at least some, if not all objects, such as sounds or line fragments. The next level of of these features. „Smart‟ is term that is used loosely to learning combines these prime objects into phonetics or describe products that contain a microprocessor and run simple shapes. These „simple shapes‟ would be similar to the programs, such as smart phones. Smart is defined as “having stick people a child draws. At each level of training the a quick mental ability” or as “a sharp pain”. Today‟s „smart objects become more complex. At a higher level associations phones‟ have no mental ability whatsoever. No machine is are built between objects from different sensory devices, capable today of anything approaching intelligence. such as the association between a word and a picture. Learning proceeds beyond the training period. Learned Current „State of the art‟ machines are generally objects can be copied into a library of training sets, available implemented as software on a computer. These machines to be used in untrained devices. This enables the creation of have no or a very limited learning ability. A computer does innate knowledge upon which more intricate applications can not have any awareness, much like a voice recorder has no be built. awareness of a great speech it has recorded. Industrial robots are cause and effect machines that repeat the same actions to Arithmetic Logic Unit (ALU) a number of limited stimuli within a simplified environment. Voice recognition is the most advanced form of direct human The ALU forms the heart of every modern to machine interactions. Its abilities have increased in recent microprocessor. Together with its routing logic it executes a years with the introduction of multiple parallel matching, in stored program by fetching instructions from memory, which a probability factor is calculated for each phonetic fetching data from registers or from memory, performing the match. The machine then programmatically picks the match function and then placing the result back into either memory that has the highest probability before mapping the phonetics or the Accumulator register. ALU‟s are generally from 8 bits to a word. In general, human recognition ability is simulated to 128 bits wide, acting on data 8 to 128 bits at a time. The by the binary comparison of elements in an incoming data major disadvantage of this processing method is that all data stream to known elements representative of an entity. A has to pass through the ALU. Its operation is sequential, with value is calculated from elements that match verses elements one instruction executed after another. This limits the to do not match. If this value is higher than a threshold then amount of data that can be processed at any moment in time. the stream matches or almost matches the entity. This is a The ALU forms a bottleneck in the data stream. Direct selection process and it will never lead to intelligent Memory Access (DMA) techniques bypass the ALU, but it machines. is limited to block moves from one part of memory to another. A synthetic brain does not require an ALU. The brain contains nothing like it. Its mechanisms require no III. ARCHITECTURAL DIFFERENCES arithmetic, but work through the association of incoming There are a considerable number of differences between waveforms with the previously stored (learned) parameters the digital architecture that is proposed here and a „von of a previous event. Neumann‟ stored program computer. The first and most DRAWBACKS OF SEQUENTIAL MEMORY obvious difference is the lack of a stored program. The synthetic brain is trained, not programmed. Programming In a computer all memory is sequential, connected to the would severely limit the ability to evolve intelligence. ALU through a data and address bus. The address bus Further, there is no Arithmetic Logic Unit (ALU). The ALU provides the offset in sequential memory that the ALU is forms the heart of every microprocessor. It performs logic acting upon. The data bus contains data to be read or written. and arithmetic functions. Another major difference is the Having one large memory block causes contention issues; lack of a separate memory block. In a synthetic brain e.g. no other process can access the memory block during the memory is scattered across the device and is accessible in time that one word is read or written. In the Synthetic Neuro- Anatomy architecture memory is distributed. Each synapse contains its own memory, allowing massive parallel access called “Higher Intelligence”, available from the end of of all stored data at the same time. All nodes can be accessed March 2013. in parallel. For example, 70% of nodes can be active within the same millisecond in a cortical column consisting of 10,000 nodes. Each node has several thousand synapses. This represents an equivalent data throughput of REFERENCES (N / t) * S i H Mühlenbein “Limitations of multi-layer perceptron networks - steps 7000 / 0.001 * 7000 towards genetic neural networks” Parallel Computing Volume 14, Issue 3, August 1990, Pages 249–260 Whereby N is the number of active nodes, t is the time ii duration in seconds and S is the number of Synapses per Mark Mayford, Steven A. Siegelbaum, and Eric R. Kandel "Synapses and node. At an average of 7000 synapses per node this Memory Storage", Cold Spring Harbor Perspectives in Biology, June represents a sustainable throughput of nearly 50 Gigabytes of 2012;4:a005751 First published April 10, 2012 data per second per device. A modest synthetic brain will iii consist of thousands of devices. Mountcastle, V.B. (July 1957). "Modality and topographic properties of single neurons of cat's somatic sensory cortex". J. Neurophysiology 20 (4): 408–34. iv Mountcastle, V. B. (1978), "An Organizing Principle for Cerebral IV. REAL-WORLD APPLICATIONS OF INTELLIGENT Function: The Unit Model and the Distributed System", in Gerald M. MACHINES Edelman and Vernon B. Mountcastle, The Mindful Brain, MIT Press v The technology that is discussed here has been tested on Pepperberg, Irene Maxine. The Alex Studies: Cognitive and a small scale component by using Field Programmable Gate Communicative Abilities of Grey Parrots. Harvard Universitiy Press, 2000 Array (FPGA) devices from Xilinx and Actel. The design is vi highly repetitive, with each node an exact replica of every Higher Intelligence, by Peter AJ van der Made. Page 72 other node. It is therefore expected that the small scale design will scale quite well to a component containing at least 10,000 nodes. The connectome for the larger scale device will be the biological model of a cortical column. The circuit‟s learning ability has been verified in an experiment by using a signal generator and spectrum analyzer software as a sound source and artificial cochlea respectively, with the output connected to a synthetic neuro-anatomy consisting of ten nodes in FPGA. The FPGA that was used was a 1.5 million gate ACTEL device on an ACTEL development board. The signal generator frequency was varied to simulate the frequency of common vowels in human speech. The synthetic neuro-anatomy learned to recognize ten sounds that were later also identified in a speech pattern. Learning time was less than 2 minutes. An obvious application for this technology is in speech recognition, speaker recognition, and extraction of speech from a noisy background environment. Other experiments show that the devices can be successfully applied in applications such as visual image recognition, robotics, emulation of the brain on small desktop computers, autonomous learning machines used in exploration, unmanned vehicles, and robotics. The advantages of using a Synthetic Neuro-Anatomy device over a traditional programmed device are a shorter development track, better quality recognition, persistent learning after the initial commission of the system and the reusability of training models. More examples and a more detailed explanation of this technology is available in a new book by Peter van der Made