=Paper=
{{Paper
|id=Vol-1348/maics2013_paper_7
|storemode=property
|title=A Platform Technology For Brain Emulation
|pdfUrl=https://ceur-ws.org/Vol-1348/maics2013_paper_7.pdf
|volume=Vol-1348
|dblpUrl=https://dblp.org/rec/conf/maics/Made13
}}
==A Platform Technology For Brain Emulation==
https://ceur-ws.org/Vol-1348/maics2013_paper_7.pdf
A Platform Technology For Brain Emulaton
Synthetic Neuroanatomy
Peter AJ van der Made M.Sc.
vWISP Pty Ltd.
Technology Park E3, suite 11, Bentley
pmade@vwisp.net.au
Abstract—A computer is a great tool for statistical analysis, in recognition engines of every biomorphic variety, in
simulation and number crunching, but their usefulness is autonomous robots and in intelligent toys. It can also be used
limited in Artificial Intelligence applications and in the in conjunction with a PC to enable the emulation of brain
simulation of biologically accurate neural networks. This is due phenomena that currently require a supercomputer.
to the sequential nature of these machines whereby all data has
to pass through the central processor in chunks of 16, 32 or 64 Keywords-component; Artificial Intelligence, Neuromorphic
bits, depending on the width of the device’s data bus. In systems, Neuromimic systems, Synthetic Neuroanatomy
contrast, the brain’s network is massively parallel and
processes the equivalent of millions of data bits simultaneously. I. INTRODUCTION
Sizable networks of biologically accurate artificial neurons Artificial Intelligence is a division of computer science.
require the resources of huge supercomputer systems Compared to the rest of the field, it has seen little success
consisting of tens of thousands of processors. Even so, attempts over its 65 year history. All attempts to emulate human level
to emulate the entire human brain are far removed from their intelligence, or to recreate it in a digital computer, have
goal. They typically emulate several hundreds of thousands
failed. Even the question “what is intelligence” cannot be
mammalian neurons of varying complexity, verses at least 100
billion in the brain.
answered with absolute certainty. A bush man has skills that
enable him to survive off the land in a harsh climate. A
So called “Artificial Neural Networks (ANNs) have little to business man has skills that allow him to survive despite a
do with how the brain actually works. They are no more than downturn in the economy. Both are intelligent, and even
data classifiers. The learning time of Multi-layer ANNs with stand out from their peers, but they will struggle to survive in
back propagation networks increases exponentially due to the each others‟ world. Intelligence is relative to the
method of learning, even when simplified training sets are environment from which it has been formed.
appliedi. The human brain is a three pound pinkish organ that is
supported in the clear, salty Cerebral Spinal Fluid (CSF). Its
A new approach is proposed here, whereby the synthetic wrinkly outer appearance does not convey its complexity.
brain is constructed as an information carrier that inserts new The brain consists of an estimated 100 to 200 billion neural
information into its network through autonomous learning cells, at least ten times as many glial cells and 100 to 200
from sensory input through feedback and Synaptic Time trillion synapses. These are approximate figures based on the
Dependent Plasticity (STDP). The information that is learned neuron count in small parts, and multiplied by the entire
in this way is applied to later recognize the recurrence of the volume of the brain, compensating for „white matter‟ that
same or similar events. It has been proved that synapses are
contains very few neurons. Its performance is poor to
memoryii that acquire information through learning. Synapses
average in tasks that require the recording and recall of
are everywhere in the brain, Therefore memory is everywhere.
Every movement, vowel, and shape etcetera has been learned
precise details. The density of active components per square
and is stored as a set of variables somewhere in synaptic millimeter is greater than that of a microprocessor chip.
memory. Learned motor functions are stored in the motor Neural cells can have up to 200,000 inputs, with an average
cortex, while learned speech patterns are stored in Broca’s and of 7000 inputs. The interconnections between these cells, the
Wernicke’s areas. Each module of the brain has been trained „connectome‟ forms a complex network that is unique to
early in life and has formed connections that are appropriate each human being. It is different between identical twins,
to its function. A new device has been created that mimics the which is an indication that it is not formed from DNA, but as
learning, association and processing techniques of the brain. a result of real-world learning and experiences. Inputs
Learned ‘training models’ can be stored and later reused as directly address local memory, but they do not simply recall
innate knowledge in new devices and they can be combined to values to be summed. Synaptic memory consists of a
form a platform for further learning. This approach has chemical junction that contains a neurotransmitter. At least
required a new processing architecture that consists of a fifty different neuro-tranmitters have been identified that
massive interconnected network of autonomous learning have persistence values ranging from about a millisecond to
nodes, each containing memory, feedback and an integrator several hundred milliseconds. Neurotransmitters can be
capable of learning and matching incoming temporal spatial either excitatory or inhibiting. In addition to these there are
waveform patterns. The platform technology can be applied in neuro-modulators that affect large groups of neurons across
prostheses that communicate with the brain, in safety devices, the entire brain. Neuro-modulators are released into the
Neuro-Spinal Fluid. Neurons match patterns of waveforms persistence from a millisecond to several hundred
that have previously been learned. The learning process milliseconds. Many different neurotransmitters exhibit
causes physiological changes in the connectome. Therefore similar characteristics. Neuromodulators affect large
the boundaries between structure acquired through learning groups of neurons simultaneously and are secreted into
and innate structure are not clearly defined. The brain is an the cerebrospinal fluid. The nodes are organized in
entwined mesh of information carrier and stored knowledge. minicolumns and hypercolumns. Vernon Mountcastle
Mathematical ability is poor. Cognitive and recognition described the columnar organization iii of the
abilities are advanced and far exceed the capabilities of any somatosensory cortex in his 1957 paper. In his 1978
supercomputer. Copying its processes is a challenge, but the paperiv he elaborated that this columnar organization is
realization of this goal appears to be well within our grasp. found everywhere in the cortex. A hypercolumn consists
Neural cells have two prime functions; to learn from input of up to 100 minicolumns. Each minicolumn consists of
and to match temporal and spatial waveform patterns. In around 80 neurons. A hypercolumn comprises a
computer simulations, much effort has been spent on computational unit, able to learn and to match complex
matching of incoming data, to the detriment of learning temporal patterns. At least 10,000 digital nodes can be
abilities. Any brain emulation that is not capable of learning mapped onto a full-custom ASIC, representing one
cannot be realistic. Due to the brain‟s structure and complete hypercolumn. Larger systems will require
distribution of neural cells which is determined by DNA, multiple devices, which can be stacked. Brain size alone
conditions at birth and nurturing, there is a natural inclination is no indication of intelligence. Parrot brains are
to learn certain skills in preference to other skills. The brain approximately 40 grams, compared to a human brain of
is an information carrier which creates a mind through 1500 grams and a bull elephant brain of around 5000
learning at many different levels. The brain consists of grams. The abilities of Alex the parrotv were astonishing.
nothing but cells and synapses. Every action that takes place Intelligence is related to brain structure and density of
in the synthetic brain must therefore be explainable from neurons.
neuron physiology. A realistic hardware emulation of the
brain, the Synthetic Neuro-anatomy is capable of learning B. Modular Training
and matching learned waveform patterns in time (temporal)
and in spatial distribution. An intelligent machine will consist of a network of many
devices. Rather than programming the machine, the machine
learns from sensory input. Training a machine that consists
of 100-200 billion neurons would be an impossible task.
A. Information carrier Therefore it is possible to train individual hypercolumns for
This new approach to Artificial Intelligence is to create a specific task, such as the recognition of sounds, syllables
an electronic information carrier that has structure allowing it and words, or visual image recognition consisting of the
to be trained, and is capable of acquiring complexity through identification of line segments. This task is then copied into a
learning rather than attempting to recreate the complexity of function library, where it can be used to upload the function
a fully developed brain and mind by programming. The to larger networks consisting of many hypercolumns. This
principal unit of each node is the neuron, which has gives the machine sufficient innate knowledge to proceed to
feedback, synaptic memory and a single output that branches learn from subsequent sensory input streams. Increasingly
out to connect to thousands of synapses on other neurons. complex learned functionality is copied into the function
The function of Glial cells is to synchronize the neural core, library over time.
and to clear it when necessary. The synthetic digital node,
consisting of the digital equivalent circuit of a neuron, glia
and synapses, is structured to;
a. Learn from input pulse patterns II. THE PRESENT STATUS OF ARTIFICIAL INTELLIGENCE
b. Match temporal / spatial input waveforms to Artificial Intelligence does not exist at this time. The
previously learned parameters term is used loosely to describe machines that respond to
c. Produce an output pulse when a waveform set has inputs by logic inference. These are cause and effect
been recognized machines, which repeat the same algorithms for each set of
d. Provide feedback to modify synaptic memory to input variables. They have a synthetic expression of
strengthen the response to future occurrences of the knowledge, not knowledge itself, and are unaware of the
same or similar temporal / spatial waveform sets. tasks that they are performing. The algorithm fails when an
The learning algorithm is a modified Synaptic Time- unexpected event occurs. Strong AI, also called „Artificial
Dependent Plasticity model, adapted to respond to General Intelligence‟ or AGI has as an objective to produce a
the repetition and intensity of input waveforms. machine that equals or exceeds human intelligence. How this
will be accomplished is not clear. The expectation is that this
Since neurotransmitters are synthesized in the will be accomplished through the programming of a
presynaptic neuron, all connections inherit the same massively fast computer using a as-yet-to-be defined
neurotransmitter. Neurotransmitters can be either technology. No mention is made of sensory organs. Without
inhibiting or excitatory, and their effects vary in an actual hardware platform, this goal remains as yet
unreachable and the subject of intense academic debate. parallel by each node. We will consider the implications of
Consider that intelligence is definedvi as: these technological differences one by one;
a. The capacity to autonomously acquire knowledge DRAWBACKS OF PROGRAMMING
and skills
b. To be aware of the self and the environment, to learn Within a programmed system, the programmer needs to
from it and to be able to interact with it be aware of all possible conditions that the system will be
c. The ability for autonomous adaptability to a new exposed to. This has major disadvantages. Failing to
environment recognize one or more situations is likely to lead to a system
d. The ability to think, reason and combine knowledge crash or unexpected behavior. Programmed systems that
to form new solutions learn are severely limited in the scope of tasks that can be
e. The ability to comprehend relationships learned. By eliminating programming altogether, the
f. A capacity for abstract thought machine is freed from these limitations. Learning within the
g. A capacity to create new ideas, philosophy and art Synthetic Neuro-anatomy occurs in small steps. The first
level of learning indoctrinates the machine with basic
A Synthetic Intelligence must include at least some, if not all objects, such as sounds or line fragments. The next level of
of these features. „Smart‟ is term that is used loosely to learning combines these prime objects into phonetics or
describe products that contain a microprocessor and run simple shapes. These „simple shapes‟ would be similar to the
programs, such as smart phones. Smart is defined as “having stick people a child draws. At each level of training the
a quick mental ability” or as “a sharp pain”. Today‟s „smart objects become more complex. At a higher level associations
phones‟ have no mental ability whatsoever. No machine is are built between objects from different sensory devices,
capable today of anything approaching intelligence. such as the association between a word and a picture.
Learning proceeds beyond the training period. Learned
Current „State of the art‟ machines are generally objects can be copied into a library of training sets, available
implemented as software on a computer. These machines to be used in untrained devices. This enables the creation of
have no or a very limited learning ability. A computer does innate knowledge upon which more intricate applications can
not have any awareness, much like a voice recorder has no be built.
awareness of a great speech it has recorded. Industrial robots
are cause and effect machines that repeat the same actions to Arithmetic Logic Unit (ALU)
a number of limited stimuli within a simplified environment.
Voice recognition is the most advanced form of direct human The ALU forms the heart of every modern
to machine interactions. Its abilities have increased in recent microprocessor. Together with its routing logic it executes a
years with the introduction of multiple parallel matching, in stored program by fetching instructions from memory,
which a probability factor is calculated for each phonetic fetching data from registers or from memory, performing the
match. The machine then programmatically picks the match function and then placing the result back into either memory
that has the highest probability before mapping the phonetics or the Accumulator register. ALU‟s are generally from 8 bits
to a word. In general, human recognition ability is simulated to 128 bits wide, acting on data 8 to 128 bits at a time. The
by the binary comparison of elements in an incoming data major disadvantage of this processing method is that all data
stream to known elements representative of an entity. A has to pass through the ALU. Its operation is sequential, with
value is calculated from elements that match verses elements one instruction executed after another. This limits the
to do not match. If this value is higher than a threshold then amount of data that can be processed at any moment in time.
the stream matches or almost matches the entity. This is a The ALU forms a bottleneck in the data stream. Direct
selection process and it will never lead to intelligent Memory Access (DMA) techniques bypass the ALU, but it
machines. is limited to block moves from one part of memory to
another. A synthetic brain does not require an ALU. The
brain contains nothing like it. Its mechanisms require no
III. ARCHITECTURAL DIFFERENCES arithmetic, but work through the association of incoming
There are a considerable number of differences between waveforms with the previously stored (learned) parameters
the digital architecture that is proposed here and a „von of a previous event.
Neumann‟ stored program computer. The first and most DRAWBACKS OF SEQUENTIAL MEMORY
obvious difference is the lack of a stored program. The
synthetic brain is trained, not programmed. Programming In a computer all memory is sequential, connected to the
would severely limit the ability to evolve intelligence. ALU through a data and address bus. The address bus
Further, there is no Arithmetic Logic Unit (ALU). The ALU provides the offset in sequential memory that the ALU is
forms the heart of every microprocessor. It performs logic acting upon. The data bus contains data to be read or written.
and arithmetic functions. Another major difference is the Having one large memory block causes contention issues;
lack of a separate memory block. In a synthetic brain e.g. no other process can access the memory block during the
memory is scattered across the device and is accessible in time that one word is read or written. In the Synthetic Neuro-
Anatomy architecture memory is distributed. Each synapse
contains its own memory, allowing massive parallel access called “Higher Intelligence”, available from the end of
of all stored data at the same time. All nodes can be accessed March 2013.
in parallel. For example, 70% of nodes can be active within
the same millisecond in a cortical column consisting of
10,000 nodes. Each node has several thousand synapses.
This represents an equivalent data throughput of
REFERENCES
(N / t) * S
i
H Mühlenbein “Limitations of multi-layer perceptron networks - steps
7000 / 0.001 * 7000 towards genetic neural networks” Parallel Computing
Volume 14, Issue 3, August 1990, Pages 249–260
Whereby N is the number of active nodes, t is the time
ii
duration in seconds and S is the number of Synapses per Mark Mayford, Steven A. Siegelbaum, and Eric R. Kandel "Synapses and
node. At an average of 7000 synapses per node this Memory Storage", Cold Spring Harbor Perspectives in Biology, June
represents a sustainable throughput of nearly 50 Gigabytes of 2012;4:a005751 First published April 10, 2012
data per second per device. A modest synthetic brain will
iii
consist of thousands of devices. Mountcastle, V.B. (July 1957). "Modality and topographic properties of
single neurons of cat's somatic sensory cortex". J. Neurophysiology 20 (4):
408–34.
iv
Mountcastle, V. B. (1978), "An Organizing Principle for Cerebral
IV. REAL-WORLD APPLICATIONS OF INTELLIGENT
Function: The Unit Model and the Distributed System", in Gerald M.
MACHINES Edelman and Vernon B. Mountcastle, The Mindful Brain, MIT Press
v
The technology that is discussed here has been tested on Pepperberg, Irene Maxine. The Alex Studies: Cognitive and
a small scale component by using Field Programmable Gate Communicative Abilities of Grey Parrots. Harvard Universitiy Press, 2000
Array (FPGA) devices from Xilinx and Actel. The design is
vi
highly repetitive, with each node an exact replica of every Higher Intelligence, by Peter AJ van der Made. Page 72
other node. It is therefore expected that the small scale
design will scale quite well to a component containing at
least 10,000 nodes. The connectome for the larger scale
device will be the biological model of a cortical column. The
circuit‟s learning ability has been verified in an experiment
by using a signal generator and spectrum analyzer software
as a sound source and artificial cochlea respectively, with the
output connected to a synthetic neuro-anatomy consisting of
ten nodes in FPGA. The FPGA that was used was a 1.5
million gate ACTEL device on an ACTEL development
board. The signal generator frequency was varied to simulate
the frequency of common vowels in human speech. The
synthetic neuro-anatomy learned to recognize ten sounds that
were later also identified in a speech pattern. Learning time
was less than 2 minutes. An obvious application for this
technology is in speech recognition, speaker recognition, and
extraction of speech from a noisy background environment.
Other experiments show that the devices can be successfully
applied in applications such as visual image recognition,
robotics, emulation of the brain on small desktop computers,
autonomous learning machines used in exploration,
unmanned vehicles, and robotics. The advantages of using a
Synthetic Neuro-Anatomy device over a traditional
programmed device are a shorter development track, better
quality recognition, persistent learning after the initial
commission of the system and the reusability of training
models.
More examples and a more detailed explanation of this
technology is available in a new book by Peter van der Made