Thomas E. Portegys MAICS 2017 pp. 9–14 Morphognosis: the shape of knowledge in space and time  Thomas E. Portegys Ernst & Young LLP New York, NY, USA tom.portegys@ey.com Abstract the function of intelligence and which are incidental is a cru- cial and difficult task. Krakauer et al. [2017] recommend that Artificial intelligence research to a great degree fo- neuroscience work takes place after the study of related be- cuses on the brain and behaviors that the brain gen- haviors. erates. But the brain, an extremely complex struc- Unfortunately, the prospects of understanding complex ture resulting from millions of years of evolution, systems through examination and dissection are questionable can be viewed as a solution to problems posed by an [Jonas and Kording, 2016]. And as for constructing a com- environment existing in space and time. The envi- plete precise brain model, it is possible, as John von Neu- ronment generates signals that produce sensory mann believed [Mühlenbein, 2009], that at a certain level of events within an organism. Building an internal spa- complexity the simplest precise description of a thing is the tial and temporal model of the environment allows thing itself. In reaction to this, some efforts, such as The Hu- an organism to navigate and manipulate the environ- man Brain Project [2015] and Numenta [Hawkins, 2004; ment. Higher intelligence might be the ability to pro- White paper, 2011], have taken the position that analysis cess information coming from a larger extent of must be complemented with synthesis and simulation to space-time. In keeping with nature’s penchant for achieve a satisfactory level of understanding. extending rather than replacing, the purpose of the From an artificial intelligence (AI) viewpoint, we must mammalian neocortex might then be to record keep in mind that the purpose of a brain is to allow an organ- events from distant reaches of space and time and ism to navigate and manipulate its environment. Thus it is a render them, as though yet near and present, to the solution to problems posed by the environment. While the older, deeper brain whose instinctual roles have earlier days of AI seemed more focused on this viewpoint, changed little over eons. Here this notion is embod- recently neuroscience has assumed perhaps an outsized role ied in a model called morphognosis (morpho = in directing AI, even to the extent of governmental encour- shape and gnosis = knowledge). Its basic structure is agement [Vogelstein, 2014]. a pyramid of event recordings called a morphognos- Some researchers maintain that the environment largely tic. At the apex of the pyramid are the most recent consists of a body for the brain to interact with. The embodied and nearby events. Receding from the apex are less brain will thus leverage the sensory and motor capabilities of recent and possibly more distant events. A mor- a body that are adapted to an environment. Robotics research- phognostic can thus be viewed as a structure of pro- ers such as Brooks [1999], Hoffmann and Pfeifer [2011] have gressively larger chunks of space-time knowledge. argued that true artificial intelligence can only be achieved by A set of morphognostics forms long-term memories machines that have sensory and motor skills and are con- that are learned by exposure to the environment. A nected to the world through a body. However, this approach cellular automaton is used as the platform to inves- belies the problem since the body, like the brain, is also a so- tigate the morphognosis model, using a simulated lution to its environment. organism that learns to forage in its world for food, Determining a model of an organism’s environment is build a nest, and play the game of Pong. more tractable than creating a brain model of an environmen- tal model. But it requires settling on what is in the world that 1 Introduction produces sensory events and reacts to motor responses. Con- The human brain is the seat of intelligence. Thus when we founding this is that we of course must use our brains to do attempt to craft intelligence, naturally we turn to it as a guide. this. There is a common and somewhat ironic tendency to de- Fortunately, neuroscience is proceeding at an astounding scribe AI inputs and outputs in human cognitive terms, i.e. pace [Kaiser, 2014; Stetka, 2016], methodically unpacking its post-processed brain output, such as symbolic variables. mysteries. Yet the complexity of the brain, with billions of Hoffman [2009] argues that evolution has shaped our neurons and trillions of synapses, remains daunting. Teasing senses and perceptual machinery to only provide information apart which aspects and features of the brain are essential to on events that are ancestrally significant, such as finding food 9 Morphognosis: the Shape of Knowledge in Space and Time pp. 9–14 and safety. Other events in the environment that we cannot structure is a pyramid of event recordings called a morphog- directly sense must be mapped through technology onto our nostic, as shown in Figure 1. At the apex of the pyramid are sensory capabilities. For example, in the age of science the the most recent and nearby events. Receding from the apex existence and use of X-rays is important, but we sense them are less recent and possibly more distant events. only indirectly, as shadows on photographic film. Indeed, Morphognosis is partially inspired by an abstract morpho- Hoffman argues that reality may be more radically alien than genesis model called Morphozoic [Portegys et al., 2017]. we can imagine. Morphogenesis is the process of generating complex struc- Epistemological offerings would seem at best too abstract tures from simpler ones within an environment. Morphozoic to be useful for framing a sensory-response environment, and is based on hierarchically nested neighborhoods within a cel- at worst useless, as in the cases of nihilism and solipsism. lular automaton. Morphozoic was found to be robust and And physics has in recent times become increasingly muddier noise tolerant in reproducing a number of morphogenesis- on the “true” nature of reality: like phenomena, including Turing diffusion-reaction systems x The arrow of time may be related to the perception of [Turing, 1952], gastrulation, and neuron pathfinding. It is entropy [Halliwell, 1994] also capable of image reconstruction tasks. x String theory demands a number of extra infinitesimal dimensions [Rickles, 2014]. 2 Description x The perception of space may be a holographic projection The morphognosis model is demonstrated in three 2D cellular [Bousso, 2002]. environments: (1) a food foraging task, (2) a nest building x Reality could be a cellular automaton [Wolfram, 2002], task, and (3) the game of Pong. The food foraging task is used a graph [Wolfram, 2015], or a simulation [Moskowitz, as a venue to further define the model. 2016]. Despite these hazards, people universally experience the 2.1 Food foraging environment as a space-time structure. And even if there is a In this task a virtual creature called a mox finds itself in a 2D different underlying substructure, the model is empirically ef- cellular world as shown in Figure 2. To find food the mox fective. The presence of mammalian brain structures for map- must navigate around various obstacles of various types (col- ping spatial events [Vorhees and Williams, 2014] provides ors). evidence for the processing of this type of information. Sim- ilarly, brain structures for sensing the passage of time have also found support [Sanders, 2015]. Using space-time as a model, it can be speculated that higher intelligence is the ability to process information aris- ing from a larger extent of space-time. And in keeping with nature’s penchant for extending rather than replacing, the purpose of the mammalian neocortex might then be to record events from distant reaches of space and time and render them, as though yet near and present, to the older, deeper brain whose instinctual roles have changed little over eons. If this is so, these structures would be repurposed to embody language and abstract concepts. Figure 2 - Mox food foraging in a 2D cellular world. Figure 3 shows a snapshot of a morphognostic describing the space-time events, in this case obstacle encounters, while the mox forages. In this case a neighborhood is configured as a 3x3 set of sectors. Cell type densities are stored instead of raw cell values to allow linear scaling of information as the hierarchy increases since storing individual cell values would result in a geomet- ric growth. The cell type density is only one of a number of possible statistical or aggregation functions that could be used. An alternative might be to look at the distribution of cell types as an image processing operation, such as taking a Laplacian, Sobel or other image operator. Figure 1 – Morphognostic event pyramid 2.1.1 Morphognostic spatial neighborhoods Building an internal spatial and temporal model of the envi- A cell defines an elementary neighborhood: ronment allows an organism to navigate and manipulate the environment. This paper introduces a model called morphog- nosis (morpho = shape and gnosis = knowledge). Its basic neighborhood0 = cell 10 Thomas E. Portegys MAICS 2017 pp. 9–14 Figure 3 - Pyramid of obstacle type densities arranged as hierarchy of 3x3 cell neighborhoods. A non-elementary neighborhood consists of an NxN set of Metamorph “execution” consists generating a morphognos- sectors surrounding a lower level neighborhood: tic for the current mox position and orientation then finding the closest morphognostic contained in the learned meta- neighborhoodi = NxN(neighborhoodi-1) morph set, where: where N is an odd positive number. 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒(𝑚𝑒𝑡𝑎𝑚𝑜𝑟𝑝ℎ𝑖 , 𝑚𝑒𝑡𝑎𝑚𝑜𝑟𝑝ℎ𝑗 ) = The value of a sector is a vector representing a histogram of 𝑛𝑒𝑖𝑔ℎ𝑏𝑜𝑟ℎ𝑜𝑜𝑑𝑠 𝑠𝑒𝑐𝑡𝑜𝑟𝑠 𝑐𝑒𝑙𝑙 𝑡𝑦𝑝𝑒𝑠 𝑐𝑒𝑙𝑙 𝑡𝑦𝑝𝑒 𝑑𝑒𝑛𝑠𝑖𝑡𝑦𝑖,𝑥,𝑦,𝑧 − the cell type densities contained within it: ∑ ∑ ∑ 𝑎𝑏𝑠 ( ) 𝑐𝑒𝑙𝑙 𝑡𝑦𝑝𝑒 𝑑𝑒𝑛𝑠𝑖𝑡𝑦𝑗,𝑥,𝑦,𝑧 𝑥 𝑦 𝑧 value(sector) = (density(cell-type0), density(cell-type1), … density(cell-typen)) 2.1.4 Artificial neural network implementation In a complex environment, generating a large number of met- The number of cells contributing to the density histogram of amorphs may be prohibitive in terms of storage and search a sector of neighborhoodi = Ni-1xNi-1 processing. Alternatively, metamorphs can be used to train an artificial neural network (ANN), as shown in Figure 4, to 2.1.2 Morphognostic temporal neighborhoods learn responses associated with morphognostic inputs. Dur- A neighborhood contains events that occur between time ing operation, a current morphognostic can be input to the epoch and epoch + duration: ANN to produce a learned response. The ANN also has these advantages: t10 = 0 t20 = 1 • Faster. t1i = t2i-1 • More compact. t2i = (t2i-1 * 3) + 1 • More noise tolerant. epochi = t1i 2.1.5 Results durationi = t2i - t1i The mox were trained in worlds featuring a number of ran- 2.1.3 Metamorphs domly placed obstacles of various types. Training was done In order to navigate and manipulate the environment, it is by “autopiloting” the mox along an optimal path to the food. This generated a set of metamorphs suitable for testing. Table necessary for an agent to be able to respond to the environ- 1 shows the results of varying the neighborhood hierarchy ment. A metamorph embodies a morphognostic→response depth in a 10x10 world. Success indicates the mean amount rule. A set of metamorphs can be learned from a manual or of food eaten, so 1 is a perfect score. It can be observed that programmed sequence of responses within a world. more obstacles tend to improve performance. This is because Metamorphs establish an important feedback: they tend to form unique landmark configurations to guide the mox. Larger neighborhoods also tend to improve perfor- • Learned morphognostics shape responses. mance. • Responses shape the learning of morphognostics. 11 Morphognosis: the Shape of Knowledge in Space and Time pp. 9–14 Figure 4 – Metamorph artificial neural network. Neighborhoods Obstacle types Obstacles Food Noise #Train Food 1 1 10 0.1 0.1 1 1 1 1 20 0.2 0.1 5 1 1 2 10 0 0.1 10 1 1 2 20 0 0.25 1 0.9 1 4 10 0 0.25 5 1 1 4 20 0 0.25 10 1 2 1 10 0.3 0.5 1 0.6 2 1 20 0.4 0.5 5 0.8 2 2 10 0.2 0.5 10 0.9 2 2 20 0.6 Table 2 – Foraging with noise. 2 4 10 0.2 2.2 Nest building 2 4 20 0.6 This task illustrates how the morphognosis model can be used 3 1 10 1 to not only navigate but also manipulate the environment. 3 1 20 0.9 3 2 10 1 3 2 20 1 3 4 10 1 3 4 20 1 Table 1 – Foraging in a 10x10 world. The next test examines how well the model performs when the test world is not a duplicate of a training world, but is similar to a set of training worlds. Thus for this, multiple training runs are used. Before each training run, the cell types Figure 5 – Nest building with gathered stones. of all the cells are probabilistically modified to a random Left: scattered stones. Right completed nest. value. A successful test run must then rely on a composite of multiple training runs. The results are shown in Table 2. Of Figure 5 left shows an environment in which a nest will be note is how performance only begins to falter under heavy constructed out of 4 stones (reddish circles) on top of an ele- noise and few training runs. vation depicted by the shaded cells. The mox must seek out the stones, pick them up, and assemble them into the com- pleted nest shown in Figure 5 right. . 12 Thomas E. Portegys MAICS 2017 pp. 9–14 For this task, the mox is capable of sensing the presence of a 2.3.2 Procedure and results stone immediately in front of it, and sensing the elevation gra- Learner was trained with multiple randomly generated initial dient both laterally and in the forward-backward direction. In ball velocities. addition to the forward and turning movements used by the • When the ball moved left and right, the learner moved foraging task, the mox is capable of picking up a stone in with the ball. front of it and dropping the stone onto an unoccupied cell in front of it. An internal sense allows the mox to know whether • When the ball moved up or down, the learner moved to it is carrying a stone. the paddle and moved it up or down. Training was done by running the mox through 10 • This was the challenge: remembering ball state repetitions on “autopilot” to build a set of metamorphs. The while traversing empty cells to the paddle so as environment was then reset and the mox tested to discover to move it correctly, then to turn and return to whether it is capable of building the nest. Over 50 trials were ball for next input. performed with 100% success. Internally, the sensory infor- mation from the stone, gradient and stone carry states were sufficient to achieve success with a neighborhood hierarchy Testing on random games: 100% successful. of only one level. 3 Conclusion 2.3 Pong game This is an early exploration of the morphognosis model. The Much of the real world is nondeterministic, taking the form novelty of the model is both the method for integrating of unpredictable or probabilistic events that must be acted knowledge of events occurring in space and time dimensions upon. If AIs are to engage such phenomena, then they must in linear complexity, and the method of expressing the behav- be able to learn how to deal with nondeterminism. In this task the game of Pong poses a nondeterministic environment. The ioral interplay of responses and sensory events. The goal of learner is given an incomplete view of the game state and un- this project is to model the environment as something that derlying deterministic physics, resulting in a nondeterminis- could plausibly be in turn modeled by an artificial brain. tic game. This task has been found to be challenging for con- The positive results on the three tasks prompt future ventional machine learning algorithms [Portegys, 2015]. investigation. Moving up the ladder of animal intelligence, 2.3.1 Game details possible next tasks include: The goal of the game is to vertically move a paddle to prevent a bouncing ball from striking the right wall, as shown in Fig- • Web building. Can a space-time memories of building ure 6. one or more training webs allow one to be built in a • Ball and paddle move in a cellular grid. quasi-novel environment? • Unseen deterministic physics moves ball in • Food foraging social signaling. Bees retain memories of grid. foraging food sources that they communicate to other • Cell state: (ball state, paddle state) bees through instinctive dancing. Can this task be cast • Ball state: (empty, present, moving into the model? left/right/up/down) • Paddle state: (true | false) The Java code is available at https://github.com/porte- • Learner orientation: (north, south, east, west) gys/MoxWorx • Responses: (wait, forward, turn right/left) References • If paddle present and orientation north or south, then for- ward response moves paddle also. [Bousso, 2002] R. Bousso. The holographic principle. Re- views of Modern Physics. 74 (3): 825–874. arXiv:hep- th/0203101.2002. [Brooks, 1999] Rodney Brooks. Cambrian Intelligence: The Early History of the New AI. Cambridge MA: The MIT Press. ISBN 0-262-52263-2. 1999. [Halliwell, 1994] J. J. Halliwell. Physical Origins of Time Asymmetry. Cambridge University Press. ISBN 0-521- 56837-4. 1994. [Hawkins, 2004] Jeff Hawkins. On Intelligence (1 ed.). Times Books. p. 272. ISBN 0805074562. 2004. Figure 6 – The game of Pong. 13 Morphognosis: the Shape of Knowledge in Space and Time pp. 9–14 [Hoffman, 2009] D. D. Hoffman. The interface theory of per- http://www.npr.org/sections/health- ception: Natural selection drives true perception to swift shots/2016/12/31/507133144/from-psychedelics-to-alz- extinction. In: Object Categorization: Computer and Hu- heimers-2016-was-a-good-year-for-brain-science. 2016. man Vision Perspectives. Ed.: S.J. Dickinson, A. [Turing, 1952] A. M. Turing. The chemical basis of morpho- Leonardis, B. Schiele & M.J. Tarr. Cambridge, Cam- genesis. Phil. Trans. Roy. Soc. London B237, 37-72. bridge University Press: 148-165. 2009. 1952. [Hoffmann and Pfeifer, 2011] M. Hoffmann and R. Pfeifer. [Vogelstein, 2014] R. J. Vogelstein. Machine Intelligence The implications of embodiment for behavior and cogni- from Cortical Networks (MICrONS) Workshop. Intelli- tion: animal and robotic case studies, in W. Tschacher & gence Advanced Research Projects Activity (IARPA). C. Bergomi, ed., 'The Implications of Embodiment: Cog- https://www.iarpa.gov/index.php/research-programs/mi- nition and Communication', Exeter: Imprint Academic, crons. 2014. pp. 31-58. 2011. [Vorhees and Williams, 2014] C. V. Vorhees and M. T. Wil- [Human Brain Project, 2015] Human Brain Project, Frame- liams. Assessing Spatial Learning and Memory in Ro- work Partnership Agreement. https://www.humanbrain- dents. ILAR Journal. 55(2), 310–332. project.eu/documents/10180/538356/FPA++An- http://doi.org/10.1093/ilar/ilu013. 2014. nex+1+Part+B/41c4da2e-0e69-4295-8e98- 3484677d661f. 2015. [Wolfram, 2002] S. Wolfram. A New Kind of Science. Wolf- ram Media. ISBN-10: 1579550088. 2002. [Jonas and Kording, 2016] E. Jonas and K. Kording. Could a neuroscientist understand a microprocessor? bioRxiv [Wolfram, 2015] S. Wolfram. What Is Spacetime, Really? 055624; doi: https://doi.org/10.1101/055624. 2016. Stephen Wolfram Blog. http://blog.stephenwolf- ram.com/2015/12/what-is-spacetime-really/. 2015. [Kaiser, 2014] U. B. Kaiser. Editorial: Advances in Neuro- science: The BRAIN Initiative and Implications for Neu- roendocrinology. Molecular Endocrinology. 28(10), 1589–1591. http://doi.org/10.1210/me.2014-1288. 2014. [Krakauer, et al., 2017] J. W. Krakauer, et al. Neuroscience Needs Behavior: Correcting a Reductionist Bias. Neuron. Volume 93, Issue 3, pp. 480 – 490. 2017 [Moskowitz, 2016] C. Moskowitz. Are We Living in a Com- puter Simulation? Scientific American. 2016. [Mühlenbein, 2009] H. Mühlenbein. Computational Intelli- gence: The Legacy of Alan Turing and John von Neu- mann, in Computational Intelligence Collaboration, Fu- sion and Emergence. Editors: Mumford, C. L. (Ed.) Vol- ume 1 of the series Intelligent Systems Reference Library pp 23-43. 2009. [Numenta White paper 2011] http://numenta.org. 2011. [Portegys et al., 2017] T. Portegys, G. Pascualy, R. Gordon, S. McGrew, B. Alicea. Morphozoic: cellular automata with nested neighborhoods as a metamorphic representa- tion of morphogenesis. In Multi-Agent Based Simulations Applied to Biological and Environmental Systems. ISBN: 978-1-5225-1756-6. 2017. [Portegys, 2015] T. Portegys. Training Artificial Neural Net- works to Learn a Nondeterministic Game. ICAI'15: The 2015 International Conference on Artificial Intelligence https://arxiv.org/abs/1507.04029. 2015. [Rickles, 2014] D. Rickles. A Brief History of String Theory: From Dual Models to M-Theory. Springer Science & Business Media. ISBN 978-3-642-45128-7. 2014. [Sanders, 2015] L. Sanders. How the brain perceives time. ScienceNews. https://www.sciencenews.org/article/how- brain-perceives-time. 2015. [Stetka, 2016] B. Stetka. From Psychedelics To Alzheimer's, 2016 Was A Good Year For Brain Science. 14