Say hello to reductio-ad-Fido. Pavlov’s dog of the classic 1927 experiments was conditioned to associate the ringing of a bell with the presentation of food. The training so complete that an involuntary response such as salivation could become triggered by the bell stimulus alone.
Such behavioural systems based learning systems, together with insights emerging into biological neural systems, were a source of inspiration to the mathematical sciences. The McCullocks-Pitt’s neuron (1943) captured this as a computational machine in which inputs modified by weights fed into an activation function resulting in a binary outcome. Weights could be positive – excitatory – or negative – labelled as inhibitory. And so was borne the neural net as depicted in its simplest form below.
The input stimulus and the signal response can be measured by observation but what of the weights? Donald O. Hebb’s 1949 publication “Organization of Behaviour” from the field of Psychology led to a breakthrough known as the “Hebbian Learning” rule. His finding paraphrased as neurons that fire together, wire together suggested that learning occurs by modification of the strength of the synaptic connections. Strengthening when the neurons simultaneously fire and weakening otherwise. Formulaically:
Weights can be calculated by iteration techniques using a set of known training input/output pairs and so a prediction made when a new input presented. Such neuronal information processing systems proved of enormous value in solving previously intractable optimization problems and became the cornerstone of the field of computational neuroscience.
Hebbian learning brings into sharp relief the adaptive, plastic nature of the brain as it continuously re-calibrates the strength of neuronal connections in response to experience or directed attention. This way memories are revised with each recall and habits formed when rewarded. For our dog, initially a food stimulus triggers a salivation response reflected as an excitatory weighting for food and inhibitory for the bell. If a bell is rung immediately preceding the occurrence of food, the dog comes to associate the sound for an expectation of food and the weighting for the bell stimulus becomes excitatory. Indeed, with adequate training the bell stimulus weight increases sufficiently to trip the salivation response even in the absence of the food stimulus. Similarly, if over time food fails to appear then the bell stimulus weakens and the salivation response returns to its default unconditioned state. Synchronicity of food, bell and salivation is lost.
This is an example of interdisciplinary interplay where insights originating from neuroanatomical structures and neurobiological reactions find their way as useful man made machines. Conversely, rendering the human brain into such simplified models aids the natural sciences in their quest for understanding. The evolution of the brain has resulted in a form with function at times at odds with the demands of modern day living. But this same curious complexity entity augurs the potential for yet more ingenious discoveries. Perhaps a next generation intelligent device that can synchronize with its human host and so truly complement the brain’s pattern matching, creative and predictive capabilities?