Intro
I totally echo with the point that life is basically about lowering entropy. A bit surprised at how widely conceived this idea is, I googled it and found one of the earliest writings about it was from Erwin Schrödinger the quantum physicist, who first used the word “negentropy” or negative entropy in his 1944 popular science book what is life to describe the dynamics of life.
However, maybe because there’s a bit of Chauvinism in me, I don’t agree that there’s not a type of intelligence more superior than others. It could be true in the context of surviving in the biosphere, as clearly all current life forms have survived billions of years through evolution; however, both in terms of the essential meaning of life, as well as potential of spreading life across the universe, I think one must measure level of intelligence in terms of information entropy. As of today, only humans have shown the ability to summarize the physical rule of the universe and proving that they apply to far beyond where we live. The author’s examples of octopus’ multitasking, birds’ visual processing, fishes fast reflexes help them win the contest of evolution, but are nothing compared to humans’ use of heat energy, computers, relativity and quantum physics, and more importantly, knowing that there are a set of rules that will be always true, in the air we breathe, in the center of a nuclear bomb, or in the universe lightyears away.
Breakthrough #1
Well written and I learned a lot.
What I didn’t know before and find interesting are adaptation and learning.
It’s interesting to learn that adaptation happens at single neuron level without the need of brain or centralized processing of distributed information. The author might mention it in later sections, this makes me think of the diminishing/ exploding gradient topic in neural net algorithms. The solutions I know of are all based on structure of the network rather than how a single neuron works (although, in a way, one could define a set of nodes and their linkage between them as one neuron).
Learning – here defined as the most basic form, the “strength” of the connections between neurons – also happens at a lot lower level than I considered before. This, to me, is more easily compared with neural network as the “weight” value “learned” by the network.
Breakthrough #2
One thing I’m interested in but not explained in depth: why would the reinforced learning and measurement of time only capable by vertebrates? These complex functions sounds unlikely to be lying in the spines, but would come from the complexity of the brains. What part/mechanism of the brain makes this possible?
Does current computer vision/ convolutional neural nets work in the same way as human vision? I think the pattern/ feature recognition makes sense, however, in the human 3d object example, I believe humans – and probably vertebrates in general – are able to ‘reconstruct’ or ‘imagine’ objects in a 3 dimensional world even when it’s just a 2D picture (or essentially the imaging in our retina would be 2D anyway). There must be certain structure of the brain that naturally interpret things with volume/ depth/ distance. Would computer vision today do that? I suspect probably not otherwise there won’t be a discussion of whether radars are needed to determine distance in self driving cars. Using wording from breakthrough #3, these systems likely still lacks ‘world model’ understanding the observations are projections of a 3D world, those objects, and ‘self’ are 3D objects moving in the world with a certain range of rules.
Breakthrough #3
It is amazing learning about the linkage between the mechanism of neocortical column and neural net models of recognition-generation structure. Before reading this section I was thinking of this kind of model more as a ‘dimensionality reduction’ technique through math tricks. This structure really explains how this type of unsupervised model can work and evolve. The fact that the neocortical columns all have same structure but will have different functions also seem to explain why these types of neural net models can succeed in many seemingly different tasks from computer vision to voice recognition to large language model, as well as explains why these models benefit so much from just scaling.
Breakthrough #4
Socializing is a key part of this breakthrough. I think this section gives a very nice perspective that explains the differences between animals that exhibit ‘social’ behaviors, that is by whether individuals would mentalize what other individuals would do, or to ‘put myself in your shoes’. Ants, bees, herds of fish are able to form large colonies and can be capable of working together in complicated tasks. However, with their simple brains and what we know about their brains so far, I would tend to think these behaviors are more ‘mechanial’ or ‘reflective’ that is selected through evolution. One ant would just follow the smell or hormone stimulations that leads to it moving food to its home, rather than being able to think ‘the ant queen would reward me this if I complete this task, and having food in home will be good to small ants that are still growing’.
Breakthrough #5
Although I did not think language is a key component of a higher level of intelligence, I do have some doubts in the current large language models and the book has one angle to explain it. The large language models were directly trained on and applied to language which itself is already a product of high intelligence, rather than being built on the foundation of language – representation of the world, represenation of the rules of the world, representations of thoughts and thoughts of others. It’s both the ‘world model’ problem as well as the foundation of language. This way, no matter how good they are at simulating the appearance of language, I always feel there’s something missing.
Breakthrough #6
I have thought about ways that humans may be able to persist to a time period much longer than the biosphere (which I might write about in other posts). However the author in this section raises a more viable path to extend the existence of intelligence – by man made AI. This indeed seems like a much more likely path given current technologies. Actually I believe, even with the technology today, human should be able to create robot factories that can reproduce themselves, with automated solar or nuclear energy, mining, robot building, and will be able to sustain the system longer than human history if humans ourselves do not destroy them. All the components should be readily available today, it just needs putting the system together, and of course, a reason for us humans to do it. I have little doubt that if humans are facing some type of bio crisis and will all have to work together to find a way out for mankind, we should be able to build this system in a few years, far more easily than sending a colony to Mars and sustain there. However as far as I know, the current AI progress lacks the “learning” ability, therefore not able to explore deeper physics rules, not able to send themselves into the universe to avoid the end of the earth, or ultimately, not able to further reduce the information entropy of itself and the world around it. If that becomes the end of human kind, I would be disappointed as it’s not much more than extinction.
Replica of Tinycup 