Blog — Intentions

Cybernetic sketches

§

Artificial intelligence will replace those who fear being replaced by it.

§

The strangest thing about artificial intelligence is that some people, after spending years formulating gigantic models capable of imitating human speech, are horrified to discover that they have created gigantic models that imitate human speech.

§

The deep learning architectures underlying current artificial intelligence are catalyzed by algorithms composed of fairly simple mathematical expressions, such as matrix multiplication, differentiation, and nonlinear functions. This alone is not enough to create an existential threat to humanity.

§

Four years ago, when I wrote about technological unemployment, the vast majority of researchers believed that artistic labor¹ could not be automated because it involved “imagination,” “creativity,” and other “eminently human” qualities. Today, those same people want to censor the artificial intelligences capable of generating artistic works. At bottom, the same false reasoning prevails: the belief that art is created ex nihilo with magic wands like genius and imagination, rather than through learning concrete techniques and drawing inspiration from hundreds of other works.

§

I imagine it must be impossible to use language, or even to think clearly, for those who complain about the term “artificial intelligence.” They must believe that calculus is done with pebbles, that geometers go out into the field to work the earth, and that philosophers waste time talking about epistemology when they should be preaching their love of wisdom.

§

Human intelligence is determined by the evolution of the species and is therefore specialized in the few parcels of reality necessary for its survival. The same can be said of the intelligence of any other animal. Using humans as the yardstick for intelligence implies an arbitrary evolutionary bias toward their everyday realities. That bias also implies a limitation on our definition of intelligence, as well as on the possible developments of artificial intelligences.

§

The scant scientific knowledge we have about intelligence and consciousness is not enough to dismiss the idea that machines can be intelligent or conscious. In any case, human standards are a poor metaphor for what artificial intelligence is, and a limitation on what it could be.

December 20, 2022

§

Appendix

Aquila non capit muscas, but since I already wasted my time watching this, let it at least serve as a warning to someone else: here are critical notes on Carlos Madrid’s conference on artificial intelligence.

1. His historical overview, although it openly claims not to be detailed, is quite imprecise. Research did not “shut down due to doses of reality” in the 70s. In fact, since the 60s, the approach of deep and probabilistic neural networks already existed (cf. Rosenblatt, Steinbuch). By the late 60s and early 70s, “learning” had already been achieved in various models. In the 70s, Seppo Linnainmaa devised the backpropagation algorithm, while Amari and Werbos used it to connect neural networks with stochastic gradient descent.

Similarly, history continued in the following decades and up to the present day in a manner different from what is presented. In general, the historical leaps made are quite egregious. By arriving at Deep Blue as the only notable specific milestone, saying that the probabilistic approach begins in the 90s with Bayesian networks, everything is wrong: it is not true that it starts in the 90s (but rather 30 years earlier), nor is it true that it begins with Bayesian networks, nor is it true that the Bayesian architecture is what is currently used for state-of-the-art models.

2. Around minute 46:20, it is said that computers are condemned to “think logically and axiomatically” according to certain rules. I would like to ask Carlos to specify in what sense and in what way the output of a transformer architecture is axiomatic.

Shortly after, in addition to alluding to “semiformal thought” (I don’t know if anyone can rigorously explain that term), he added something like: humans differ from machines because they also think with “geometric and topological symbols.” Could he explain exactly what he means by that and why it differs from “thinking logically or algebraically,” or why that constitutes intelligence, or why that differentiates us from machines? Could he also justify why, according to him, a machine is incapable of geometric representations but capable of algebraic ones? And while we’re at it, what is the nature of the multidimensional representations that a transformer learns during training, and in what sense are the vector and matrix transformations it performs not geometric? What parts of geometry are impossible to formulate in algebraic terms, and among those, which ones restrict the development of intelligence?

3. Regarding the Chinese Room argument (~47:25), although it has its kernel of truth, it is a caricature of the models. Modern models do not have “rules” to follow to the letter, nor do they have a “manual” outside of which they cannot function. It seems to me that Carlos is thinking of standard algorithms and not of the architectures that have been developed for decades. Precisely something interesting about these models is their nondeterministic nature. They have “emergent” properties, they do not function based on previously defined operations, and they are capable of offering coherent and true responses to completely new situations or problems (never before “seen” by the model).

In any case, in the room example, can Carlos (or any other living or past human) explain exactly what mental and operative processes the person inside the room would have to carry out for us to admit that they understand, comprehend, and reason? Can he give us the exact list of requirements necessary to say that a given entity reasons or does not reason? How many Nobel Prizes does he estimate would be needed to answer these questions?

To be clearer: we do not know how to define what psychological and neurological processes take place when we “understand,” so alluding to those processes is improper, especially to argue that machines do not satisfy them. We do not have a rigorous standard to define whether they satisfy them or not, and we will not have one until we better understand how human intelligence works. In the meantime, the criteria of distinction are highly questionable, especially when we get into the details (as was the case in the lecture).

4. Minute ~51:00. Carlos says, as if in critique, that if you remove the data from a model it stops working. I imagine Carlos says this because he thinks humans are capable of thinking ex nihilo.

Shortly after, he says that ChatGPT (and Google Translate) return frequent phrases because they perform “linear combinations between vectors.” The models are especially useful, among other things, because they use nonlinear functions, so I do not know what he means by that phrase. The models are also especially unique because they do not return “frequent phrases,” but rather often novel combinations made possible by the stochastic components underlying them (again: that is why we did not stay with Bayes).

5. Minute ~53:30. “They had to tell DALL-E that each image is a cat.” Either Carlos thinks that nobody ever had to point out to us what a cat is and that we know it a priori, or Carlos expects the reader to be immensely generous and translate this as “DALL-E does not have the same capacity for knowledge generalization as a human.”

6. Minute ~55:30. The characterization of how a model “interprets” an image is quite strange and confusing. He says the “program” only “sees pixels, not ears or eyes.” What exactly does “seeing ears” mean? Could Carlos explain what processes a human carries out to “see ears,” from the photoreceptors in the retina, through the electrical signals traversing the optic nerve, to the processing carried out by the primary visual cortex? Does he think the brain does not process the information it perceives and therefore “sees” the ear “raw” and represents it exactly as it arrived at the eyes? In particular, what does he think of the research that has been conducted since 1959, when Hubel and Wiesel discovered that different groups of neurons in cats respond to different shapes, shadows, and inclinations; ergo, biological visual processing does not occur holistically, “by ears and eyes,” but by structures, patterns, and simpler features? To put it plainly, both human vision and deep learning vision, besides requiring prior processing of information, operate by “features,” so the “seeing ears” argument strikes me as strange, weak, and confusing. And I also find it hard to understand why he believes objects cannot be differentiated in deep learning, when that is precisely one of its applications.

He also complains that the “program extracts statistical patterns from bits.” According to him, how then should models learn to represent objects in order to “see” like humans? If since 1959 it has not been demonstrated that animals also see by patterns, then how do they learn and how do they see? Why is it objectionable for perception to be guided by patterns? But more importantly, is it even desirable for machines to be exactly like us? Why is it so essential for intelligence that they visually process phenomena the way we do?

7. If intelligence is corporeal, apothetic, and operative, why is a simulation or a robot, which can perfectly satisfy all of that, not intelligent?

8. Minute ~1:01:00. Carlos complains that Google Translate needs certain context to translate “jack” instead of “cat.” I imagine he can right now translate “banco” into English without any additional context. Could he tell me his answer? And, if he would be so kind, could he also explain in detail why humans do not need to contextualize words to translate them into another language, or even to communicate?

9. Minute ~1:04:42. “As the program learns more in one trend, it becomes reluctant to learn in another direction.” What exactly does he mean by that? Overfitting? That the value of each “weight” in the model is fixed after training? If it is either of those things, there are quite elementary techniques to resolve it. Furthermore, in what sense does he make this critique? Are humans not also clearly biased as a function of their experience and knowledge?

Shortly after, he raises, as if in critique, that machines have the problem of induction and do not deal well with black swans. Have humans already solved that and is that why they are intelligent?

10. Minute ~1:16:41. “DALL-E is not intelligent because it uses pixels instead of brushes”? That is the “material substrate” that prevents calling a machine intelligent? What happens if DALL-E is integrated with a robot capable of painting its outputs with a brush? Where does the “corporeal” objection stand then? These robots that manipulate and throw objects and walk on the same ground as us, why are they not intelligent according to that same criterion?

11. Again, I find it strange that the models are reduced to Bayes’ theorem. The most commonly used probabilistic components (Markov processes, cross-entropy, the softmax function, negative log likelihood, etc.) are clearly distinct from the theorem and from Bayesian networks. Moreover, all of them are conjugated with many other elements of differential and vector calculus, linear algebra, mathematical optimization, etc., so selling them as simple probability is quite misleading. A strictly probabilistic model is not capable of reaching the same level of sophistication as ChatGPT.

In sum, Carlos did not define what he understands by intelligence (and who can claim to be capable of doing so?), but he did deny, for ambiguous, imprecise, erroneous, contradictory, and obscure reasons, that machines are intelligent. I perceived the eagerness to say that “artificial intelligence” is neither intelligent nor artificial, which is the same as getting angry because “calculus” does not deal with infinitely small pebbles, or saying that philosophy is a sham because it does not devote itself to preaching the love (philos) of wisdom (sophia). I do not see where one wants to arrive with that level of superficial terminological analysis. And, at the end of it all, nothing was said about the truly interesting topics that arise with these models (such as the emergent properties and originality I mentioned, or the similarities and differences between biological and computational processes, or the question of how we integrate understanding into them if we do not even know how it works or what it is). Carlos spent two hours stumbling over defective reasoning to finally say the same thing that anyone who has used ChatGPT (and half-understood any popular article on the subject) could have said: that models do not have intentionality or reasoning like humans, and that, well, you know, they work with those Bayes things and probability, something like that, you get the idea. I do not think one needs to stumble through an imprecise history, nor through an erroneous characterization, nor through false reasoning, nor through citations of parochial academics, nor through still-inconclusive sciences, nor through allusions to a term as ambiguous as “intelligence,” to say that machines do not reason like us. In the end, to use his own turn of phrase, one need not resolve the obscure with the more obscure.

PS: I am quite certain that Carlos Madrid saw my comments on more than one occasion, but he never responded or reacted in any way whatsoever. Let the reader judge why.

March 14, 2023

Musical bibliography