Ghost in The Shell?

I recently rewatched the AlphaGo documentary. I'd already seen it a while ago, but this viewing left me with some thoughts and questions I hadn't considered before. AlphaGo's mastery of Go stands as an incredible achievement in AI research, but it also made me question whether everything that I consider to be really true and valuable actually means anything. The black box of AlphaGo leaves deep ripples on the fabric of human pride, and made me question what it means to really be alive.

Mastery and Enlightenment, or True Liberty

David Foster Wallace has an old essay aptly titled "Roger Federer as Religious Experience":

"Almost anyone who loves tennis and follows the men's tour on television has, over the last few years, had what might be termed Federer Moments. These are times, as you watch the young Swiss play, when the jaw drops and eyes protrude and sounds are made that bring spouses in from other rooms to see if you're O.K.

The Moments are more intense if you've played enough tennis to understand the impossibility of what you just saw him do. We've all got our examples…given Agassi's position and world-class quickness, Federer had to send that ball down a two-inch pipe of space in order to pass him, which he did, moving backwards, with no setup time and none of his weight behind the shot. It was impossible. It was like something out of "The Matrix." I don't know what-all sounds were involved, but my spouse says she hurried in and there was popcorn all over the couch and I was down on one knee and my eyeballs looked like novelty-shop eyeballs."

The Bhagavad Gita denotes multiple different paths to enlightenment: Bhakti, loving devotion to a deity; Dhyana, the active practice of meditation; and Karma, which, in my interpretation, means perfecting your craft until you are able to do it flawlessly; that is, enlightenment through mastery.

Performing at the highest level allows a human being to forget his earthly form, and essentially become a vessel for his art, thus liberating himself (at least temporarily) from the grasp of Samsara:

"Any man who wants to master the essence of my strategy must research diligently, training morning and evening. Thus can he polish his skill, become free from self, and realise extraordinary ability. He will come to possess miraculous power."

— Musashi, Book of Five Rings

Miyamoto Musashi is widely considered to be the greatest swordsman of all time, and has become a national legend in Japan. His writings on martial arts are still deeply studied today; he was also an accomplished painter, and had many exchanges with Takuan Soho, a famous Zen master. Their letters can be found in The Unfettered Mind. This print by Kuniyoshi depicts Musashi.

Notice how Musashi emphasises that a master is one who has become free from the confines of his own being: he has achieved liberty, enlightenment, absolute freedom by focusing on his art.

Someone like Roger Federer gives us a small glimpse into what this means: when we see a "Federer moment", the only possible reaction we can have is to simply look on in amazement, having witnessed what seems like a flash of the divine. I often have a similar feeling while listening to Bach: it is completely impossible for me to understand how a human mind can create such ornate beauty. The only conclusion that I've been able to come to is that Bach mastered his craft to such an extent that his music was directly fed by some kind of higher power. It seems like this is a pretty universal experience when listening to him:

"Beethoven tells you what it's like to be Beethoven and Mozart tells you what it's like to be human. Bach tells you what it's like to be the universe."

— Douglas Adams

At a certain level, some performances can be accurately captured by only a single word: divine.

The conclusion I make of this is that achieving true mastery really does allow one to somewhat transcend the bonds of regular human existence, and peer into something endlessly beautiful. Indeed, when one thinks of what it means to be human, nothing is more truly human, truly free, than the flawless work of a master. By mastering one's craft, one notices the significant difference between existing and being alive. What better way to express the boundless joy of life than through incredible creation?

This point of view is often heavily emphasised in schools of Zen Buddhism; the practice and deep understanding of martial arts, painting, or poetry were considered essential pursuits for adepts of the school, since developing in these fields allowed one to physically express his level of enlightenment. This is why both Musashi and Hozoin, despite being known for fighting prowess, are also considered extremely accomplished Zen masters. It's also worth noting that a similar system existed in ancient Chinese courts: nobles were expected to master the four arts of music, calligraphy, painting, and Go.

Clearly, these four disciplines weren't chosen at random; although I am not an expert in Chinese history and customs, I think it's reasonable to assume that they were chosen as benchmarks for a person's ability to deeply feel and think; basically, be alive. The deeper one experiences the human condition, the more likely this person is to be a masterful poet, painter, writer, or Go player. Hence a strong marker of being deeply human - understanding, wise, conscious - is achieving genius-level mastery.

What is Go?

Go is an abstract strategy board game, currently believed to be the oldest existing board game which still enjoys regular play around the world. It first arose in China, about 2,500 years ago; by the 5th century, it had reached Korea, and 200 years later, it became popular in Japan. Some literary works of the Nara period, like The Tale of Genji and The Pillow Book explicitly mention the game as a favorite pastime of Japanese aristocrats.

In Japan, the game mainly took hold in courts and temples: by the end of the 16th century, daimyo (lords) hired professional Go teachers, who would stay in palaces as masters in residence. The most notable example of this is Honin'bo Sansa, a Nichiren Buddhist priest who was the first man to be awarded the honorary title of Meijin (literally meaning "Brilliant Man", or master). Sansa privately tutored Hideyoshi, Nobunaga, and Tokugawa, the three lords who successfully united feudal Japan, and later went on to found Japan's first Go school.

Honin'bo Sansa, who also went by his Dharma name of Nikkai, was one of the first nationally recognized Go masters. Here he is in prayer and is wearing the robes of a Buddhist priest.

Throughout the Edo period, this title was passed from master to master, bestowed upon whoever was the strongest Go player of a given generation. Over time, a ranking system was created to help classify players of different levels: this was the ancient Japanese form of Elo. The same system of kyu and dan classification is still in use today, 1 dan being equivalent to a black belt; the highest possible rank is 9 dan, reserved for the highest levels of professional play.

It's interesting to note that until the nineteenth century, the vast majority of Japan's strongest players were Buddhist monks, who saw the game as another vehicle towards enlightenment alongside traditional arts such as swordsmanship and flower arrangement (ikebana). Many (almost all) samurai were practicing Buddhists, and often considered the game as an excellent tool for overcoming greed, anger, and fear. In this respect, the tradition of kido (the way of Go) stood alongside budo (the way of martial arts) as a means of deepening wisdom.

Today, Go is still incredibly popular and has about 20 million active players, most of which are based in East Asia.

Although the rules of Go are extremely simple, the game is very strategically rich and has proven to be a kind of "last stand" for AI researchers; while Deep Blue beat Garry Kasparov in 1997, AlphaGo beat the world's best Go player almost 20 years later, in 2016.

This is largely due to Go's emergent complexity, which is an order of magnitude higher than that of chess; there are more possible Go positions than atoms in the universe, so even if every single possible atom was used to build a supercomputer for brute-force computation of all potential Go plays, it would take over a million years for the program to make a few moves. For this reason, a Go program has long been considered the Holy Grail of AI research.

Go also appears to require more strategic intuition than chess, making it significantly harder for programs to devise "game plans", or holistically analyse a position and create a general plan to follow. In chess, the relatively small number of total positions allows one to create a basic chess AI through a simple minmax algorithm which considers all the possible outcomes from a given situation, and chooses the one which maximises estimated win rate. Obviously, a more advanced algorithm would work better, but it is not very difficult to create a basic chess AI which can consistently beat very strong players. In the words of I.J. Good, one of the British mathematicians who worked alongside Turing (!) at Bletchley Park:

"Go on a computer? – In order to program a computer to play a reasonable game of Go, rather than merely a legal game – it is necessary to formalise the principles of good strategy, or to design a learning program. The principles are more qualitative and mysterious than in chess, and depend more on judgement. So, I think it will be even more difficult to program a computer to play a reasonable game of Go than of chess."

This quote is from 1965. It took only 32 years to solve chess, and just over 50 years to solve Go. By comparison with the field of math, it took humans multiple centuries to figure out Fermat's last theorem, and we still have no idea what to do with the Riemann hypothesis.

The rules of Go are as follows:

Two players sit across from each other, taking turns placing black and white stones on crosses, delineated on a 19x19 grid: this is the playing board.

The objective of the game is to capture territory with your stones; once you fully enclose a given space with your pieces, it becomes yours. An unbroken chain of your stones must be aligned alongside each other for an area to count as captured. The number of crosses inside your enclosed area are counted, and then added to your count as a certain number of points. For example, in the diagram below, the top-center part of the board is fully enclosed by a chain of black stones, which form a diamond shape. The territory inside these stones is now considered captured by black, and two points are added to black (since there are two crosses inside the area).

Like in chess, it is also possible to capture enemy pieces. The same principle as before applies: if you fully surround a stone, or chain of stones, of the opposing color, you can capture it and remove the pieces from the board. Each captured stone is equivalent to one point, so the same number of points is awarded for a captured stone or area.

A stone is considered surrounded when it doesn't have any more 'liberties'. This is best illustrated through an example:

Above, a white stone is placed down on a cross. Before any black stones are placed down, the white stone has four adjacent crosses onto which another white stone can be placed to start creating a chain; these four spots are the stone's 'liberties'. If black surrounds these adjacent crosses before the white stone is connected to a larger chain, the white stone is left without any liberties (Diagram 6), and black removes the stone from the board, placing it in his pile of captured stones.

Unlike chess, once a stone is put down, it cannot be moved or adjusted. It will either remain in its position until the end of game, or be removed after capture.

The image below shows the liberties of a chain:

The three white stones in the bottom left are all connected, so for black to capture this chain, it needs to fully surround all the adjacent liberties (the crosses labeled 1, 2, 3, 4, 5, 6, 7, 8).

That is pretty much it - there are a few other small details, but these are all the basic rules one needs to know to begin playing. The game continues until someone either resigns or runs out of stones; the captured areas and stones are then counted, and whoever has the most points wins.

As you can see, the rules are extremely simple and elegant; however, they are intricate enough to create a beautifully nuanced game.

The Wonder of AlphaGo

In February of 2016, DeepMind published a paper in Nature which announced the first flashes of AlphaGo's brilliance. Artificial intelligence had struggled for decades to beat even low-level professional players; however, the paper claimed that DeepMind's team made a breakthrough and beat Fan Hui, the reigning European Go champion. This also meant that AlphaGo could beat every single other Go program.

AlphaGo didn't just beat Fan Hui: it crushed him 5-0. Sweep.

Five months later, DeepMind challenged one of the world's strongest Go players, Lee Sedol, to another series of five matches; the winner was promised a hefty $1 million prize. Many Korean pros joked before the match that this was the easiest prize money in Go history; in the pre-game conference, Lee Sedol confidently claimed that he doesn't doubt his victory.

A few weeks before the match, Lee Sedol was awarded the Myeongin title, the Korean equivalent to Meijin. He is widely considered to be the strongest Go player of the last decade, holding the record for second-most international victories won. At the time that AlphaGo played him, he was ranked the fifth best player in the world.

By the end of the series, the world of Go was shocked: the final score was 4-1, in AlphaGo's favor. This happened in March of 2016, when AI experts unanimously agreed that the defeat of any professional player - let alone the best one in the world - was at least a decade away. Out of the five games, the Korean champion managed to win only his fourth one; and, even then, he managed to do this by playing a move later dubbed "The Hand of God", or a 1 in 10,000 move which could have only been played by a genius. The four losses were all by resignation.

Funnily enough, this wasn't the first 1 in 10,000 move that was played during the series: after the end of the first game, Lee Sedol already remarked that AlphaGo was capable of making moves no human would have ever played. After the end of the second game, AlphaGo's move 37 was, surprise, also dubbed a 1 in 10,000 move. I think Sedol's reaction to the move speaks for itself:

"I thought AlphaGo was based on probability calculation and it was merely a machine. But when I saw this move, I changed my mind. Surely, AlphaGo is creative. This move was really creative and beautiful…this move made me think about Go in a new light. What does creativity mean in Go? It was a really meaningful move."

Lee Sedol playing AlphaGo

Fan Hui, who helped the DeepMind team after losing to AlphaGo, commented:

"This move is very special because with this move, all the stones played before work together - they all connect. It looks like a network, linked everywhere. It's very special…very special. Move 37 - very beautiful…you know, we are Go players. In China, in Korea, in Japan, we see Go like art. We are artists, you know…we play our best for Go. So please, be gentle with Lee Sedol, he's a very, very, good player, he is a great player. I was in the room, I saw him, he wanted to win - he tried everything, everything. It's just that…we can't."

Keep in mind that these comments are coming from some of the best Go players in human history.

Despite their achievement, even the DeepMind team had mixed reactions:

"I couldn't celebrate. It was fantastic that we'd won, but there was such a big part of me that saw this man trying so hard, and being so…disappointed."

The leader of DeepMind, Demis Hassabis (former chess prodigy), also had some very interesting remarks after AlphaGo's victory against Sedol:

"Even I feel a bit ambivalent about it, given I'm a games player, and Go is the pinnacle of board games…but I really like the statement one of the top Chinese professionals said: if AlphaGo wins, maybe we'll really start to see what this game's about."

(Start to see? After 2000 years? It sounds insane, but seems like it's true.)

It's also important to note that AlphaGo doesn't play from a "database" of possible moves. Rather, it's an RL algorithm which taught itself how to play, endlessly playing millions of games against itself. Thore Graepel, one of DeepMind's lead scientists, explained:

"Although we have programmed this machine to play, we have no idea what moves it will come up with. Its moves are an emergent phenomenon from the training. We just create the data sets and the training algorithms. But the moves it then comes up with are out of our hands—and much better than we, as Go players, could come up with."

Within about a year, AlphaGo played a series of three games against the reigning world champion, China's Ke Jie, ranked the best player in the world. Although Jie's initial comments were strikingly similar to Sedol's, the result was the same: 3-0 for AlphaGo. By 2019, Lee Sedol retired from professional Go, citing that:

"I'm not at the top even if I become number one…this is an entity that cannot be defeated."

"What surprised me the most was that AlphaGo showed us that moves humans may have thought are creative, were actually conventional. I think this will bring a new paradigm to Go."

Imagine if such praise was said to a human player instead of a machine - clearly, such a person would be considered an unparalleled genius, an absolute master of his craft. AlphaGo didn't win by brute force calculation - AlphaGo won through its creativity, and managed to create stunning beauty. I think this is insane, in the best and worst way possible - because no one understands what this really means.

As I mentioned before, mastery, creativity, and the creation of breathtaking beauty are benchmarks of a person's mind, and were even used as expressions of enlightenment, deep awareness and consciousness. Is it possible that AlphaGo showed us glimpses of this consciousness? Is this the very freedom from self Musashi spoke about?

Thoughts from Hofstadter and Penrose

Like almost everyone remotely interested in AI, my first brush with the subject came through Hofstadter's wonderful GEB. Today, I'd bet almost every single computer science and math department in the world has at least one copy of the book. You should definitely read it if you haven't yet, it's probably the best bird's eye view of music, art, computer science, math, Zen Buddhism, and cognitive science I've ever read. Even though all of these topics seem completely disconnected or arbitrary, Hofstadter actually shows how many of them exhibit similar structures (in the words of abstract algebra, they are isomorphic), and how these same structures - strange loops - could be responsible for the human mind's ability to be conscious. The book was also a watershed since it was one of the first books to systematically analyse this and apply it to the field of AI; Hofstadter was one of AI's earliest (if not most famous) pioneers.

Without going too deeply into the actual contents of the book, the thesis of Hofstadter's main ideas is simple to follow: one can consider consciousness as a loop. Indeed, what does it mean to be conscious? At a basic level, you could argue that consciousness is an extension of Descartes' argument. Once my mind loops back on itself and recognises that "I" exist, I am conscious, since I have become aware of my existence. This looping, which Hofstadter conjectures to be an essential ingredient of consciousness, is called a strange loop.

At the crux of Hofstadter's idea, though, isn't just that strange loops create our minds; the most interesting part of the idea is that strange loops are not unique to human brains. In fact, they arise in many possible contexts, such as in Bach's music (the fugue) or in mathematics (Gödel's theorems). Since mathematics, which contains these loops, can be thought of as a formal system derived from simple axioms and rules of inference, then perhaps our minds, too, can be considered as formal systems: after all, the same loop emerges in both, and is considered responsible for the feeling of "I" inherent to consciousness. So, a set of computable, simple rules can give rise to extremely rich emergent phenomena: namely, our minds. Quoting from the book:

"We have come to the point where we can develop one of the main theses of this book: that every aspect of thinking can be viewed as a high-level description of a system which, on a low level, is governed by simple, even formal, rules. The "system", of course, is a brain-unless one is' speaking of thought processes flowing in another medium, such as a computer's circuits. The image is that of a formal system underlying an "informal system"-a system which can, for instance, make puns, discover number patterns, forget names, make awful blunders in chess, and so forth. This is what one sees from the outside: its informal, overt, software level. By contrast, it has a formal, hidden, hardware level (or "substrate") which is a formidably complex mechanism that makes transitions from state to state according to definite rules physically embodied in it, and according to the input of signals which impinge on it …this vision seems to imply that, at bottom, the brain is some sort of a "mathematical" object. Actually, that is at best a very awkward way to look at the brain. The reason is that, even if a brain is, in a technical and abstract sense, some sort of formal system, it remains true that mathematicians only work with simple and elegant systems, systems in which everything is extremely clearly defined-and the brain is a far cry from that, with its ten billion or more semi-independent neurons, quasi-randomly connected up to each other. So mathematicians would never study a real brain's networks. And if you define "mathematics" as what mathematicians enjoy doing, then the properties of brains are not mathematical. The only way to understand such a complex system as a brain is by chunking it on higher and higher levels, and thereby losing some precision at each step. What emerges at the top level is the "informal system" which obeys so many rules of such complexity that we do not yet have the vocabulary to think about it. And that is what Artificial Intelligence research is hoping to find."

In this sense, Hofstadter's argument can be considered a modification of the Church-Turing thesis; although he presents multiple versions of the thesis himself, I think the clearest formulation is the following (taken from chapter XVII of GEB):

"CHURCH-TURING THESIS, REDUCTIONIST'S VERSION: All brain processes are derived from a computable substrate."

Obviously, this is the strongest possible claim one could make to support the development of AI as a means of simulating the human mind. Hofstadter also presents a slightly modified version, which essentially says the same thing:

"CHURCH-TURING THESIS, MICROSCOPIC VERSION: The behavior of the components of a living being can be simulated on a computer. That is, the behavior of any component (typically assumed to be a cell) can be calculated by a [general recursive function] to any desired degree of accuracy, given a sufficiently precise description of the component's internal state and local environment.

This version of the Church-Turing Thesis says that brain processes do not possess any more mystique-even though they possess more levels of organization-than, say, stomach processes. It would be unthinkable in this day and age to suggest that people digest their food, not by ordinary chemical processes, but by a sort of mysterious and magic "assimilation". This version of the CT-Thesis simply extends this kind of commonsense reasoning to brain processes. In short, it amounts to faith that the brain operates in a way which is, in principle, understandable. It is a piece of reductionist faith."

If Hofstadter is correct - that the physical substrate of our minds (neurons) is mainly tasked with providing the hardware for an emergent software which can also arise out of computable functions - then our brain is not anything particularly special. All of its functions could be replicated in a computer, albeit through electric circuits rather than nerve cells. This means that all levels of human intelligence can exist in the form of an AI.

However, one could always make an argument against this which runs along the following lines:

"Sure, the AI is computing something amazing, but it's just following the commands of a program, and not actually thinking. We humans obviously don't do this when we think - therefore, as impressive as AlphaGo's feat was, it doesn't actually understand anything. It is just computing, albeit in a very complex way."

If Hofstadter's thesis is true, then this doesn't really matter, because it would imply that our brains are just doing the same thing: computing something in a very high-level way. However, it is a point worth considering, and Hofstadter returns to it in a 1982 Metamagical Themas essay, titled "On the Seeming Paradox of Mechanizing Creativity":

"It is a commonly heard statement that…an 'unanalyzable leap of the imagination' takes place when a great mind comes up with a new idea. [Creativity] is all felt to be somehow too deep down, too hidden, too occult a gift, to be mechanical in any sense. Creativity, in fact, is perhaps one of the last refuges of the soul. 'You may mechanize your logic', says the English professor to the computer scientist, 'but you'll never lay a finger on poetry'…Is this kind of statement irrational? Is it a reflection of a deep-seated fear that even this most sacred aspect of humanity is doomed?…Why make such a big deal out of an activity of the human mind which, like every other activity in life, has shades and degrees? After all, the creative blurs with the mundane…is it possible that here lies the elusive difference between the living and the dead, the human and the machine, the mental and the mechanical?

…About the touchy question of the mechanization of the mental, many educated people feel that, although a machine may … do a creditable job of acting like a person, any machine's performance will remain lackluster and dull…you'll simply be able to tell that it is unoriginal…This sense that you will eventually be able to "just tell"…seems to depend upon a tacit assumption about human thought: that [the] creative spark is not the exclusive property of just a few rare individuals, but quite to the contrary, it is an intrinsic ingredient of the everyday mental activity of everyone. In short, it seems that people who feel that machines - even intelligent ones - will always remain duller than minds are tacitly relying on the following thesis: creativity is part of the very fabric of all human thought, rather than some esoteric, rare, exceptional, and fluky by-product of the ability to think…

I see creativity and insight, for machines no less than for people, as intimately bound up with intelligence."

Hofstadter then quotes from Dean Wooldridge's Mechanical Man: The Physical Basis of Intelligent Life:

"When the time comes for egg laying, the wasp Sphex builds a burrow for the purpose and seeks out a cricket which she stings in such a way as to paralyze but not kill it. She drags the cricket into the burrow, lays her eggs alongside, closes the burrow, then flies away, never to return. In due course, the eggs hatch and the wasp grubs feed off the paralyzed cricket, which has not yet decayed, having been kept in the wasp equivalent of a deepfreeze. To the human mind, such an elaborately organized and seemingly purposeful routine conveys a convincing flavor of logic and thoughtfulness - until more details are examined. For example, the wasp's routine is to bring the paralyzed cricket to the burrow, leave it on the threshold, go inside to see that all is well, emerge, and then drag the cricket in. If the cricket is moved a few inches away while the wasp is inside making her preliminary inspection, the wasp, on emerging from the burrow, will bring the cricket back to the threshold, but not inside, and will then repeat the preparatory procedure of entering the burrow to see that everything is all right. If again the cricket is removed a few inches while the wasp is inside, once again she will move the cricket up to the threshold and reenter the burrow for a final check. The wasp never thinks of pulling the cricket straight in. On one occasion this procedure was repeated forty times, with the same result."

Of course, it is tempting to accuse AlphaGo of simply exhibiting the same "Sphexish" behaviour: despite being deeply intricate and playing genius moves, it is simply executing an algorithm.

But what is the difference, then, between high-level Go players such as Lee Sedol and AlphaGo? Is Lee Sedol not also executing an (albeit very intricate) algorithm? Naturally, expert players have a high degree of intuition, which cannot be taught; perhaps this is a uniquely human feature. But if AlphaGo is capable of demonstrating the same seeming leaps of intuition, such as with move 37, is it not just doing the same thing? Just like Lee Sedol, AlphaGo learned to play by playing millions of games and analyzing its mistakes; it does not have a massive collection of possible games in its memory which it decides to jump upon in a given position. Rather, just like Sedol, it thinks ahead - and appears to display genuine creativity and intuition. Of course, maybe it is just "pretending" to be creative, when under the hood it is not actually doing the same "magical" thinking players like Sedol could achieve. But is there really a tangible difference? In my opinion, there doesn't seem to be, and this strongly convinces me of the fact that a variant of the Church-Turing thesis must be at least somewhat true.

Hofstadter has more to say about this (commenting on the wasp):

"This is a rather shocking revelation of the mechanical underpinning, in a living creature, of what looks like quite reflective behaviour. There seems to be something supremely unconscious about the wasp's behaviour here…I propose to call this quality sphexishness, and its opposite antisphexishness, and then I propose that consciousness is simply the possession of antisphexishness to the highest possible degree. They are two extremes along a continuum."

He then gives multiple different examples along both sides of the spectrum: on one side, the Sphex wasp itself; in the middle, slot machine addicts; towards the higher end, mathematicians who use the same technique over and over, making advances in many different fields despite relying on the "same old trick" (the path integral comes to mind); and social trends which people blindly follow.

Clearly, it's very possible for humans to exhibit sphexish behaviour (as mentioned above); does this mean we are, occasionally, not conscious? I don't think so, but it does show that even the benchmark for consciousness - the human mind - is fallible to getting stuck in constantly repeating patterns, which, to the outsider, seem very mechanical. Clearly, such occasional mechanical blunders must then be forgivable for a computer!

But what sets the mechanical apart from the 'non-mechanical'?

"I would summarize it by saying that it is a general sensitivity to patterns…the possession of this ability to break out of loops of all sorts seems the antithesis of the mechanical. Or, to put it another way around, the essence of the mechanical seems to be in its lack of novelty and its repetitiveness…this is why the wasp, the dog, even some humans seem so mechanical…an antisphexish being watches its own high-level patterns."

It's impossible for me to imagine what someone like Lee Sedol thinks when playing, but this description is exactly what AlphaGo was doing during its game! Analyzing its own patterns (quite literally, patterns on the Go board), it was able to learn from its mistakes and become an incredibly skilled program.

I think it is fair to say that creativity, in and of itself, is the ultimate marker of consciousness and humanity; even more so when this is at the highest level of skill. Clearly, AlphaGo has demonstrated deep ingenuity and creativity, and I think this qualifies it as an extremely primitive, but yet, in some form, conscious entity. Perhaps like a very specialized amoeba?

The difference between something like AlphaGo, for example, and Deep Blue is Deep Blue's brute force approach: the program did not stop, reflect, analyse its position. Even though it stumped Kasparov and was an incredible achievement, it doesn't seem, to me, that Deep Blue came anywhere close to exhibiting the same level of creativity and intuition which AlphaGo managed to capture in its games.

On the other side of the AI argument against Hofstadter, there is the Lucas-Penrose line of arguments, which basically states that since a machine is unable to come up with something like Gödel's theorem, whereas a human mind can, there must be something fundamentally different between the two. Penrose takes this argument further and begins to argue for there being a quantum physical basis for the mind, meaning that any simple computer, no matter how powerful, would be unable to achieve general intelligence - unless it is a quantum computer. This is explored in Penrose's book The Emperor's New Mind, which was the second big work on the philosophy of mind I came across after reading GEB. I don't want to go further into the Lucas-Penrose counterargument in this post, but maybe will in a sequel - for now, this idea is enough for our purposes, which is to illustrate the fact that no one knows what AlphaGo really is and how powerful it can become, whether that means being close to intelligence, or not at all, or perhaps AI will never go that far.

As much as anyone, I'd like to believe that there is something inherently special about the human mind, and especially its ability to create beauty: however, something like AlphaGo makes me really question that.

On one hand, this is extremely depressing: there is nothing particularly unique about the human condition, we are just biological robots running on very simple formal rules.

To be honest I don't really buy this, though. I think the contrary is even more important: even if there is nothing particularly special about the makeup of the human body, mind, consciousness, this assembly still somehow gives rise to love, joy, and the entire spectrum of boundless human experience, which is universally felt to be mysterious and incredibly powerful. The potential for our collective "special" experience to be realised inside a silicone mind doesn't make our minds any less than amazing; if anything, it shows how much beauty can emerge from simple circuits. AlphaGo helps to beautifully illustrate this: although it beat two thousand years of collective human knowledge, it was also created by humans, and therefore is still, fundamentally, the product of human ingenuity.

I am also deeply moved by the seemingly endless elegance of Go. This is the oldest board game in the world, and AlphaGo helped players realise that our thousands of years of study came nowhere close to discovering true creativity in the game. Do you see the pattern? From simple rules emerges endless beauty. Emmanuel Lasker, a former world chess champion and professional mathematician, spoke on the virtue of Go:

"While the Baroque rules of Chess could only have been created by humans, the rules of Go are so elegant, organic, and rigorously logical that if intelligent life forms exist elsewhere in the universe, they almost certainly play Go."

Going back to the very beginning of the post, I mentioned that exhibition of mastery in some specific forms of art can be considered as an example of enlightenment, or absolute freedom from self. By this reasoning, isn't is possible to expect an AGI to one day be something like a Bodhisattva? A few years ago I was in a math camp where I met someone who genuinely believed this to be the case - he was a very serious Tibetan Buddhist, and considered Maitreya to actually be the first incarnation of AGI. It's also worth noting that Sam Altman took the Bodhisattva vows with Jack Kornfield less than a year from the writing of this post.

This is a stark contrast to the standard "paper clip maximiser AI" which has been often touted as the exemplary dangerous AI. But I do think it's a plausible scenario, and one which is perhaps significantly closer than we realise.

I don't remember where I read or saw this, but I once saw a quote along these lines:

Who is more talented? Leonardo, or the man who copies him so well, you'd never realise he's a fake?

Art is always plagiarism or revolution, after all.