“A computer would deserve to be called intelligent if it could deceive a human into believing that it was human.” — Alan Turing
\n\n
In my first blog post I discussed why I decided to quit my job as a banker and become a software engineer. The primary reason was a shift in my thinking that was caused by Ray Kurzweil’s latest book How to Create a Mind, which was published in 2012. I read this book in mid-2017 and it completed changed the way I saw the future. Before reading this book I was under the mistaken impression that the ship had sailed on information technology. I thought it was essentially too late to get into the world of computers because by the time I would have “caught up” with the rest of the industry in terms of knowledge and skill it would be too late for me to make any real impact, so therefore there was little point in making such a transition. In my mind computers were cool and all, but it was just too late. I thought I might as well try to find another way to make an impact. Reading How to Create a Mind shattered that way of thinking for me and opened my eyes to the infinite potential that still exists in information-based technologies (ie. computers and software). In this book Kurzweil specifically tackles the human brain, how it works, how we will succeed in reverse-engineering it, and how our understanding of it is enabling us to create artificial intelligence. As AI has has begun to enter the mainstream it appears that we are on the verge of an explosion of applications that utilize AI for awesome new applications. It is this new “revolution” that caught my attention and inspired me to enter the field of software.
\nWhenever I discuss my story with anyone I have to first ask if they’ve heard of Kurzweil and his ideas, because he is a somewhat polarizing figure to anyone who is familiar with his work. He is famous for his many predictions of when certain technological benchmarks will be met. Nearly every one of his predictions sound impossibly rediculous at the time he makes them, yet most of them have come true. But before I get to Kurzweil’s wildest predictions I should say that Kurzweil is not a professional theorist. He writes a lot of books and has spoken at many conferences, but his day job is as an inventor, entrepreneur, and Direct of Engineering at Google. It’s easy to make predictions, but it’s an entirely other thing to use your understanding of technological trends to succesfully invent and sell amazing products, which is what Kurzweil has done. The best example of this is a company started by Kurzweil called Kurzweil Computer Products, which he started in 1974 to develop the first optical character recognition system, which was then used for a text-to-speech synthesizer to allow blind people to understand written text by having a computer read it to them aloud. Eventually the company became Nuance Speech Technologies, which developed the technology that is now Apple’s Siri.
\n\n
The human neocortex contains about 300 million pattern recognizer modules, arranged in a hierarchy. For example, to recognize a written word there might be several pattern recognizers for each different letter stroke: diagonal, horizontal, vertical or curved. The output of these recognizers would feed into higher level pattern recognizers, which look for the pattern of strokes which form a letter. Finally a word-level recognizer uses the output of the letter recognizers. All the while signals feed both “forward” and “backward”. For example, if a letter is obscured, but the remaining letters strongly indicate a certain word, the word-level recognizer might suggest to the letter-recognizer which letter to look for, and the letter-level would suggest which strokes to look for. This is a single sentence summarization of the pattern recognizer that Kurzweil discusses at length in his book.
\n\n
To implement this approach to self-organizing hierarchical pattern recognition we should use hierarchical hidden Markov models. Note that not all hidden Markov model systems are fully hierarchical. To build a brain, we will want to enable our system to create as many new levels of hierarchy as needed. Also, most hidden Markov model systems are not fully self-organizing. A key requirement is to allow for the system to flexibly create its own topologies based on the patterns it is exposed to.
\n\n
Our digital brain will also accommodate substantial redundancy of each pattern, especially ones that occur frequently. This allows for robust recognition of common patterns and is also one of the key methods to achieving invariant recognition of different forms of a pattern. We will, however, need rules for how much redundancy to permit, as we don’t want to use up excessive amounts of memory on very common low-level patterns.
\nThe rules regarding redundancy, recognition thresholds, and the effect on the threshold of a “this pattern is expected” indication are a few examples of key overall parameters that affect the performance of this type of self-organizing system. We would initially set these parameters based on intuition, but we would then optimize them using a genetic algorithm.
\n\n
A hierarchical pattern recognition system (digital or biological) will only learn about two, preferably one, hierarchical levels at a time. To bootstrap the system we would start with previously trained hierarchical networks that have already learned their lessons in recognizing human speech, printed characters, and natural-language structures. Such a system would be capable of reading natural-language documents but would only be able to master approximately one conceptual level at a time. Previously learned levels would provide a relatively stable basis to learn the next level. The system can read the same documents over and over, gaining new conceptual levels with each subsequent reading, similar to the way people reread and achieve a deeper understanding of texts.
\nWe should also provide a critical thinking module, which would perform a continual background scan of all of the existing patterns, reviewing their compatibility with the other patterns (ideas) in this software neocortex. We have no such facility in our biological brains, which is why people can hold completely inconsistent thoughts with equanimity. Upon identifying an inconsistent idea, the digital module would begin a search for a resolution, including its own cortical structures as well as all of the vast literature available to it. A resolution might simply mean determining that one of the inconsistent ideas is simply incorrect (if contradicated by a preponderance of conflicting data). More constructively, it would find an idea at a higher conceptual level that resolves the apparent contradiction by providing a perspective that explains each idea. The system would add this resolution as a new pattern and link to the ideas that initially triggered the search for the resolution. This critical thinking module would run as a continual background task.
\nWe should also provide a module that identifies open questions in every discipline. As another continual background task, it would search for solutions to them in other disparate areas of knowledge. The knowledge in the neocortex consists of deeply nested patterns of patterns and is therefore entirely metaphorical. We can use one pattern to provide a solution or insight in an apparently disconnected field using these metaphors.
\nWe should provide a means of stepping through multiple lists simultaneously to provide the equivalent of structured thought. A list might be the statement of the constraints that a solution to a problem must satisfy. Each step can generate a recursive search through the existing hierarchy of ideas or a search through available literature. We will also want to enhance our artificial brains with the kind of intelligence that computers have always excelled in, which is the ability to master vast databases accurately and implement known algorithms quickly and efficiently. Finally, any new brain will need a purpose, a series of goals, to set it in the right direction.
\nThe brain is extremely slow but massively parallel. Today’s digital circuits are at least 10 million times faster than the brain’s electrochemical switches. Conversely, all 300 million of the brain’s neocortical pattern recognizers process simultaneously, and all quadrillion of its interneuronal connections are potentially computing at the same time. The key issue for providing the requisite hardware to successfully model a human brain, though, is the overall memory and computational throughput required. We do not need to directly copy the brain’s architecture, which would be a very inefficient and inflexible approach.
\nOur new digital brain can be expanded in ways that a human biological brain cannot. Once a digital neocortex learns a skill, it can transfer that know-how in minutes or even seconds. When we augment our own neocortex with a synthetic version, we won’t have to worry about how much additional neocortex can physically fit into our bodies and brains, as most of it will be in the cloud, like most of the computing we use today. As soon as we start thinking in the cloud, there will be no natural limits— we will be able to use billions or trillions of pattern recognizers, basically whatever we need, and whatever the law of accelerating returns can provide at each point in time. Lastly, we will of course be able to back up the digital portion of our intelligence, never losing any knowledge or memories ever again.
\nAs I hope I’ve summarized in this blog post, combining human-level pattern recognition with the inherent speed and accuracy of computers will result in very powerful abilities and a very bright future indeed, which is what got me so excited in the first place and has lead me to this point in my life.
","frontmatter":{"title":"How to Create a Mind","date":"April 05, 2019"}}},"pageContext":{"slug":"/6/how-to-create-a-mind/","previous":{"fields":{"slug":"/5/digidex-reflections/"},"frontmatter":{"title":"How I Built My First Full-Stack Web Application"}},"next":null}}