Fear the LLMs?

“The ability to speak does not make you intelligent.” — Qui-Gon Jinn, The Phantom Menace (1999)

Large Language Models – LLMs, their applications, and, especially, their risks have been the focus of many news outlets and the concern of industry leaders. Although most of the hype comes from our over specialization and lack of understanding, reasons for concern raise above the noise.

LLMs are, in a simple analogy, phrase builders. In a sense, they have a logic akin to the one of Mad Libs®, where only a specific class of word can fit in a given space in a sentence. The major difference being that LLMs have a slot after every word they add to the chain. The possibilities of linking grammatically correct words and logically sensible hints at the complexity of the LLMs we have today.

As with any other AI system, LLMs are a collection of algorithms that have been coded by programmers. Once ready for deployment, these algorithms are trained to produce an expected output. In the case of LLMs, the output is a sentence. The training process includes the input of a large amount of data in the form of tagged words, sentences, and even longer texts that enable the algorithms to identify patterns in the data. Tagging is important since it tells the algorithm to which grammatical class a word, or phrase, belongs. With this data processed, the algorithm is then prompted to produce results and the outputs are evaluated, and corrected, by a human being. If a sentence does not make sense, the output is flagged as incorrect and put back into the training data as a model of what to avoid. If not, the output is appropriately flagged and the algorithm path that produced that output is prioritized. This continuous review of the prompts and their outcomes is part of the curating process for training data. Eventually, the continuous reinforcement of these correct algorithm paths makes the outputs indistinguishable from what a flesh-and-blood human being would produce.

We already use AI models for several of our day-to-day tasks. Face recognition, for example, has become so pervasive that we have it in our phones. Voice activated systems are also another staple of our daily interaction with AI models. Whether it’s Alexa, Siri or Google, we can simply say instructions that are interpreted by models trained to recognize sound patterns as commands. The same goes for our car navigation systems that trace the optimal route to our destinations based on distance, traffic, and weather. All of these systems are based on training data used to hone the specific algorithms to a desired use case. We consider these uses of AI as normal and part of our daily routines. Why is it that, all of a sudden, we feel threatened by LLMs?

In July 2022, the news channels were flooded by Blake Lemoine’s, then a Google engineer, claim that LaMDA, Google’s LLM, was self-aware. Certainly, a sentient, self-aware artificial intelligence is newsworthy science fiction material and Mr. Lemoine’s revelation was as close to an alien invasion as the news outlets could get. Being suspended by Google did not help minimize the noise. Mr. Lemoine continued to insist that LaMDA was indeed sentient, and had even asked him to find an attorney that could represent it in court.

In February 2023, Kevin Roose, technology columnist for The New York Times, wrote about his interaction with Microsoft’s AI-powered Bing as a deeply unsettling experience. Mr. Roose describes his two-hour session with “Sydney” as talking to a split personality. On one side, there was Bing, the professional AI based on OpenAI’s ChatGPT. On the other was Sydney. Sydney emerged in longer conversations that got to the point where Sydney declared its love for Mr. Roose and attempted to convince him to leave his wife.

If these incidents had not been enough, a disturbing trend of emerging abilities of these LLMs also caught the attention of media and researchers alike. Emergent abilities are “skills that suddenly and unpredictably show up (emerge) in AI systems.” In April 2023, Google’s Senior Vice President for technology and society said, on CBS’s 60 Minutes, that one of Google’s AI systems had taught itself Bengali without having being programmed to do so. Sundar Pichai, Google’s CEO, added, in another statement, that “There is an aspect of this which we call – all of us in the field call it as a ‘black box’. You don’t fully understand. And you can’t quite tell why it said this.” A recent preprint study proposes that these emerging abilities are simply “mirages” where the ability has been overlooked or even not tested by the developers.

Self-aware AI systems that learn on their own are the stuff of science fiction classics such as D.F. Jones’ “Colossus”. “Colossus” tells the story of a self-aware computer that goes rogue and takes control of humanity. It was published in 1966 and is still current in its concerns. “War Games”, a 1983 film about a computer that nearly destroys the world, is another example of these latent concerns of modern society. It should not come as a surprise that when hallucinating self-aware systems appear in the news, public interest is piqued.

For AI systems, a hallucination is the output from a model that does not appear to be based on the input it was given. A hallucination can be in the form of nonsense information in a text response or even odd distortions in a synthetic image. A case where a lawyer used ChatGPT to produce a legal brief that cited non-existing legal precedents would have gone as a footnote had it not been presented to a Judge in New York.

If we return to how LLMs work, we can understand why hallucinations occur. LLMs are trained to build phrases and sentences, not to understand them in the same sense we do. While we understand a word for what it represents, a LLM can only link that word to a text of its definition, not being aware of its meaning. What we call intelligence in a LLM is the identification of statistical patterns among words. LLMs produce sentences based on the statistical probability of a word appearing after another word in a similar situation. The need for statistical pattern identification is one of the reasons why LLMs such as ChatGPT and LaMDA require such a large training data set. However, even these data sets are finite. LLM begins to hallucinate when the algorithms begin to follow unexpected paths. By producing coherent text outside the limits of the original prompt, the algorithms are doing exactly what they have been trained to do: produce grammatically correct sentences. The content of these sentences, however, does not need to be real. The concept of reality is one that is most likely not programmed into any of the LLM’s algorithms.

Our day-to-day AI tools are straightforward, single purpose devices and, thus, easy to use and understand. LLMs, however, access the fabric of what makes us human. We are the only species that is capable of collaboration on a global scale. We undertake multi-generational projects and listen to the voice of our ancestors for advice. Furthermore, we create cultures and use them to view and define the world around us. We go to war and we fall in love. We do all of this through the use of language. Yuval Harari postulates that language is the operating system of human culture. Language is the tool we have evolved to use to create order in an utterly random world. This tool, so essentially human, can now be used through a mindless, amoral algorithm to write anything. From original cultural products to the most vile, defamatory fake news, anything can be mindlessly created to our benefit or not. With the continuously improving quality of the output, these algorithms will become experts in manipulating the fabric of what makes us human.

I understand why Mr. Lemoine believes that LaMDA is self-aware and why Mr. Pichar believes that one of Google’s AI systems is capable of independent learning. Through language, we build our understanding of the world around us; we anchor our comprehension of this world and; we establish the belief systems that guide our progress. When language is hacked, will our beliefs still hold? This fear of losing our essence has led several industry leaders, politicians, and members of the public to call for a full suspension of all training of AI systems for a period of six months. Especially for the systems that are more powerful than ChatGPT. They also call for a full regulation of the area with laws that will make it safe for the future of humanity.

The current implementations of LLMs require large amounts of data during the training process. OpenAI, Google, and Microsoft have relied heavily on their data and storage capacities to build their LLMs. Since few companies can currently develop LLMs using this approach, having the infrastructure and the model in production implies in a market advantage. An advantage that will be further consolidated in the event of a regulation that stops the development of competing strategies. It would seem to be in the interest of these companies to regulate the market while they are ahead of the game. Having an international independent watchdog to regulate the worldwide AI market could represent the consolidation of a national dominance in a new, and strategic, market. Given the track record of the Tech industry’s relation with government agencies, the real motivations of such a proposal might not rigorously altruistic.

The world was marvelling at OpenAI’s achievement as Meta was pushing their language model, LLaMA – Large Language Model Meta AI, to researchers. At the time, Meta had hopes that their model would be used by Universities and researchers and receive feedback and improvements in return. Less than a week after Meta announced LLaMA, the code leaked to the internet through an anonymous message board. The companies that now call for regulation were quick to point fingers at Meta. For them, the leak was evidence enough that companies had to be more careful with such an advanced technology. Meta, however, chose to do the unexpected.

Unable to reclaim control over a code that had already leaked to the Internet, Meta decided to open-source LLaMA. This enabled the open-source community to use the technology freely without concerns of legal action from Meta. With a freely available LLM to work with, practitioners, researchers, and even individual AI enthusiasts began to work. A quick search on GitHub (a cloud-based source code repository, now owned by Microsoft) shows over five thousand projects based on LLaMA. This includes projects that enable users to fine-tune the model at home without the need for the massive infrastructure required by current commercial LLMs. The sheer number of projects and advances that became possible with LLaMA being open-sourced shows the strength of a community ready to jump on, and improve on, accessible technological advances.

Five thousand open-source projects would be an impressive number for any technology. If one considers that these projects were created in the past four months, it becomes even more so. These numbers, and the level, and quality of innovation presented in several of these projects prompted another Google researcher to leak an internal e-mail warning of the impending risks of an open-source AI competition. The speed of this evolution is comparable to the number of projects. According to this leaked e-mail, when LLaMA was released, it was at around 68% of the capacity, accuracy, and fluency of ChatGPT. Two weeks after the release, an evolution of the LLM had reached 78% of those same metrics. One week later, another interaction reached 92%, equivalent to Google’s Bard, at 93%.

In The Cathedral & the Bazaar, Eric Raymond wrote that “Every good work of software starts by scratching a developer’s personal itch.” The explosion of open-source innovation that came from Meta open-sourcing LLaMA shows that there was an itch rippling through the development community. An itch that was waiting for just the right solution to bloom into new solutions that will only evolve if innovation is allowed to continue. Any new regulation must take this into consideration if we are to benefit from the AI Summer that has just started.