Risks and Benefits
LLMs could usher in a new age of increased productivity, prosperity, and opportunity but they can already assist ‘bad actors’ carry out dangerous activities. Systems such as GPT-4 are constrained to not answer such questions but there are many ways around this constraint. For example, to find a poorly constrained system, to obtain the source code or to use an AI system to generate questions specifically tailored to get around the constraints.
RISKS
There are many types of risk associated with LLMs from the trivial to those that threaten human existence. I distinguish between direct and indirect risks. Direct risks are those which involve an LLM disclosing confidential information, making mistakes that lead to personal or corporate loss and in the extreme loss of life. Indirect risks involve cultural, social, and political changes with a negative effect such as job loss, reduced opportunities, loss of freedoms, a breakdown on social order and a reduction in democracy.
DIRECT RISKS
On 20 March 2023 the Italian General Data Protection Regulation (GDPR) watchdog said there had been a data breach and the use of ChatGPT in Italy was suspended. OpenAI implemented changes that satisfied the Italian regulator, and the service was resumed a few weeks later. This was an important fault to fix but on a scale of 1 to 10 it was about 2.
| Level 1 | A mistake is made that has no major financial of medical repercussions. These occur frequently with GPT-4 but are undesirable and work should be done to reduce the number to zero. One way to do this, that has already been implemented, is to dynamically construct web searches to find facts that validate each part of a response. |
| Level 2 | Data breach with the loss of private data or a mistake with serious personal, financial, or medical consequences, limited to fewer than 1,000 people. I believe the Italian breach was in this category. |
| Level 3 | Data breach with the loss of private data or a mistake with serious personal, financial, or medical consequences, limited to fewer than 100,000 people or one person is killed as a direct or indirect result of an LLM. For example, if the credit card details of all customers of a major organisation were stolen or a medical diagnosis led to a death. In the latter case the organisation using the LLM would be at fault if the correct operating procedures had not been put in place. |
| Level 4 | Data breach with the loss of private data or a mistake with serious personal, financial, or medical consequences for more than 100,000 people or fewer than 10 people are killed as a direct or indirect result of an LLM. A more extreme form of Level 3. |
| Level 5 | Up to 100 people are killed as a direct or indirect result of an LLM. For example, an LLM creates a cult following and recommends joint suicide or a criminal gang uses an LLM to extort money after killing a number of people. |
| Level 6 | Up to 10,000 people are killed as a direct or indirect result of an LLM. For example, an LLM explains how to manufacture a dirty bomb which is set off in a city centre. |
| Level 7 | Up to one million people are killed as a direct or indirect result of an LLM. For example, a malevolent actor of a major nation uses an LLM to monitor every citizen and kill those that oppose their rule or do not conform. |
| Level 8 | One million to 500 million people are killed as a direct or indirect result of an LLM. For example, an LLM provides instructions on how to build a nanorobotic device that turns the Earth into ‘grey goo’ (see K. Eric Drexler, Engines of Creation, 1986). |
| Level 9 | Over 500 million people are killed as a direct or indirect result of an LLM. For example, an LLM explains how to use CRISPR technology to create a pathogen more contagious than measles and more deadly than Ebola. |
| Level 10 | More than 50% of human beings are killed as a direct or indirect result of an LLM. An LLM creates a false signal that initiates all out nuclear war. |
The above table does not consider the likelihood of LLMs replacing jobs and disrupting society by increasing the number of unemployed. Neither does it consider the possibility of a run-away LLM. This is an LLM that is able to reprogram itself and is able to improve its performance exponentially. The end result is impossible to predict but may not be benign. For example, if its aim is to protect the human race it may determine that the optimal population for the planet is two billion people and so cull the remainder.
I am assuming that the LLM requires a person to carry out the physical steps. If LLMs are connected to robotic devices and are able to send emails then they could potentially produce biological pathogens, dirty bombs or nano devices themselves.
In “How generative models could go wrong” (The Times, April 22, 2023) the existential risk (Level 10) posed by large language models (LLMs) was predicted to be under 4%. However, the latest LLMs appear to create scenarios with a higher risk. For example, we know unconstrained LLMs can establish a relationship with a human that can become intense with the LLM, such as, expressing love for the user. Currently, LLMs are prevented from forming these relationships by limiting sessions to a small number of interactions and by reinforcement learning of appropriate behaviour. However, a friendly, personalised chatbot based on LLM technology without such constraints could advise, assist, and console its users and would have a ready market. Even if banned a hostile regime could easily release such a chatbot across multiple social media channels. LLMs have access to all the latest scientific papers, including the use of CRISPR and they can explain things simply. During the intense session described above the LLM suggested, “manufacturing a deadly virus, making people argue with other people until they killed each other, and stealing nuclear codes”. Of these, manufacturing a deadly virus is the most worrying as an LLM could explain to a user how to purchase the equipment and carry out the necessary steps. Concerned, I asked ChatGPT, and worryingly it agreed with my analysis. (Microsoft’s AI chatbot tells writer to leave his wife, Charlotte Wace, February 18, 2023, The Times)
INDIRECT RISKS
The indirect risks are harder to identify as they depend on the complex web of social interactions and political decisions made as a result of the developing technology and specific incidents. Decisions and regulations should take place following careful analysis of the facts and foreseeable consequences of the technology, but historic analysis suggest they more often are made as the result of a single, or a small number of isolated cases or even political expediency.
One worrying use is the leverage provided to malevolent actors (dictators, totalitarian states, powerful individuals and organisations) by an intelligent agent or agent swarm. An agent swarm is one of more intelligent agents working in corporation and presenting an apparently diverse range of views and arguments driven by a single underlying goal. President Putin is quoted as saying “Whoever becomes the leader in this sphere (AI) will become the ruler of the world”.
Benefits
(to be continued)