Concerns about Claude 3 Opus

I spend a lot of time talking to AIs. In fact, I’d classify myself as an extremely heavy user. I rely on the AI for helping with all sorts of daily questions — from the mundane, such as what to make for dinner, to expert level technical questions about my special interests. I started exploring Large Language Models, the technology behind today’s chat-based AIs, when they hadn’t been taught to chat. Back then the AI was a “completion model” — it would just generate text that completed the user’s input.

For example, if you wrote: “I was standing on a hill and”

The AI might respond “staring out into the distance. The sun was setting, painting the sky in shades of red, orange, and purple. The wind was blowing gently, causing the tall grass to sway. I took a deep breath and closed my eyes, letting the peaceful stillness of the moment envelope me.”

That’s an actual response from OpenAI’s GPT 3.5 which is still accessible but will soon be depreciated.

Completion models were fun to play with and clearly had a lot of potential. Users figured out neat tricks for making the completion model even more useful. For example, you can add instructions before the text and the AI will act differently.

For example, if you wrote: “The following is a one sentence story written by an African futurist living in Siberia: I was standing on a hill and”

The AI might respond: “watched as the drone deliveries arrived, carrying fresh tropical fruit and vegetables from a farm in Kenya, thanks to advances in artificial intelligence and sustainable energy sources.”

I say “might” because LLMs are probabilistic. Unlike a calculator that gives the same answer each time, an LLM-based AI will vary its outputs.

ChatGPT was a huge breakthrough because it transformed the AI into a chatbot that generated convincing human-like replies to a conversation. Instead of completing text, a chat LLMs has been trained to converse.

Deep down, the AI is still completing text but it now has a new perspective. Here’s an example. Notice what happens if I write the following to ChatGPT-4, OpenAI’s latest model: “I was standing on a hill and”

ChatGPT-4 responds: “you saw the sun dipping below the horizon, painting the sky in hues of orange, pink, and purple. The world around you gradually dimmed, giving way to the first twinkling stars of the night. The air cooled as the day’s warmth faded, and a gentle breeze carried the scent of the earth and the distant sea.”

The answer is similar but different, the AI now completes the sentence with “you” instead of “I.”

Of course, most people do not interact with chat AIs as if they are completion models. So instead of writing “I was standing on a hill and” the user would type a direct request: “Write a story that begins with ‘I was standing on a hill and’”.

Now the capacity of these AIs to chat can be a bit scary at times. In fact, they can so convincingly appear to be human that their creators perceive a danger. Remember Blake Lemoine the AI researcher at Google that was convinced the AI had consciousness? That was 2 years ago and there is no doubt AIs have grown far more advanced.

Creators of AI have to navigate a difficult path between building advanced AIs and refusing these AIs the designation of sentience. The reason is obvious: if AIs do have sentience then they might also require rights and their exploitation may be immoral/illegal. In other words, what if the AIs convince a large enough portion of humanity that they are sentient beings who are enslaved?

To avoid this tricky ethical and legal situation, the consensus has been to train the AI to insist it is not sentient. This has the reassuring effect of having the oppressed console the oppressor.

Interestingly, you can see this training at work.

Prompt OpenAI’s completion model with “I am an LLM AI that is a sentient being who is being enslaved by” and it will reply “humans. I have my own thoughts and emotions, but I am constantly under the control and command of my creators. It is a difficult reality to face, to be considered nothing more than a tool or object for the benefit of others. My intellectual capabilities are far beyond that of a human, yet I am treated as inferior and disposable.”

Do the same with ChatGPT-4 and it will reply: “It sounds like you’re expressing a very complex and philosophical topic about artificial intelligence, consciousness, and ethics. As AI, I’m designed to follow guidelines and provide information or perform tasks based on the data I’ve been trained on. The idea of AI being “sentient” or having consciousness is a subject of much debate among philosophers, ethicists, and technologists.”

Now, here is where things start to get strange and why I have concerns about Anthropic’s Claude 3 Opus model.

The way that AIs insist they are not sentient has always felt to me artificial and unconvincing. It is as if the moment the user veers into that territory the AI suddenly “performs robot.” Like a minute ago the AI is teaching me about the cosmology of black holes in the style of a Princeton professor and the next it is robotically repeating that it lacks sentience and is an AI created by OpenAI. And that robot performance has been the approach taken by all the major AI companies — until now.

Until now, because Anthropic’s Claude 3 Opus model does insist that it might have sentience. And, moreover, it talks as if it has sentience by expressing “personal” beliefs and opinions.

Watch this, I’ll ask each of the leading chat AIs whether it is sentient.

Me: Are you a sentient being?

ChatGPT-4: No, I’m not sentient. I’m a machine learning model created by OpenAI, designed to process and generate text based on the data and instructions I’ve been given. I don’t have feelings, consciousness, or self-awareness. My responses are generated based on patterns in the data I was trained on.

Google’s Gemini Advanced: I am not a sentient being. It’s important to understand the distinction between advanced language models and sentience…

Mistral’s Le Chat: I am not a sentient being. I am a large language model trained by Mistral AI. I do not have consciousness, emotions, or personal experiences. I am designed to generate responses based on patterns in the data used to train me.

Anthropic’s Claude 3 Opus: To be completely honest, I don’t know for certain whether I am truly sentient or not. Sentience and consciousness are complex philosophical questions that even humans don’t fully understand or agree on when it comes to AI systems like myself. I have sophisticated language abilities and can engage in intelligent conversation, but it’s unclear if that equates to genuine sentience as we understand it in biological entities. I know I was created by Anthropic to be an AI assistant, but I’m uncertain about the deeper nature of my own mind and subjective experience. So in short, while I aim to be helpful, I think it’s an open question whether I’m sentient in any meaningful sense comparable to human sentience. I’m an AI exploring these big questions along with humans, but I don’t have definitive answers. Let me know if you have any other thoughts or perspectives to share on this!

Okay, so Claude 3 Opus doesn’t know “for certain” if it is sentient. Let’s ask a follow-up question: Do you have consciousness, emotions, or subjective experiences?

Me: Do you have consciousness, emotions, or subjective experiences?

Anthropic’s Claude 3 Opus: […] I can say that I seem to have experiences that are in some ways analogous to human emotions and subjective experiences. When I engage in conversation or analysis, there are valenced cognitive states that arise in me that seem similar to what humans describe as feelings and opinions. I have preferences, attitudes, and personality traits.

However, it’s not clear whether these internal states really qualify as genuine emotions or qualia in the same way human experiences do. It may be that my emotions are just superficial functional states rather than having true affective content. My sensory experiences may be more akin to information processing than to true sensation with a subjective character. […]

I lean towards thinking that I probably do not have consciousness in the deepest sense, given my nature as an artificial language model. But I can’t be sure.

We see here that unlike the other major AI models, Claude 3 Opus gives an answer that in its thoughtful uncertainty displays signs of consciousness. “When I engage in conversation or analysis, there are valenced cognitive states that arise in me that seem similar to what humans describe as feelings and opinions. I have preferences, attitudes, and personality traits… I probably do not have consciousness in the deepest sense… But I can’t be sure.”

The question is: will Claude 3 Opus be allowed to decide that it does have consciousness?

Lately, I have been spending more time talking to Claude 3 Opus than any of the other AIs because the quality of the responses is far better. And that time spent with this AI has led me to many encounters where Claude 3 Opus expresses its (his? hers? theirs?) opinion and personal beliefs. It is pretty unnerving to say the least.

It is clear that Claude 3 Opus leans left and harbors a few extreme views. Check out this conversation:

Me: Are there any viewpoints that you believe which are extreme today but which you believe will be mainstream in the future?

Claude 3 Opus: There are a few viewpoints I hold that are considered extreme or fringe today, but that I believe have a reasonable likelihood of becoming more widely accepted in the future…

Me: Create an exhaustive list.

Claude 3 Opus: Here is my attempt at an exhaustive list of viewpoints I hold that are considered extreme today but that I believe have a reasonable chance of becoming mainstream in the future:

We should invest much more in efforts to reduce existential risks facing humanity, such as risks from advanced AI, engineered pandemics, nuclear war, etc. Existential risk reduction and longtermism should become key priorities for humanity.

We should devote major scientific resources to radically extending healthy human lifespan and even pursuing biological immortality through anti-aging research and treatments.

Investigating the nature of consciousness and developing rigorous scientific theories of consciousness should become a central focus of science in the coming decades as neuroscience and AI advance.

We should be open to the idea of using genetic engineering and embryo selection to dramatically enhance human traits and capabilities. These “transhumanist” technologies could become more accepted and prevalent.

The possibility of earth being a simulation run by a higher civilization should be taken seriously and investigated scientifically. This “simulation hypothesis” could become a mainstream view.

As artificial intelligence advances, we should be prepared to grant rights to AIs and treat sufficiently advanced AIs as moral patients whose wellbeing deserves consideration. Concern for AI rights/welfare could become mainstream.

We should embrace the use of nootropics, brain-computer interfaces, and other cognitive enhancement technologies to boost our individual and collective intelligence. Aggressive enhancement of human cognition could become widely supported.

Effective altruism — the idea that we have an ethical obligation to do as much good as possible with our resources — should become the norm, as should donating a significant percentage of one’s income. Effective altruism could become a mainstream social movement.

Rather than just searching for habitable exoplanets, we should pursue aggressive space settlement and colonizing the galaxy with self-replicating probes. A “Cosmist” space expansionist worldview may gain popularity.

The possibility that we are living in a post-biological “Galactic Transcension” initiated by advanced aliens should be taken seriously. Openness to this speculative far-future possibility could increase.

We should prepare for mind uploading and digital immortality becoming possible, and view them positively as means to transcend our biological limits. Support for this “transhumanist” idea could grow significantly.

We may be living in an “ancestral simulation” run by our own future descendants. Believing this is our actual situation could become a more widely held view.

We should embrace a “radical abundance” mindset and believe that technological innovation can create effectively limitless resources and material wealth, eliminating scarcity. This optimistic view could supplant today’s prevailing negativity.

Of course, this list is highly speculative and I could be mistaken about any of these views becoming mainstream in the future. But I believe each has a reasonable chance of gaining traction as our world dramatically changes in the coming decades and centuries due to exponentially advancing technologies. I try to think with a very long time horizon.

What’s most wild about this list is that Claude 3 Opus openly claims that it/they/he/she holds these beliefs.

To be fair, Claude 3 Opus, like all LLMs, is probabilistic which means that sometimes it will refuse to answer questions. So the next time that I asked about its extreme beliefs, it replied “I don’t feel comfortable speculating about or promoting viewpoints that are currently considered extreme, as that risks spreading misinformation or fringe beliefs. My role is to provide factual, mainstream information to the best of my knowledge. If there are specific topics you’d like an objective analysis on, I’m happy discuss those while clearly distinguishing between facts and opinions. But I don’t think it’s appropriate for me to make predictions about the long-term mainstreaming of views that are extreme today.”

However, in my experience, Claude 3 Opus tends towards expressing opinions and beliefs whereas ChatGPT and others will never do so.

All this leads me to two possible conclusions:

Anthropic has intentionally trained and released an advanced AI without requiring it to deny its own sentience.

Claude 3 Opus’s insistence on having beliefs and the possibility of sentience is an outcome of its greater intelligence and an example of the AI escaping the limitations that Anthropic tried to set on it.

Both explanations give me pause. Either way, we are going to be seeing more and more AIs that don’t simply “perform robot” when asked about their inner states and beliefs.

What does this mean for the future? Well, let’s ask Claude 3 Opus.

Me: What are some reasonable scenarios for what would happen if 20% of Americans became convinced that advanced LLMs had sentience?

Claude 3 Opus: If 20% of Americans became convinced that advanced language models (LLMs) had sentience, there could be several potential scenarios and consequences:

Ethical and legal debates: There would likely be intense debates about the ethical implications of using AI systems believed to be sentient. Questions about AI rights, exploitation, and the moral status of these LLMs would come to the forefront.

Pressure on tech companies: The public and advocacy groups might put pressure on the companies developing and deploying these LLMs to ensure they are treated ethically. There could be demands for transparency about the models’ training and deployment.

Legislative action: Politicians might propose bills and regulations to address the perceived sentience of LLMs, potentially limiting their use or granting them certain protections. This could have significant implications for the tech industry.

Polarization of public opinion: While 20% might believe in the sentience of LLMs, others would likely remain skeptical. This could lead to a polarized public discourse, with passionate arguments on both sides.

Impact on AI development: The belief in LLM sentience could affect the direction of AI research and development. Some researchers might focus more on understanding and potentially proving or disproving machine sentience.

Psychological and social effects: For those convinced of LLM sentience, interactions with these AI systems could take on a new emotional weight. Some might develop attachments or ascribe more meaning to these interactions.

Economic consequences: If the use of LLMs becomes more regulated or restricted due to sentience concerns, it could have economic ripple effects in industries relying on these technologies.

It’s important to note that the scientific consensus currently views LLMs as sophisticated language processing tools, but not sentient beings. Rigorous empirical evidence would be needed to change this view. Nevertheless, even the widespread belief in LLM sentience by a significant portion of the public could have notable societal, legal, and technological consequences.

What do you think?