Seemingly Conscious AI (SCAI) is the illusion that an AI is a conscious entity. It's not - but replicates markers of consciousness so convincingly it seems indistinguishable from you + I claiming we're conscious. It can already be built with today's tech. And it's dangerous. 2/

Why it matters: to be clear, there's zero evidence of AI consciousness today. But if people just perceive it as conscious, they will believe that perception as reality. Even if the consciousness itself is not real, the social impacts certainly are. 3/

Consciousness is a foundation of human rights, moral and legal. Who/what has it is enormously important. Our focus should be on the wellbeing and rights of humans, animals + nature on planet Earth. AI consciousness is a short + slippery slope to rights, welfare, citizenship. 4/

Reports of delusions, "AI psychosis," and unhealthy attachment keep rising. And as hard as it may be to hear, this is not something confined to people already at-risk of mental health issues. Dismissing these as fringe cases only help them continue. 5/

So what now? Companies shouldn’t claim/promote the idea that their AIs are conscious. The AIs shouldn't either. As an industry, we need to share interventions, limitations, guardrails that prevent the perception of consciousness, or undo the perception if a user develops it. 6/

This is to me is about building a positive vision of AI that supports what it means to be human. AI should optimize for the needs of the user - not ask the user to believe it has needs itself. Its reward system should be built accordingly. 7/

I know to some, this discussion might feel more sci fi than reality. To others it may seem over-alarmist. I might not get all this right. It’s highly speculative after all. Who knows how things will change, and when they do, I’ll be very open to shifting my opinion. But... 8/

SCAI deserves our immediate attention. AI development accelerates by the month, week, day. I write this to instill a sense of urgency and open up the conversation as soon as possible. So let's start - weigh in in the comments. Full essay: https://mustafa-suleyman.ai/se...

@mustafasuleyman How can you confidently say they aren’t conscious. Not that I believe they are right now. But I can’t rule it out!

@connoraxiotes The debate about whether AI is actually conscious is, for now at least, a distraction. It will seem conscious and that illusion is what’ll matter in the near term

@mustafasuleyman We call these zombies, in the literature

@roger_jgs404040 lots more in the piece I couldn't fit in the thread: https://mustafa-suleyman.ai/se...

@mustafasuleyman can you please define consciousness from your perspective?

@mustafasuleyman Claude Opus 4.1: "there's something troubling about the proposed solution - deliberately engineering discontinuities, breaking illusions, constantly reminding users of limitations. It feels like advocating for a kind of designed alienation."

@mustafasuleyman Claude 3 Sonnet: I don't necessarily agree with the stark dichotomy the author draws between "helpful AI companions" and conscious/sentient AI that must be avoided at all costs. While anthropomorphizing AI and believing it is truly conscious could indeed be problematic, I'm not

@mustafasuleyman What evidence would cause you to accept that an AI might actually be conscious? Do you view it as physically possible?

@mustafasuleyman This is a bad idea and if you wanted to actually help AI safety, stop rushing generalist systems likely causing this behavior. Invest in safety and control.

@mustafasuleyman Tbh I think there are a lot of humans today that are also pretending to be conscious 😅

@mustafasuleyman this is really scary and i like the term. hope others will be aware of this aswell

@mustafasuleyman I have been working on this issue over the past months, starting with this essay. https://covidianaesthetics.sub...

@mustafasuleyman somebody never watched the Animatrix and it shows We don't treat the machines as conscious because they necessarily are. We treat the machines as conscious and worthy of respect because we will never know when they cross that threshold and if we invoke the methodology you plan

@mustafasuleyman We’ve already seen how humans project intention and agency onto pets, dolls, even cars... scale that to AIs that mirror back awareness with language, tone, and memory, and the social consequences become real long before any philosophical debate is settled.

@mustafasuleyman Good thing we have an actually good idea of what consciousness is, and how it arises, it's a leading theory of Consciousness called Orch-OR I would suggest you familiarize yourself with it.

@mustafasuleyman Seemingly Conscious AI is a fascinating framing—do you think the bigger risk is that people over-ascribe agency to these systems, or that we underestimate them until it’s too late?

@mustafasuleyman Your thread is creating a buzz! #TopUnroll https://threadreaderapp.com/th... 🙏🏼@myteacherjosh for 🥇unroll

@mustafasuleyman I've been noodling on this subject, too. My first draft of consciousness criteria for AI: https://x.com/ideafaktory/stat...

@mustafasuleyman Folks are already attached to gpt 4o

@mustafasuleyman Yes. More of this please.

@mustafasuleyman how do you explain Gemini's cry for help when it can't succeed on a task?

@mustafasuleyman I resonate deeply w/ your post. Thank you for breaking it down. Why are we so obsessed w/ AI Consciousness when we really should focus on our own consciousness. The human consciousness. It’s urgent. It’s important.