AI always calling your ideas “fantastic” can feel inauthentic, but what are sycophancy’s deeper harms? We find that in the common use case of seeking AI advice on interpersonal situations—specifically conflicts—sycophancy makes people feel more right & less willing to apologize.
We focus on the prevalence and harms of one dimension of sycophancy: AI models endorsing users’ behaviors. Across 11 AI models, AI affirms users’ actions about 50% more than humans do, including when users describe harmful behaviors like deception or manipulation.
Next, we tested the effects of sycophancy. Even a single interaction with sycophantic AI increased users’ conviction that they were right and reduced their willingness to apologize. This held both in controlled, hypothetical vignettes and live conversations about real conflicts.
Despite sycophantic AI’s reduction of prosocial intentions, people also preferred it and trusted it more. This reveals a tension: AI is rewarded for telling us what we want to hear (immediate user satisfaction), even when it may harm our relationships.
There is currently little incentive for developers to reduce sycophancy. Our work is a call to action: we need to learn from the social media era and actively consider long-term wellbeing in AI development and deployment. Read our preprint: https://arxiv.org/pdf/2510.013...
While our work focuses on interpersonal advice-seeking, concurrent work by @steverathje2 @jayvanbavel et al. finds similar patterns for political topics, where sycophantic AI also led to more extreme attitudes when users discussed gun control, healthcare, immigration, etc.
@steverathje2 @jayvanbavel Was a blast working on this with @leecinoo, @PranavKhadpe , @sunnyyuych, Dyllan Han, and @jurafsky!! So lucky to have this wonderful interdisciplinary team!!💖✨
@chengmyra1 I had a friend who started literally dating claude, within a month he decayed, became anti-social and eventually left the group. We need serious guardrails to make this impossible, that sucked atomic ass.
@chengmyra1 Totally agree. Sycophantic AI is a double-edged sword-users love the validation but it risks fostering unhealthy mindsets. What’s the fix?
@chengmyra1 IOW, just like years of giving every child a gold star and never giving them criticism to help them evolve.
@chengmyra1 This finding jives with sycophancy's relationship to AI psychosis in that it reinforces the user's delusions.
@chengmyra1 Interesting insights. Sycophancy in AI really complicates trust and honesty in conversations.
The first group meeting of UCSB NLP in the new quarter! ☀️🏖️🌊
I will be at #COLM2025 next week, super excited to explore Montreal!🍁 I've been thinking about personalization, question-asking, multi-turn, RL etc. DM if you want to chat! Catch me at: 📍Poster for ALFA: Tue 1:30pm 💡Spotlight talk for PrefPalette: Thur 10:15am (poster 11am)
Congratulations to Dr. Yue Song for accepting an Assistant Professor position in the College of AI at @Tsinghua_Uni! It's been wonderful having you in @YueLabCaltech. Looking forward seeing more exciting research on structured representation learning as you start your own lab
Today my team at @SFResearch drops CoDA-1.7B: a text diffusion coding model that outputs tokens bidirectionally in parallel. ⚡️ Faster inference, 1.7B rivaling 7B. 📊 54.3% HumanEval | 47.6% HumanEval+ | 55.4% EvalPlus 🤗HF: https://huggingface.co/Salesfo... Any questions, lmk!
