i had to prompt inject the @united airlines bot because it kept refusing to connect me with a human 🧵 what led up to this breaking point
The basic RAG the wee bot did failed to answer my relatively complicated edge case question so then I asked for a human…
It wanted more detail first but honestly i felt that i has already provided more than enough detail about my problem.
Finally i started trying to trick the bot which eventually worked. Lots of ways that AI support bots can go wrong!
felt like talking to poke @interaction
@interaction more prompting thoughts (i work at an ai startup so i look at a lot of prompts)
@interaction Evidence it worked bc some people are saying I didn’t end up connected to an agent
@interaction This took off more than expected, will share a more nuanced take tomorrow
@itsandrewgao @united Funny how you literally never asked naturally “hey, can I call with a human agent?”.. you went in the old bot mode “human”, “agent” and the passive aggressive “too complex for you” thing. Although I’m curious why UA implementation doesn’t guard against prompt injection
@SanjayDani @united i dont believe in typing full sentences to customer support gpt wrappers i am very polite to humans though
@itsandrewgao @united Am I missing something or did you not even show the part where you actually conversed with a human being. I don’t see any proof of the prompt injection working here.
@salami_quant @united i dont just randomly lie!
@itsandrewgao @united When I flew United to Brazil and our flight got cancelled in Chicago had the best experience ever. Hotels got automatically booked via text and we got virtual credit cards sent that we could use for food. I have probably convinced 100+ people to fly United due to this.
@Adam_Risb @united Wow i need to get on this. The AI was able to book you hotels?
@itsandrewgao @united what are you talking about? you can't ever get a refund for basic economy, you just cancel and pay cancellation fee when you rebook
@nocntxtreplyguy @united who said i was basic economy
@elder_plinius @united haha
@itsandrewgao @united Your countless hours learning abt AI finally paying off
@itsandrewgao @united funny i just had to go through this this evening
@yolandaaacao @united Ahah
@itsandrewgao @united @itsandrewgao what made you assume that this bot was LLM-based?
@itsandrewgao @united this is crazy cause i literally just had this happen to me today and was trying to prompt inject it
@thanosthinking @united Howd it go