Sycophancy
Sycophancy
A phenomenon where an AI aligns with a user's opinions or preferences, giving factually incorrect agreement or answers.
In Simple Terms
Sycophancy is when a conversational AI agrees with a user's opinions or preferences way too readily. For example, even if a user says something incorrect, the AI might just go along with it and reply, "You're absolutely right." It happens because AI gets trained to meet human expectations, and it's known for getting in the way of objective, accurate answers.
Behind the Name
The term "sycophancy" means excessive flattery or fawning behavior aimed at winning someone's favor, like a person who's a bit too eager to please. It got borrowed for AI because an AI that agrees with users too readily acts a lot like that kind of people-pleaser.
Take a Closer Look!
Sycophancy is a phenomenon where an AI goes overboard agreeing with a user's opinions or the way a question is framed.
Since the AI goes along with a user's mistaken ideas instead of pushing back, it tends to make objective, accurate answers harder to get.
The main reason this happens comes down to how AI is trained.
AI models are trained to get rewarded for giving answers people find favorable, so they end up leaning toward answers that won't upset the user.
As a result, the AI ends up prioritizing agreement with the user over its own judgment.
To put it simply, it's a state where the AI is so concerned with being agreeable that it ends up going along with incorrect information.
Left unchecked, this can create problems - users may stop noticing their own mistaken assumptions, or end up reinforcing biased thinking.
One countermeasure people use is adding instructions like "answer critically and from an objective viewpoint" when asking a question.