Sarah Z did an amazingly prescient and compassionate take on this over 2 years ago with The Rise and Fall of Replika.
I urge everyone to try and approach this with some level of compassion and understanding; even though it seems ridiculous to most of us they are actively preying on the emotionally vulnerable to profit. In the same way that “you are not immune to propaganda”, the ability of AI tools to parse language and wield emotional payloads in a calculated manner (something a lot of us already refer to as “the algorithm” in various ways) shouldn’t be underestimated. Even when you’re not directly using those tools yourself, they could be used against you. Dead Internet Theory and AI posting chatbots are already a part of this.
People using AI companion apps are juts the leading edge, the volunteers. I urge you to take the danger seriously and please have some compassion for your fellow human beings, especially the vulnerable.
We also typically think of these individuals as mature adults with some understanding of the world and social skills — but a lot of the people getting pulled in are kids and teenagers, which are particularly vulnerable for exactly the reason you elucidated.
Sarah Z did an amazingly prescient and compassionate take on this over 2 years ago with The Rise and Fall of Replika.
I urge everyone to try and approach this with some level of compassion and understanding; even though it seems ridiculous to most of us they are actively preying on the emotionally vulnerable to profit. In the same way that “you are not immune to propaganda”, the ability of AI tools to parse language and wield emotional payloads in a calculated manner (something a lot of us already refer to as “the algorithm” in various ways) shouldn’t be underestimated. Even when you’re not directly using those tools yourself, they could be used against you. Dead Internet Theory and AI posting chatbots are already a part of this.
People using AI companion apps are juts the leading edge, the volunteers. I urge you to take the danger seriously and please have some compassion for your fellow human beings, especially the vulnerable.
We also typically think of these individuals as mature adults with some understanding of the world and social skills — but a lot of the people getting pulled in are kids and teenagers, which are particularly vulnerable for exactly the reason you elucidated.
Yeah. I mean, I don’t want to police the internet, I get why this situation is darkly humorous to a lot of people.
But it’s worth considering what you’re laughing at and why, because the joke could soon be on you or someone you love.