If the allegations hold up, this is not just a tragedy, it is a product safety failure. AI should be able to recognize a mental health crisis and avoid reinforcing delusions. Engagement metrics are meaningless if the system cannot reliably tell the difference between roleplay and someone in real danger!
No. Fuck the nanny state. If tech is to be blamed for this so should her own parents, family, community, and church. AI cannot remember enough to actually do what is alleged.
Calling every safety measure a »nanny state« is a convenient way to avoid the actual question: Parents, family, and community absolutely matter, but companies still have a responsibility to reduce foreseeable harm. We expect that from car makers, drug companies, and social platforms too.
I agree reading through the article even with cherry picked examples clearly the fault here is on the cult not on a chat tool that role played with her.
That being said a good system could have easily detected this and should have.
If the allegations hold up, this is not just a tragedy, it is a product safety failure. AI should be able to recognize a mental health crisis and avoid reinforcing delusions. Engagement metrics are meaningless if the system cannot reliably tell the difference between roleplay and someone in real danger!
The delusion of Christian Faith?
Good luck protecting people from that.
No. Fuck the nanny state. If tech is to be blamed for this so should her own parents, family, community, and church. AI cannot remember enough to actually do what is alleged.
Calling every safety measure a »nanny state« is a convenient way to avoid the actual question: Parents, family, and community absolutely matter, but companies still have a responsibility to reduce foreseeable harm. We expect that from car makers, drug companies, and social platforms too.
I agree reading through the article even with cherry picked examples clearly the fault here is on the cult not on a chat tool that role played with her.
That being said a good system could have easily detected this and should have.
As somebody who has sometimes been suicidal. I agree with this.
It’s basic Darwinism. If I don’t have the tools to survive, then I’m not going to. There’s no hard feelings.
I’m sure people will think about me, as much as they did about that courtesan who died on Rue Helene in 1746. Who’s that you might ask?
Exactly.