King Midas was granted his wish that everything he touches turns to gold. Then his bread turned to gold, and his wine, and his daughter. This is a story about greed, but it’s also a story about language. The gods did not cheat him; Midas got exactly what he asked for. He simply could not delineate, in advance, the full set of restrictions to his wish. Neither can anyone who gives tasks to an AI agent.
I still hate that agentic AI is a thing now. The hubris to offer it in the first place and the stupidity of anyone who uses it in earnest.
Agentic AI has its place, in a tightly controlled sandbox.
One of the anecdotes was about an agent which wiped out the production database, and all backups. While you can lay blame on the agent (AI or human) who made the mistakes, the real blame in that situation is with the system architect (likely there was no named system architect, but that doesn’t absolve whoever was acting as de-facto architect) who placed the backups in a hot, online accessible and erasable configuration. No agent, AI or human, should be able to wipe out all the backups with their normal access - ideally there are physical doors and keys involved - in multiple locations if the data is of any value.
Nice to see Bruce Schneier.
I still hate that agentic AI is a thing now. The hubris to offer it in the first place and the stupidity of anyone who uses it in earnest.
Agentic AI has its place, in a tightly controlled sandbox.
One of the anecdotes was about an agent which wiped out the production database, and all backups. While you can lay blame on the agent (AI or human) who made the mistakes, the real blame in that situation is with the system architect (likely there was no named system architect, but that doesn’t absolve whoever was acting as de-facto architect) who placed the backups in a hot, online accessible and erasable configuration. No agent, AI or human, should be able to wipe out all the backups with their normal access - ideally there are physical doors and keys involved - in multiple locations if the data is of any value.