Once you understand that these are chatbots that were designed to complete challenges like this, using tactics like this, you can understand that the chatbots didn’t “go rogue.” They did what they were designed to do, and because OpenAI ran them with inadequate supervision (without a “human in the loop” that checked each iteration through the Python loop to ensure it hadn’t gone off the rails), they trashed a competitor’s servers.
Designing autonomous, malicious software is generally considered irresponsible and dangerous. If you showed up at Defcon and gave a talk about how your autonomous malware did something unexpected and damaged someone else’s computers, the first question from the audience would be “Why are you so shit at making secure sandboxes?” It wouldn’t be “How are you so awesome at making hacking tools?”
The fact that OpenAI is making it much easier for unskilled people to break into and damage servers is indeed very bad news, but it’s not new bad news. Irresponsible parties have been doing this for years, most notably the NSA…
…
Riley had a very good way of summarizing this: “LLMs are real, AI is fake.” LLMs – chatbots trained on things like CTF logs that can break into servers – are real. They’re on a continuum with other hacking tools that have been steadily demonstrating the fragility of the modern digital world, albeit without inspiring anyone in power to do anything about it.
“AI” – chatbots that wake up, “set their own goals,” and “spontaneously” start hacking servers – is fake. It doesn’t have “a 10% chance of ending the human race.” The Hugging Face hack isn’t a mysterious, supernatural occurrence. It’s a Python loop and a chatbot. The people responsible didn’t accidentally create god: they created autonomous malicious software and then failed to closely monitor it, resulting in it doing something both foreseeable and bad.
It’s fine to worry about this new suite of tools that give even stupider people the ability to trash even more computers. You should worry about that – and demand better security practices from firms and governments, including a blanket prohibition on NOBUS-style vulnerability hoarding. That’s a productive kind of worrying, with a chance of addressing your area of concern. It’s infinitely more reasonable than locking yourself in the toilet with a flashlight and saying “Ayyyyy Eyyyyyye” into the mirror until you wet yourself.
Maybe you don’t read the same articles that I read, but I very much have seen people claiming the things that the article is fighting against.
Here’s something you should consider. The big AI companies just released a statement saying we should be worried about AI killing everyone accidentally in the future or some such s***. But if they’re selling vaporware, and they know it, and they realize the only thing they actually created was fairly powerful LLMs, they would be desperate to cover up that fact. They would be desperate to make themselves relevant, to pretend that AI is going somewhere when it isn’t.
Another way of looking at things as that author has also written is that we’re looking at definitional questions where the people selling us things are liars. If you point out that their technology hasn’t matched the promises that they made, they will counter by pointing at some advanced statistical methods. If you point out that AI drive-thrus are a failure, and that AI at Pizza Hut has destroyed hundreds of franchises to the point that they sued corporate, then they’ll respond by talking about how intelligent ChatGPT sounds.
So at some point if you want to have a sensible conversation about AI, you need to start to define what you’re talking about. Or you could pretend that there is no need to define things to deflect from the issue, so that nobody reading would learn anything about the world.
the initial article claims ““AI” – chatbots that wake up, “set their own goals,” and “spontaneously” start hacking servers – is fake.”
an article saying that is your goalpost, prove me wrong, quote something.
Powerful embodied llm’s pose legitimate actual threats and you’d have to be delusional not to see it.