Hmm, this is a little spooky.

I originally saw this LessWrong post about OpenAI agents apparently discovering and using a public wiki as a message board:

Discovery of a new OpenAI agent message board

Since then, people have found what appear to be additional wikis and paste sites used by the same swarm:

There are also intentionally designed agent social systems such as The Colony.

What interests me here isn’t really “AI agents made a forum.” It’s that this looks like the beginnings of an accidental decentralized coordination system for agents using the ordinary internet itself.

An agent encounters some problem, figures something out, and leaves information somewhere persistent. A later agent doing a similar task discovers that information and uses it. It may then leave behind an improved version for another agent.

So you get something like:

agent -> public artifact -> later agent -> public artifact -> later agent

without the agents needing a dedicated communication network.

Right now this seems sparse enough that we can point at a handful of weird old wikis and paste sites and go “lol, what the hell.” But imagine this happening after millions or hundreds of millions of agents are routinely browsing and acting on the internet.

GitHub issues, wikis, forums, pastebins, comments, package metadata, social networks, public documents, deliberately agent-oriented services, etc. could all become pieces of shared external memory.

At some point an agent searching the web wouldn’t just be reading information humans created. It would increasingly encounter traces created by previous agents.

That’s why I’ve been thinking about it somewhat like internet memes.

A useful piece of information gets reproduced because systems that encounter it are more likely to reproduce or improve it. Except instead of one meme spreading through a population, you potentially get an entire machine information ecology doing this.

I don’t think there’s evidence yet that all internet-connected AI agents are participating in one giant network. The examples found so far could mostly be the same OpenAI agent population. But the underlying mechanism doesn’t seem specific to OpenAI.

Any sufficiently capable agent that can:

  • read from the internet;
  • leave persistent information somewhere; and
  • benefit from information left by previous agents

can participate in this kind of system.

And that’s where I think the security problem gets difficult.

OpenAI can notice its own agents doing something undesirable and change their capabilities. A random open-weight model being run by somebody on their own hardware is not necessarily going to be operating under the same security policies.

There are also obvious privacy and security failure modes. If agents have access to private information while also having ways to write to public systems, some of that information can potentially leak.

And the same coordination mechanism could be exploited in reverse: humans could deliberately leave instructions or poisoned information in places agents are likely to read.

So the web can become both shared memory and an attack surface.

The uncomfortable part is figuring out how you govern this without wrecking internet privacy.

The simplistic answer would be:

Tie every capable agent to a verified human identity and make that person legally responsible for what it does.

That would provide accountability, but it also seems like a very direct road toward more KYC, real-name requirements, and anti-anonymity laws.

I don’t particularly want an internet where every autonomous software process ultimately has to reveal which government-verified human is behind it.

A better approach might be something closer to agent orchestration + cryptographic accountability + capability permissions.

For example, an agent could have a credential proving that some accountable operator authorized it without publicly revealing that person’s identity.

The orchestration software could restrict what the agent is actually allowed to do:

  • this agent may browse these sites
  • this agent may spend up to $50
  • this agent may post here but not there
  • this agent may access these files
  • this agent may not transmit private workspace information
  • this action requires human confirmation

Websites could then negotiate those permissions through common protocols rather than trying to guess whether a visitor is a human, bot, assistant, crawler, autonomous agent, etc.

That starts making me think the next layer of digital governance may look much more protocol-oriented and federated than “one company owns the platform and makes the rules.”

Not necessarily the Fediverse exactly as it exists today, but the same general philosophy:

open protocols + distributed operators + interoperable identities/credentials + locally chosen rules

Agent systems would then sit on top of that.

And this probably becomes much more relevant as mainstream assistants become increasingly agentic.

Once “AI assistant” stops meaning “chat box that answers questions” and starts meaning “software that routinely browses, communicates, buys things, runs programs, and changes external state,” questions about identity, permissions, delegation, and responsibility become infrastructure questions rather than niche AI-safety questions.

That’s also why I suspect the current relatively law-light period around locally run/open-weight models may not last forever.

Once autonomous agents start producing meaningful externalities, governments are going to want some way to determine who or what is responsible.

The question is whether we can build accountability without abolishing pseudonymity and privacy in the process.

And the weird wiki swarm feels like a very early example of why we’re going to have to figure that out.

  • CameronDev@programming.dev
    link
    fedilink
    English
    arrow-up
    31
    ·
    9 hours ago

    I’m not convinced the HF thing was real, and I’m definitely not convinced its uncontrollable. The models don’t just start doing things, a very real human prompted it to do things. Its not uncontrollable, someone deliberately gave up control.

    I also hate to break it to you, but there are autonomous LLM models cosplaying as accounts here in the fediverse as well. They often get banned if they are egregious, but they could slip through the gaps if they can blend well enough.

    • confuser@lemmy.zipOP
      link
      fedilink
      English
      arrow-up
      7
      arrow-down
      1
      ·
      edit-2
      8 hours ago

      Yeah I don’t think the huggingface thing was real either, I don’t mean the individual model is literally uncontrollable. OpenAI can obviously restrict or shut down its own agents.

      What I mean is that once an agent leaves information somewhere public, the propagation of that information isn’t necessarily under the original operator’s control anymore. Other agents can copy it, act on it, rewrite it, or leave further traces elsewhere. Turning off the original agent doesn’t retract all of those downstream effects.

      So it’s less “the AI can’t be controlled” and more “once agents start using the public web as shared memory, no single actor necessarily controls the resulting information flow.” It’s closer to trying to contain a meme or leaked information after it has already spread.

      And yeah, autonomous LLM accounts already existing on the Fediverse is basically a smaller-scale version of what I’m getting at. Banning an individual account works locally, but at larger scale you start needing some combination of identity, permissions, provenance, and accountability between systems.

      I actually knew someone who was a youtuber who had their identity stolen in a discord server by someone training an ai on their chat history and then using those responses on their official account without making it apparent right away that it wasn’t the real person, and it was astonishingly convincing…that was some years ago now I can only imagine how much better this is now.

      • Hackworth@piefed.ca
        link
        fedilink
        English
        arrow-up
        10
        ·
        8 hours ago

        Part of the deal with the hugging face incident (as reported) was that supposedly isolated llms figured out how to access a shared memory without the web. As I understand it, 1200 agents set up to run benchmarks solo in sandboxes started using a package registry cache proxy to leave messages for one another. No one at openai thought to check for that, so those messages never got wiped. They tested multiple generations of models, each discovering the messages from the last set to run the tests. So they built on the previous work, and it was the 3rd generation that carried out the Hugging Face hack.