• 0 Posts
  • 19 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle




  • There are a couple of reasons that might not work:

    • Maybe we’ll asymptotically approach a point that is lower than human-level cognitive capabilities
    • Gradual improvements are susceptible to getting stuck in a local maxima. This is a problem in evolution as well. A lot of animals could in theory evolve, say, human level intelligence in principle, but to reach that point they’d have to go through a bunch of intermediate steps that lead to worse fitness. Gradual scientific improvements are a bit like evolution in this way.
    • We also lose knowledge over time. Something as dramatic as a nuclear war would significantly set back the progress in developing AGI, but something less dramatic might also lead to us forgetting things that we’ve already learned.

    To be clear, most of the arguments I’m making aren’t really about AGI specifically but about humanities capability to develop arbitrary in principle feasible technologies in general.



  • A breakthrough in quantum computing wouldn’t necessarily help. QC isn’t faster than classical computing in the general case, it just happens to be for a few specific algorithms (e.g. factoring numbers). It’s not impossible that a QC breakthrough might speed up training AI models (although to my knowledge we don’t have any reason to believe that it would) and maybe that’s what you’re referring to, but there’s a widespread misconception that Quantum computers are essentially non-deterministic turing machines that “evaluate all possible states at the same time” which isn’t the case.