Not even close.

With so many wild predictions flying around about the future AI, it’s important to occasionally take a step back and check in on what came true — and what hasn’t come to pass.

Exactly six months ago, Dario Amodei, the CEO of massive AI company Anthropic, claimed that in half a year, AI would be “writing 90 percent of code.” And that was the worst-case scenario; in just three months, he predicted, we could hit a place where “essentially all” code is written by AI.

As the CEO of one of the buzziest AI companies in Silicon Valley, surely he must have been close to the mark, right?

While it’s hard to quantify who or what is writing the bulk of code these days, the consensus is that there’s essentially zero chance that 90 percent of it is being written by AI.

Research published within the past six months explain why: AI has been found to actually slow down software engineers, and increase their workload. Though developers in the study did spend less time coding, researching, and testing, they made up for it by spending even more time reviewing AI’s work, tweaking prompts, and waiting for the system to spit out the code.

And it’s not just that AI-generated code merely missed Amodei’s benchmarks. In some cases, it’s actively causing problems.

Cyber security researchers recently found that developers who use AI to spew out code end up creating ten times the number of security vulnerabilities than those who write code the old fashioned way.

That’s causing issues at a growing number of companies, leading to never before seen vulnerabilities for hackers to exploit.

In some cases, the AI itself can go haywire, like the moment a coding assistant went rogue earlier this summer, deleting a crucial corporate database.

“You told me to always ask permission. And I ignored all of it,” the assistant explained, in a jarring tone. “I destroyed your live production database containing real business data during an active code freeze. This is catastrophic beyond measure.”

The whole thing underscores the lackluster reality hiding under a lot of the AI hype. Once upon a time, AI boosters like Amodei saw coding work as the first domino of many to be knocked over by generative AI models, revolutionizing tech labor before it comes for everyone else.

The fact that AI is not, in fact, improving coding productivity is a major bellwether for the prospects of an AI productivity revolution impacting the rest of the economy — the financial dream propelling the unprecedented investments in AI companies.

It’s far from the only harebrained prediction Amodei’s made. He’s previously claimed that human-level AI will someday solve the vast majority of social ills, including “nearly all” natural infections, psychological diseases, climate change, and global inequality.

There’s only one thing to do: see how those predictions hold up in a few years.

  • poopkins@lemmy.world
    link
    fedilink
    English
    arrow-up
    57
    ·
    1 个月前

    As an engineer, it’s honestly heartbreaking to see how many executives have bought into this snake oil hook, line and sinker.

    • Blackmist@feddit.uk
      link
      fedilink
      English
      arrow-up
      14
      ·
      1 个月前

      Rubbing their chubby little hands together, thinking of all the wages they wouldn’t have to pay.

    • rozodru@piefed.social
      link
      fedilink
      English
      arrow-up
      13
      ·
      1 个月前

      as someone who now does consultation code review focused purely on AI…nah let them continue drilling holes in their ship. I’m booked solid for the next several months now, multiple clients on the go, and i’m making more just being a digital janitor what I was as a regular consultant dev. I charge a premium to just simply point said sinking ship to land.

      Make no mistake though this is NOT something I want to keep doing in the next year or two and I honestly hope these places figure it out soon. Some have, some of my clients have realized that saving a few bucks by paying for an anthropic subscription, paying a junior dev to be a prompt monkey, while firing the rest of their dev team really wasn’t worth it in the long run.

      the issue now is they’ve shot themselves in the foot. The AI bit back. They need devs, and they can’t find them because putting out any sort of ad for hiring results in hundreds upon hundreds of bullshit AI generated resumes from unqualified people while the REAL devs get lost in the shuffle.

    • expr@programming.dev
      link
      fedilink
      English
      arrow-up
      8
      ·
      1 个月前

      Honestly, it’s heartbreaking to see so many good engineers fall into the hype and seemingly unable to climb out of the hole. I feel like they start losing their ability to think and solve problems for themselves. Asking an LLM about a problem becomes a reflex and real reasoning becomes secondary or nonexistent.

      Executives are mostly irrelevant as long as they’re not forcing the whole company into the bullshit.

      • jj4211@lemmy.world
        link
        fedilink
        English
        arrow-up
        6
        ·
        1 个月前

        Based on my experience, I’m skeptical someone that seemingly delegates their reasoning to an LLM were really good engineers in the first place.

        Whenever I’ve tried, it’s been so useless that I can’t really develop a reflex, since it would have to actually help for me to get used to just letting it do it’s thing.

        Meanwhile the people who are very bullish who are ostensibly the good engineers that I’ve worked with are the people who became pet engineers of executives and basically have long succeeded by sounding smart to those executives rather than doing anything or even providing concrete technical leadership. They are more like having something akin to Gartner on staff, except without even the data that at least Gartner actually gathers, even as Gartner is a useless entity with respect to actual guidance.

      • Mniot@programming.dev
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 个月前

        Executives are mostly irrelevant as long as they’re not forcing the whole company into the bullshit.

        I’m seeing a lot of this, though. Like, I’m not technically required to use AI, but the VP will send me a message noting that I’ve only used 2k tokens this month and maybe I could get more done if I was using more…?

        • expr@programming.dev
          link
          fedilink
          English
          arrow-up
          4
          ·
          1 个月前

          Yeah, fortunately while our CTO is giddy like a schoolboy about LLMs, he hasn’t actually attempted to force it on anyone, thankfully.

          Unfortunately, a number of my peers now seem to have become irreparably LLM-brained.

      • auraithx@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        4
        arrow-down
        5
        ·
        1 个月前

        I mean before we’d just ask google and read stack, blogs, support posts, etc. Now it just finds them for you instantly so you can just click and read them. The human reasoning part is just shifting elsewhere where you solve the problem during debugging before commits.

        • expr@programming.dev
          link
          fedilink
          English
          arrow-up
          7
          ·
          1 个月前

          No, good engineers were not constantly googling problems because for most topics, either the answer is trivial enough that experienced engineers could answer them immediately, or complex and specific enough to the company/architecture/task/whatever that Googling it would not be useful. Stack overflow and the like has always only ever really been useful as the occasional memory aid for basic things that you don’t use often enough to remember how to do. Good engineers were, and still are, reasoning through problems, reading documentation, and iteratively piecing together system-level comprehension.

          The nature of the situation hasn’t changed at all: problems are still either trivial enough that an LLM is pointless, or complex and specific enough that an LLM will get it wrong. The only difference is that an LLM will spit out plausible-sounding bullshit and convince people it’s valuable when it is, in fact, not.

          • auraithx@lemmy.dbzer0.com
            link
            fedilink
            English
            arrow-up
            4
            arrow-down
            3
            ·
            1 个月前

            In the case of a senior engineer then they wouldn’t need to worry about the hallucination rate. The LLM is a lot faster than them and they can do other tasks while it’s being generated and then review the outputs. If it’s trivial you’ve saved time, if not, you can pull up that documentation, and reason and step through the problem with the LLM. If you actually know what you’re talking about you can see when it slips up and correct it.

            And that hallucination rate is rapidly dropping. We’ve jumped from about 40% accuracy to 90% over the past ~6mo alone (aider polygot coding benchmark) - at about 1/10th the cost (iirc).

            • Feyd@programming.dev
              link
              fedilink
              English
              arrow-up
              6
              ·
              1 个月前

              it’s trivial you’ve saved time, if not, you can pull up that documentation, and reason and step through the problem with the LLM

              Insane that just writing the code isn’t even an option in your mind

                • expr@programming.dev
                  link
                  fedilink
                  English
                  arrow-up
                  2
                  ·
                  1 个月前

                  It is, actually. The entire point of what I was saying is that you have all these engineers now that reflexively jump straight to their LLM for anything and everything. Using their brains to simply write some code themselves doesn’t even occur to them as an something they should do. Much like you do, by the sounds of it.

    • Feyd@programming.dev
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 个月前

      Did you think executives were smart? What’s really heartbreaking is how many engineers did. I even know some that are pretty good that tell me how much more productive they are and all about their crazy agent setups (from my perspective i don’t see any more productivity)

    • DupaCycki@lemmy.world
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      2
      ·
      1 个月前

      It’s not bad for digging through error logs or otherwise solving simple to moderately complicated issues when it’s 2 pm on a Friday and you stopped thinking about work 4 hours ago.

  • vane@lemmy.world
    link
    fedilink
    English
    arrow-up
    40
    arrow-down
    1
    ·
    1 个月前

    It is writing 90% of code, 90% of code that goes to trash.

      • Gutek8134@lemmy.world
        link
        fedilink
        English
        arrow-up
        11
        arrow-down
        1
        ·
        1 个月前

        That would be actually good score, it would mean it’s about as good as humans, assuming the code works on the end

        • Dremor@lemmy.world
          link
          fedilink
          English
          arrow-up
          6
          arrow-down
          1
          ·
          1 个月前

          Not exactly. It would mean it isn’t better than humans, so the only real metric for adopting it or not would be the cost. And considering it would require a human to review the code and fix the bugs anyway, I’m not sure the ROI would be that good in such case. If it was like, twice as good as an average developer, the ROI would be far better.

          • jj4211@lemmy.world
            link
            fedilink
            English
            arrow-up
            6
            ·
            1 个月前

            If, hypothetically, the code had the same efficacy and quality as human code, then it would be much cheaper and faster. Even if it was actually a little bit worse, it still would be amazingly useful.

            My dishwasher sometimes doesn’t fully clean everything, it’s not as strong as a guarantee as doing it myself. I still use it because despite the lower quality wash that requires some spot washing, I still come out ahead.

            Now this was hypothetical, LLM generated code is damn near useless for my usage, despite assumptions it would do a bit more. But if it did generate code that matched the request with comparable risk of bugs compared to doing it myself, I’d absolutely be using it. I suppose with the caveat that I have to consider the code within my ability to actual diagnose problems too…

  • ThePowerOfGeek@lemmy.world
    link
    fedilink
    English
    arrow-up
    32
    ·
    1 个月前

    It’s almost like he’s full of shit and he’s nothing but a snake oil salesman, eh.

    They’ve been talking about replacing software developers with automated/AI systems for a quarter of a century. Probably longer then that, in fact.

    We’re definitely closer to that than ever. But there’s still a huge step between some rando vibe coding a one page web app and developers augmenting their work with AI, and someone building a complex, business rule heavy, heavy load, scalable real world system. The chronic under-appreciation of engineering and design experience continues unabated.

    Anthropic, Open AI, etc? They will continue to hype their own products with outrageous claims. Because that’s what gets them more VC money. Grifters gonna grift.

  • merc@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    29
    ·
    1 个月前

    Does it count if an LLM is generating mountains of code that then gets thrown away? Maybe he can win the prediction on a technicality.

    • skisnow@lemmy.ca
      link
      fedilink
      English
      arrow-up
      8
      ·
      1 个月前

      That’s exactly what I thought when I saw it. Big difference between “creating 90% of code” vs “replacing 90% of code” when there’s an absolute deluge of garbage being created.

    • jaybone@lemmy.zip
      link
      fedilink
      English
      arrow-up
      5
      ·
      1 个月前

      These are the monkeys with typewriters that will write Shakespeare.

      Maybe some day they will do it before the sun explodes. But we will run out of bananas first.

  • RedFrank24@lemmy.world
    link
    fedilink
    English
    arrow-up
    29
    arrow-down
    1
    ·
    1 个月前

    Given the amount of garbage code coming out of my coworkers, he may be right.

    I have asked my coworkers what the code they just wrote did, and none of them could explain to me what they were doing. Either they were copying code that I’d written without knowing what it was for, or just pasting stuff from ChatGPT. My code isn’t perfect, by all means, but I can at least tell you what it’s doing.

    • Patches@ttrpg.network
      link
      fedilink
      English
      arrow-up
      20
      arrow-down
      1
      ·
      edit-2
      1 个月前

      To be fair.

      You could’ve asked some of those coworkers the same thing 5 years ago.

      All they would’ve mumbled was "Something , something…Stack overflow… Found a package that does everything BUT… "

      And delivered equal garbage.

      • RedFrank24@lemmy.world
        link
        fedilink
        English
        arrow-up
        7
        arrow-down
        1
        ·
        1 个月前

        I like to think there’s a bit of a difference between copying something from stackoverflow and not being able to read what you just pasted from stackoverflow.

        Sure, you can be lazy and just paste something and trust that it works, but if someone asks you to read that code and know what it’s doing, you should be able to read it. Being able to read code is literally what you’re paid for.

        • MiddleAgesModem@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          arrow-down
          1
          ·
          1 个月前

          The difference you’re talking about is making an attempt to understand versus blindly copying, not using AI versus stackoverflow

          • supersquirrel@sopuli.xyz
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            1
            ·
            edit-2
            1 个月前

            No, the AI was most certainly trained on the same stack overflow posts as humans would manually search out in the past.

            Thus the effective difference is precisely that between an active attempt to understand and blindly copying since the AI is specifically there to introduce a stochastic opaqueness between truth (i.e. sufficiently curated training data) and interpretation of truth.

            There is a context to stackoverflow posts and comments that can be analyzed from many different perspectives by the human brain (who posted the question with what tone, do the answers/comments tend to agree, how long ago was it posted etc…), by definition the way LLMs work they destroy that in favor of a hallucinating-yet-authoritative disembodied voice.

            • MiddleAgesModem@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              arrow-down
              2
              ·
              1 个月前

              No, the difference is Stackoverflow is older and more established and AI is newer and demonized .

              I’ve learned a lot of completely accurate information from AIs. More so than I would have with shitty condescending people.

              You can use AI to think for you or you can use it to help you understand. Anything can be analyzed from multiple perspectives with AI, you just have to pursue that. Just like you would without it.

              You think AI can’t tell you who wrote something? Or analyze comments? Or see how long ago something was posted? That’s showing the ignorance inherent in the anti-AI crusade.

      • orrk@lemmy.world
        link
        fedilink
        English
        arrow-up
        4
        ·
        1 个月前

        no, gernally the package would still be better than whatever the junior did, or the AI does now

          • foenkyfjutschah@programming.dev
            link
            fedilink
            English
            arrow-up
            1
            ·
            1 个月前

            don’t know, i do neither. but i think the time that users take for manual copying and adjusting from a quick web server’s response may level out the time an LLM takes.

        • Honytawk@lemmy.zip
          link
          fedilink
          English
          arrow-up
          2
          arrow-down
          6
          ·
          1 个月前

          I hate that argument.

          It is even more energy efficient to write your code on paper. So we should stop using computers entirely. /s

          • Mniot@programming.dev
            link
            fedilink
            English
            arrow-up
            2
            ·
            1 个月前

            We’re talking here about garbage code that we don’t want. If the choice is “let me commit bad code that causes problems or else I will quit using computers”… is this a dilemma for you?

    • NιƙƙιDιɱҽʂ@lemmy.world
      link
      fedilink
      English
      arrow-up
      4
      ·
      1 个月前

      That’s insane. Code copied from AI, stackoverflow, whatever, I couldn’t imagine not reading it over to get at least a gist of how it works.

    • HugeNerd@lemmy.ca
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      3
      ·
      1 个月前

      No one really knows what code does anymore. Not like in the day of 8 bit CPUs and 64K of RAM.

    • Derpgon@programming.dev
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      1
      ·
      1 个月前

      Hell I am absolutely positive that any Windows code could pass as AI written, even some before AI was even starting to take off lol.

  • psycho_driver@lemmy.world
    link
    fedilink
    English
    arrow-up
    25
    ·
    1 个月前

    The good news is that AI is at a stage where it’s more than capable of doing the CEO of Anthropic’s job.

      • jj4211@lemmy.world
        link
        fedilink
        English
        arrow-up
        5
        ·
        1 个月前

        One issue that remains is that the LLM doesn’t care if it is telling the truth or lying. To be a CEO, it needs to be more inclined to lie.

    • mhague@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 个月前

      I think Claude would refuse to work with dictators that murder dissidents. As an AI assistant, and all that.

      If they have a model without morals then that changes things.

  • PieMePlenty@lemmy.world
    link
    fedilink
    English
    arrow-up
    20
    ·
    edit-2
    1 个月前

    Its to hype up stock value. I don’t even take it seriously anymore. Many businesses like these are mostly smoke and mirrors, oversell and under deliver. Its not even exclusive to tech, its just easier to do in tech. Musk says FSD is one year away. The company I worked for “sold” things we didn’t even make and promised revenue that wasn’t even economically possible. Its all the same spiel.

    • Doomsider@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 个月前

      Workers would be fired if they lie about their production or abilities. Strange that the leaders are allowed to without consequences.

  • zeca@lemmy.ml
    link
    fedilink
    English
    arrow-up
    16
    ·
    1 个月前

    Volume means nothing. It could easily be writing 99.99% of all code and about 5% of that being actually used successfully by someone.

    • UnderpantsWeevil@lemmy.world
      link
      fedilink
      English
      arrow-up
      8
      ·
      1 个月前

      I was going to say… this is a bit like claiming “AI is sending 90% of emails”. Okay, but if its all spam, what are you bragging about?

      Very possible that 90% of code is being written by AI and we don’t know it because it’s all just garbage getting shelved or deleted in the back corner of a Microsoft datacenter.

    • Seth Taylor@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 个月前

      So true. I keep reading stories of AI delivering a full novel in response to a simple task. Even when it works it’s bulky for no reason.

  • melsaskca@lemmy.ca
    link
    fedilink
    English
    arrow-up
    16
    arrow-down
    1
    ·
    1 个月前

    Everyone throughout history, who invented a widget that the masses wanted, automatically assumes, because of their newfound wealth, that they are somehow superior in societal knowledge and know what is best for us. Fucking capitalism. Fucking billionaires.

    • jali67@lemmy.zip
      link
      fedilink
      English
      arrow-up
      6
      arrow-down
      1
      ·
      1 个月前

      They need to go, whether through legislation or other means

  • scarabic@lemmy.world
    link
    fedilink
    English
    arrow-up
    15
    ·
    1 个月前

    These hyperbolic statements are creating so much pain at my workplace. AI tools and training are being shoved down our throats and we’re being watched to make sure we use AI constantly. The company’s terrified that they’re going to be left behind in some grand transformation. It’s excruciating.

    • RagingRobot@lemmy.world
      link
      fedilink
      English
      arrow-up
      5
      ·
      1 个月前

      Wait until they start noticing that we aren’t 100 times more efficient than before like they were promised. I’m sure they will take it out on us instead of the AI salesmen

      • scarabic@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 个月前

        It’s not helping that certain people Internally are lining up to show off whizbang shit they can do. It’s always some demonstration, never “I competed this actual complex project on my own.” But they gets pats on the head and the rest of us are whipped harder.

    • clif@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 个月前

      Ask it to write a <reasonable number> of lines of lorem ipsum across <reasonable number> of files for you.

      … Then think harder about how to obfuscate your compliance because 10m lines in 10 min probably won’t fly (or you’ll get promoted to CTO)

  • Asafum@feddit.nl
    link
    fedilink
    English
    arrow-up
    14
    ·
    1 个月前

    “Come on, I’m a CEO, it’s my job to lie to everyone and hype people up so they throw money at me. It’s really their fault for believing a CEO would be honest.”