A Florida man is facing 20 counts of obscenity for allegedly creating and distributing AI-generated child pornography, highlighting the danger and ubiquity of generative AI being used for nefarious reasons.

Phillip Michael McCorkle was arrested last week while he was working at a movie theater in Vero Beach, Florida, according to TV station CBS 12 News. A crew from the TV station captured the arrest, which made for dramatic video footage due to law enforcement leading away the uniform-wearing McCorkle from the theater in handcuffs.

  • macniel@feddit.org
    link
    fedilink
    arrow-up
    86
    arrow-down
    24
    ·
    4 months ago

    I don’t see how children were abused in this case? It’s just AI imagery.

    It’s the same as saying that people get killed when you play first person shooter games.

    Or that you commit crimes when you play GTA.

        • timestatic@feddit.org
          link
          fedilink
          English
          arrow-up
          16
          arrow-down
          1
          ·
          4 months ago

          But this is the US… and its kind of a double standard if you’re not arrested for drawing but for generating it.

            • ContrarianTrail@lemm.ee
              link
              fedilink
              English
              arrow-up
              2
              arrow-down
              1
              ·
              4 months ago

              The core reason CSAM is illegal is not because we don’t want people to watch it but because we don’t want them to create it which is synonymous with child abuse. Jailing someone for drawing a picture like that is absurd. While it might be of bad taste, there is no victim there. No one was harmed. Using generative AI is the same thing. No matter how much simulated CSAM you create with it, not a single child is harmed in doing so. Jailing people for that is the very definition of a moral panic.

              Now, if actual CSAM was used in the training of that AI, then it’s a more complex question. However it is a fact that such content doesn’t need to be in the training data in order for it to create simulated CSAM and as long as that is the case it is immoral to punish people for creating something that only looks like it but isn’t.

    • CeruleanRuin@lemmings.world
      link
      fedilink
      English
      arrow-up
      5
      arrow-down
      14
      ·
      edit-2
      4 months ago

      Not a great comparison, because unlike withh violent games or movies, you can’t say that there is no danger to anyone in allowing these images to be created or distributed. If they are indistinguishable from the real thing, it then becomes impossible to identify actual human victims.

      There’s also a strong argument that the availability of imagery like this only encourages behavioral escalation in people who suffer from the affliction of being a sick fucking pervert pedophile. It’s not methadone for them, as some would argue. It’s just fueling their addiction, not replacing it.

    • TallonMetroid@lemmy.world
      link
      fedilink
      English
      arrow-up
      11
      arrow-down
      46
      ·
      4 months ago

      Well, the image generator had to be trained on something first in order to spit out child porn. While it may be that the training set was solely drawn/rendered images, we don’t know that, and even if the output were in that style, it might very well be photorealistic images generated from real child porn and run through a filter.

        • Saledovil@sh.itjust.works
          link
          fedilink
          arrow-up
          5
          ·
          4 months ago

          Wild corn dogs are an outright plague where I live. When I was younger, me and my buddies would lay snares to catch to corn dogs. When we caught one, we’d roast it over a fire to make popcorn. Corn dog cutlets served with popcorn from the same corn dog is popular meal, especially among the less fortunate. Even though some of the affluent consider it the equivalent to eating rat meat. When me pa got me first rifle when I turned 14, I spent a few days just shooting corn dogs.

        • emmy67@lemmy.world
          link
          fedilink
          arrow-up
          1
          arrow-down
          2
          ·
          edit-2
          4 months ago

          It didn’t generate what we expect and know a corn dog is.

          Hence it missed because it doesn’t know what a “corn dog” is

          You have proven the point that it couldn’t generate csam without some being present in the training data

          • ContrarianTrail@lemm.ee
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            2
            ·
            edit-2
            4 months ago

            I hope you didn’t seriously think the prompt for that image was “corn dog” because if your understanding of generative AI is on that level you probably should refrain from commenting on it.

            Prompt: Photograph of a hybrid creature that is a cross between corn and a dog

            • emmy67@lemmy.world
              link
              fedilink
              arrow-up
              1
              arrow-down
              2
              ·
              4 months ago

              Then if your question is “how many Photograph of a hybrid creature that is a cross between corn and a dog were in the training data?”

              I’d honestly say, i don’t know.

              And if you’re honest, you’ll say the same.

              • ContrarianTrail@lemm.ee
                link
                fedilink
                English
                arrow-up
                3
                arrow-down
                1
                ·
                4 months ago

                But you do know because corn dogs as depicted in the picture do not exists so there couldn’t have been photos of them in the training data, yet it was still able to create one when asked.

                This is because it doesn’t need to have been seen one before. It knows what corn looks like and it knows what a dog looks like so when you ask it to combine the two it will gladly do so.

                • emmy67@lemmy.world
                  link
                  fedilink
                  arrow-up
                  1
                  arrow-down
                  1
                  ·
                  edit-2
                  4 months ago

                  But you do know because corn dogs as depicted in the picture do not exists so there couldn’t have been photos of them in the training data, yet it was still able to create one when asked.

                  Yeah, except photoshop and artists exist. And a quick google image search will find them. 🙄

                  • ContrarianTrail@lemm.ee
                    link
                    fedilink
                    English
                    arrow-up
                    2
                    arrow-down
                    1
                    ·
                    4 months ago

                    And this proves that AI can’t generate simulated CSAM without first having seen actual CSAM how, exactly?

                    To me, the takeaway here is that you can take a shitty 2 minute photoshop doodle and by feeding it thru AI it’ll improve the quality of it by orders of magnitude.

      • MagicShel@programming.dev
        link
        fedilink
        arrow-up
        41
        arrow-down
        3
        ·
        4 months ago

        An AI that is trained on children and nude adults can infer what a nude child looks like without ever being trained specifically with those images.

              • Cryophilia@lemmy.world
                link
                fedilink
                arrow-up
                3
                arrow-down
                1
                ·
                4 months ago

                No, I’m admitting they’re stupid for even bringing it up.

                Unless their argument is that all AI should be illegal, in which case they’re stupid in a different way.

                • LustyArgonian@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  arrow-down
                  2
                  ·
                  edit-2
                  4 months ago

                  Do you think regular child porn should be illegal? If so, why?

                  Generally it’s because kids were harmed in the making of those images. Since we know that AI is using images of children being harmed to make these images, as the other posters has repeatedly sourced (but also if you’ve looked up deepfakes, most deepfakes are of an existing porn and the face just changed over top. They do this with CP as well and must use CP videos to seed it, because the adult model would be too large)… why does AI get a pass for using children’s bodies in this way? Why isn’t it immoral when AI is used as a middle man to abuse kids?

                  • Cryophilia@lemmy.world
                    link
                    fedilink
                    arrow-up
                    3
                    arrow-down
                    1
                    ·
                    4 months ago

                    Since we know that AI is using images of children being harmed to make these images

                    As I keep saying, if this is your reasoning then all AI should be illegal. It only has CP in its training set incidentally, because the entire dataset of images on the internet contains some CP. It’s not being specifically trained on CP images.

          • LustyArgonian@lemmy.world
            link
            fedilink
            English
            arrow-up
            1
            arrow-down
            3
            ·
            edit-2
            4 months ago

            Yes exactly. That people are then excusing this with “well it was trained on all.public images,” are just admitting you’re right and that there is a level of harm here since real materials are used. Even if they weren’t being used or if it was just a cartoon, the morality is still shaky because of the role porn plays in advertising. We already have laws about advertising because it’s so effective, including around cigarettes and prescriptions. Most porn, ESPECIALLY FREE PORN, is an ad to get you to buy other services. CP is not excluded from this rule - no one gets free lunch, so to speak. These materials are made and hosted for a reason.

            The role that CP plays in most countries is difficult. It is used for blackmail. It is also used to generate money for countries (intelligence groups around the world host illegal porn ostensibly “to catch a predator,” but then why is it morally okay for them to distribute these images but no one else?). And it’s used as advertising for actual human trafficking organizations. And similar organizations exist for snuff and gore btw. And ofc animals. And any combination of those 3. Or did you all forget about those monkey torture videos, or the orangutan who was being sex trafficked? Or Daisy’s Destruction and Peter Scully?

            So it’s important to not allow these advertisers to combine their most famous monkey torture video with enough AI that they can say it’s AI generated, but it’s really just an ad for their monkey torture productions. And even if NONE of the footage was from illegal or similar events and was 100% thought of by AI - it can still be used as an ad for these groups if they host it. Cartoons can be ads ofc.

      • lunarul@lemmy.world
        link
        fedilink
        arrow-up
        15
        arrow-down
        1
        ·
        edit-2
        4 months ago

        we don’t know that

        might

        Unless you’re operating under “guilty until proven innocent”, those are not reasons to accuse someone.

    • KillerTofu@lemmy.world
      link
      fedilink
      arrow-up
      11
      arrow-down
      50
      ·
      4 months ago

      How was the model trained? Probably using existing CSAM images. Those children are victims. Making derivative images of “imaginary” children doesn’t negate its exploitation of children all the way down.

      So no, you are making false equivalence with your video game metaphors.

      • fernlike3923@sh.itjust.works
        link
        fedilink
        arrow-up
        47
        arrow-down
        1
        ·
        4 months ago

        A generative AI model doesn’t require the exact thing it creates in its datasets. It most likely just combined regular nudity with a picture of a child.

        • finley@lemm.ee
          link
          fedilink
          English
          arrow-up
          6
          arrow-down
          21
          ·
          4 months ago

          In that case, the images of children were still used without their permission to create the child porn in question

          • MagicShel@programming.dev
            link
            fedilink
            arrow-up
            27
            arrow-down
            4
            ·
            4 months ago

            That’s not really a nuanced take on what is going on. A bunch of images of children are studied so that the AI can learn how to draw children in general. The more children in the dataset, the less any one of them influences or resembles the output.

            Ironically, you might have to train an AI specifically on CSAM in order for it to identify the kinds of images it should not produce.

          • fernlike3923@sh.itjust.works
            link
            fedilink
            arrow-up
            6
            arrow-down
            2
            ·
            4 months ago

            That’s a whole other thing than the AI model being trained on CSAM. I’m currently neutral on this topic so I’d recommend you replying to the main thread.

          • CeruleanRuin@lemmings.world
            link
            fedilink
            English
            arrow-up
            7
            arrow-down
            5
            ·
            4 months ago

            Good luck convincing the AI advocates of this. They have already decided that all imagery everywhere is theirs to use however they like.

      • macniel@feddit.org
        link
        fedilink
        arrow-up
        21
        arrow-down
        1
        ·
        4 months ago

        Can you or anyone verify that the model was trained on CSAM?

        Besides a LLM doesn’t need to have explicit content to derive from to create a naked child.

        • KillerTofu@lemmy.world
          link
          fedilink
          arrow-up
          4
          arrow-down
          28
          ·
          4 months ago

          You’re defending the generation of CSAM pretty hard here in some vaguely “but no child we know of” being involved as a defense.

          • macniel@feddit.org
            link
            fedilink
            arrow-up
            12
            arrow-down
            2
            ·
            4 months ago

            I just hope that the Models aren’t trained on CSAM. Making generating stuff they can fap on ““ethical reasonable”” as no children would be involved. And I hope that those who have those tendancies can be helped one way or another that doesn’t involve chemical castration or incarceration.

      • Diplomjodler@lemmy.world
        link
        fedilink
        arrow-up
        8
        ·
        4 months ago

        While i wouldn’t put it past Meta&Co. to explicitly seek out CSAM to train their models on, I don’t think that is how this stuff works.

      • grue@lemmy.world
        link
        fedilink
        English
        arrow-up
        3
        arrow-down
        8
        ·
        4 months ago

        But the AI companies insist the outputs of these models aren’t derivative works in any other circumstances!