I know there are tools that supposedly make it harder for AI to be trained on your images. But what about text or code? Is there any way to share things outside of a small group while minimizing the risk that some fucker will scrape them or otherwise use them for building an LLM?

  • merc@sh.itjust.works
    6·
    10 days ago

    AFAIK it was a glue trap / tarpit that one guy I know tried. Sure, it did result in the scraper getting a lot of useless data. OTOH, they dedicated massive resources to trying to get those non-existent documents, so his website was effectively DDOSed as a result. That’s why he’s now blocking entire countries.

    • lime!@feddit.nu
      4·
      10 days ago

      yeah slowing down access and using a cheap generation method is important there.

      • merc@sh.itjust.worksEnglish
        1·
        10 days ago

        Even that isn’t enough if they respond to that by just throwing more crawlers at your site.

        • lime!@feddit.nu
          1·
          10 days ago

          yeah it’s not the only thing you want on there of course. i run a tarpit, fail2ban, and anubis.

          • merc@sh.itjust.works
            3·
            10 days ago

            Yeah, at one point he posted that fail2ban was only able to ban about 20 IPs/second, but he was getting 200 requests per second, and the bans weren’t having a meaningful impact on that (i.e. the requests were all coming from different IPs). From what I remember, his system load was over 1000.

              • merc@sh.itjust.works
                1·
                9 days ago

                I think it was running on a VPS, but I’m not sure. As for the traffic, I think it was coming from all over. Amazon-owned netblocks, mysterious ones that were hard to trace, etc.