lemmy.4d2.org
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
ItWasntMe@discuss.online to Fuck AI@lemmy.worldEnglish · 1 month ago

Aggressive AI scrapers are making it kinda suck to run wikis

weirdgloop.org

external-link
message-square
41
link
fedilink
  • cross-posted to:
  • technology@beehaw.org
  • technology@lemmy.world
167
external-link

Aggressive AI scrapers are making it kinda suck to run wikis

weirdgloop.org

ItWasntMe@discuss.online to Fuck AI@lemmy.worldEnglish · 1 month ago
message-square
41
link
fedilink
  • cross-posted to:
  • technology@beehaw.org
  • technology@lemmy.world
Bots are currently scraping the internet for LLM training data at unprecedented rates[1][2][3], driving up costs and destabilizing public-facing websites. I want to talk about how this has been particularly difficult for wikis, and has gotten much worse in the last few months.
  • Thorry@feddit.org
    link
    fedilink
    arrow-up
    15
    ·
    1 month ago

    Yes, I use this block list as well as my own additions (mostly IPs of misbehaving bots):

    https://github.com/mitchellkrogza/apache-ultimate-bad-bot-blocker

    It’s specifically for Apache, but that’s what I use. There are more of these kinds of lists available.

Fuck AI@lemmy.world

fuck_ai@lemmy.world

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !fuck_ai@lemmy.world

“We did it, Patrick! We made a technological breakthrough!”

A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.

AI, in this case, refers to LLMs, GPT technology, and anything listed as “AI” meant to increase market valuations.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 827 users / day
  • 3.59K users / week
  • 8.67K users / month
  • 17.3K users / 6 months
  • 6 local subscribers
  • 7.44K subscribers
  • 3.13K Posts
  • 46.5K Comments
  • Modlog
  • mods:
  • Proud Cascadian@lemmy.world
  • MrMcGasion@lemmy.world
  • TootSweet@lemmy.world
  • BigMikeInAustin@lemmy.world
  • cynar@lemmy.world
  • drmeanfeel@lemmy.world
  • pavnilschanda@lemmy.world
  • CriticalMedicine@lemmy.world
  • WonderfulWanderer@lemmy.world
  • Communist@lemmy.ml
  • eatCasserole@lemmy.world
  • SpaceNoodle@lemmy.world
  • NutWrench@lemmy.world
  • Soup@lemmy.cafe
  • Avicenna@lemmy.world
  • Tinks@lemmy.world
  • wizblizz@lemmy.world
  • corus_kt@lemmy.world
  • TrickDacy@lemmy.world
  • TheFriar@lemm.ee
  • Queen HawlSera@lemm.ee
  • andrew_bidlaw@sh.itjust.works
  • MeDuViNoX@sh.itjust.works
  • 33550336@lemmy.world
  • Nougat@fedia.io
  • Lost_My_Mind@lemmy.world
  • The D Quuuuuill@slrpnk.net
  • scratsearcher 🔍🔮📊🎲@sopuli.xyz
  • e8d79@discuss.tchncs.de
  • ThefuzzyFurryComrade@pawb.social
  • BE: 0.19.17
  • Modlog
  • Legal
  • Instances
  • Docs
  • Code
  • join-lemmy.org