We always have this misconception about what “non-deterministic” means. It doesn’t really mean what most people here think it does… And machine learning algorithms aren’t really non-deterministic in the way people think they are.
For example, you’d train an image classifier to categorize pictures correctly. If there’s a cat on it, it kinda needs to return “cat”. It’s not like they roll dice and deliberately return “cauliflower” instead. It doesn’t always do it correctly… I think that’s what you were trying to say with the non-determinism?! But we have classifiers who do it correctly in 99 point something % of cases…
So the way I think it could work is: HuggingFace uses Gradio to render those “spaces”. So every time an image element is rendered, it could pipe it through an image classifier and see whether there’s nude female breasts in the picture. If it “thinks” there are, discard the image and don’t return it to the user… It’d use a lot of compute, so be expensive. And probably come with a lot of false positives. But it’s not like it’s impossible to do these things. That’s usually how these safety/censorship measurements (“guardrails”) are set up.
The false positive from non-determinsm would appear, if you provide instructions “don’t give me a cat, ever!” it will return a cat sometimes if it is related in the model’s weights.
But yes perhaps the platform can run a filter that rejects nsfw output for any model that is being run as a demonstration on their servers, but it can’t really prevent people downloading a model and setting up an environment that makes nude image edits themselves.
Hmmh. A classifier doesn’t really have any instructions. Just an input (the image) and an output (the predicted category label). So that’s not really an issue, I think.
Downloading and running the model someplace else obviously is. I always like to think of software as a tool. It’s practically impossible to design an axe in a way it can’t do harm to anyone. Same with computers. And I don’t even know if there’s a point in trying. For liberal people, it’d probably be fine to offer dangerous items like an axe, gun or AI model, knowing it could hypothetically be used for nefarious purposes.
I think the red line is doing it, after being well aware the person is planning to commit a crime with the gun. That’s kind of unethical?! And maybe we should try to avoid handing out axes or random car keys to the town drunk or a clumsy 5 year old, as well. I think that’s pretty much where service providers are at. Once it’s a service, they have all the data… What prompt goes in and what image comes out, it all happens on their servers. And they know perfectly well what’s going on and who’s doing it. I think that’s the point it becomes some sort of obligation for them to care. In theory, at least.
But HuggingFace is -by far- not amongst the worst offenders. I believe most of the bad stuff happens on other websites. And there’s dedicated services for AI pr0n without any censorship. So I doubt they’ll have an impact in the broad picture.
We always have this misconception about what “non-deterministic” means. It doesn’t really mean what most people here think it does… And machine learning algorithms aren’t really non-deterministic in the way people think they are.
For example, you’d train an image classifier to categorize pictures correctly. If there’s a cat on it, it kinda needs to return “cat”. It’s not like they roll dice and deliberately return “cauliflower” instead. It doesn’t always do it correctly… I think that’s what you were trying to say with the non-determinism?! But we have classifiers who do it correctly in 99 point something % of cases…
So the way I think it could work is: HuggingFace uses Gradio to render those “spaces”. So every time an image element is rendered, it could pipe it through an image classifier and see whether there’s nude female breasts in the picture. If it “thinks” there are, discard the image and don’t return it to the user… It’d use a lot of compute, so be expensive. And probably come with a lot of false positives. But it’s not like it’s impossible to do these things. That’s usually how these safety/censorship measurements (“guardrails”) are set up.
The false positive from non-determinsm would appear, if you provide instructions “don’t give me a cat, ever!” it will return a cat sometimes if it is related in the model’s weights.
But yes perhaps the platform can run a filter that rejects nsfw output for any model that is being run as a demonstration on their servers, but it can’t really prevent people downloading a model and setting up an environment that makes nude image edits themselves.
Hmmh. A classifier doesn’t really have any instructions. Just an input (the image) and an output (the predicted category label). So that’s not really an issue, I think.
Downloading and running the model someplace else obviously is. I always like to think of software as a tool. It’s practically impossible to design an axe in a way it can’t do harm to anyone. Same with computers. And I don’t even know if there’s a point in trying. For liberal people, it’d probably be fine to offer dangerous items like an axe, gun or AI model, knowing it could hypothetically be used for nefarious purposes.
I think the red line is doing it, after being well aware the person is planning to commit a crime with the gun. That’s kind of unethical?! And maybe we should try to avoid handing out axes or random car keys to the town drunk or a clumsy 5 year old, as well. I think that’s pretty much where service providers are at. Once it’s a service, they have all the data… What prompt goes in and what image comes out, it all happens on their servers. And they know perfectly well what’s going on and who’s doing it. I think that’s the point it becomes some sort of obligation for them to care. In theory, at least.
But HuggingFace is -by far- not amongst the worst offenders. I believe most of the bad stuff happens on other websites. And there’s dedicated services for AI pr0n without any censorship. So I doubt they’ll have an impact in the broad picture.
AI output on images need a content regulator? Like an independent AI screening tool similar to material safety testing agencies?
Has to pass multiple passes but then we are just accepting the tech burden of this technology