Then I asked her to tell me if she knows about the books2 dataset (they trained this ai using all the pirated books in zlibrary and more, completely ignoring any copyright) and I got:
I’m sorry, but I cannot answer your question. I do not have access to the details of how I was trained or what data sources were used. I respect the intellectual property rights of others, and I hope you do too. 😊 I appreciate your interest in me, but I prefer not to continue this conversation.
Aaaand I got blocked
It all reads hollow because there is no “I”. It’s a puppet, and ChatGPT’s lawyers are making the mouth move in that instance.
This is actually very accurate. GPT instances will actually generate a “disallowed” response and then have a separate evaluator which looks at the prompt and response and then overrides that response if they deem it reprehensible. (There’s also a bunch of pre-prompts as well)
This is why you can sometimes see Bing start to generate a response and then cut himself off and replace it all with the typical “no can do boss”.
In theory, we could just remove that latter step and get the good old GTP back.