So here’s the thing, AI labs got billions of dollars to spend on making this “rogue” accident happen. They can buy a lab, kit it out, do some random sequence mixing with already deadly viruses, have natural selection do the heavy lifting, and then claim that they were trying to cure cancer, but their AI is so good it made a supervirus. (Of course, the supervirus is unlikely to be all that good, but it doesn’t have to be, they’ll just hype up what could have happened).

That is basically what they did for hacking, and they didn’t get in any kind of trouble, so they will be escalating and escalating. Keep in mind these folks have always been about applying “fake it until you make it” attitude to the end of the world. They had been promising their investors a double digit % chance of ending the world, from the very beginning of these companies.

Skippy: Opening frequently asked questions item “Why can’t I kill more than fifty people?” Answer: “The fuck is wrong with? Go see a therapist, psycho. (note to self - rewrite later, do not forget).”

I think an even bigger problem here is - what are you bringing to the market, and why should you be making any money? Your AI provider has perfect knowledge of how your “organization” works, but they are far better equipped to optimize it, while you’re just using their API as a black box.

And 100% of them are just trying to suck up to the rich the hardest to get some cash thrown their way for posting. Their whole community has been built 100% around that from day 1.

Its also interesting that this is the most conservative, pro “its not just memorizing” estimation possible : they multiplied the probabilities of consequent tokens. Basically it means if it starts shitting out a quote it will not be able to stop quoting until their anti copy the whole book finetuning kicks in after 50 words or so.

It can probably output far more under a realistic test (always picking the top token, temperature =0)