Although there are a few ways to mitigate the risk, the only way to block it is to get AI to differentiate instructions from data, which is impossible today.
It is - and tons of bugs are just about fucking it up or just not doing it at all. Incomplete or faulty parsing and validation of untrusted input is absurdly common. It got better in the past decades, but most software developers write worse code than AI by now (not me though; obviously, I am still better than the best AI).
I just can’t anymore. Isn’t that like, the basic thing any program does? Who runs these companies?
It is - and tons of bugs are just about fucking it up or just not doing it at all. Incomplete or faulty parsing and validation of untrusted input is absurdly common. It got better in the past decades, but most software developers write worse code than AI by now (not me though; obviously, I am still better than the best AI).