cross-posted from: https://lemmy.blahaj.zone/post/46091883
Though nobody expected it in the first place, AI is not perfect and can make mistakes (obviously 🙄) What’s your experience with this? I’d love to hear about it.
cross-posted from: https://lemmy.blahaj.zone/post/46091883
Though nobody expected it in the first place, AI is not perfect and can make mistakes (obviously 🙄) What’s your experience with this? I’d love to hear about it.
Not too long ago, I was assigned Review on a clearly vibecoded PR. It was for a feature that parsed hourly values from a csv file and put them into the Database. Easy enough at first glance, just loop through the hours of the day and grab the value for each hour, with an error if your file doesn’t contain all the hours of the day. But for some reason my coworker not only decided to introduce a whole new “hour of the day” type to loop over instead of looping over numbers, but that type explicitly contained 25 values. It was the numbers 0 to 24. There was even a comment next to it saying that it had 25 values. All test cases were green too, because the test data was generated specifically to satisfy the tests. I had to explain to a grown ass adult that the day only has 24 hours and it wouldn’t work because there will never be a value for “24:00” in the real data. That specific coworker already made really stupid mistakes before AI was a thing, but at least those mistakes were easy enough to spot and didn’t secretly propagate themselves through thousands of lines of code.
The level of incompetence that AI can mask is absolutely stunning. (And this precisely is probably why the AI hype resembles so much Hans Christian Andersens Tale “The Emperor’s new Clothes”.)
I had to explain to a senior embedded software architect that you need to use locks to access and change the same variable in multiple threads in concurrent C++ code.
Make the variable a word size relevant to the processor’s memory access width (eg 32bit), don’t use atomic locks, mark it as volatile, write a comment about how things are perfect this way and lockers are losers; then rig the struct it’s in to have a 50% chance of misaligning the 32bits (so two cycles are required to read/write to it) every time someone changes and recompiles the code >:D
(Yes I no longer work as a programmer, no I don’t know why they don’t want me)
I once saw someone claim that slopcoded bugs would be prevented by the AI testing its code and tried to reason that no test from within the system could fully prevent errors within that same system. This is basically the kind of thing I had in mind: Manipulating the tests to fit the results instead of altering the code to fit the tests.
Testing has to come from outside, and with real data. Otherwise it’s just a meme of the AI giving itself a medal for passing its own tests.
Sort of a Trump Turing Test
Please don’t tell me you’re not using FIFA compliant LLMs? That’s negligence!
That is now even augmented by AI writing all of specs, docs, and comments. I am not joking - I was told that in an interview.
Wow. Mind blowing that people actually behave like this (and think implementing AI in this circumstance is a good idea)