Which is exactly what the original tweet was saying, is it not? Honestly, another LLM can read the code better. He’s still performing rigorous reviews.
I found debugging code with LLM most productive when I am in the loop. Don’t get me wrong it can quite often find tricky bugs, those requiring some sort of reasoning (i.e connecting together multiple relevant pieces of information to arrive at the conclusion). However it often also suggest fixing issues that are almost always irrelevant in practice yet the fix complicates and bloats the code. LLM itself even accepts it when confronted. That is the main problem, in an attempt to overachieve at the task it is given, it can do deceptive or impractical things that can have negative effects. You might try to fix this with a config file but it just turns into a tug-of-war. So I find it more practical and trustable to simply eyeball the bugs it has found, implement (or ask LLM to implement) corrections to those and be amazed at how it found some of those bugs.
Ofcourse if you have asked LLM to write the software from scratch, you don’t stand much chance of vetting the bugs via eyebaling. You have to spend much more time to understand how relevant they are. So your only option really is to defend the position of fully autonomous LLM coders…
For sure, but you don’t accept code from an intern without review much less have them write a full software without any supervision.
Which is exactly what the original tweet was saying, is it not? Honestly, another LLM can read the code better. He’s still performing rigorous reviews.
I found debugging code with LLM most productive when I am in the loop. Don’t get me wrong it can quite often find tricky bugs, those requiring some sort of reasoning (i.e connecting together multiple relevant pieces of information to arrive at the conclusion). However it often also suggest fixing issues that are almost always irrelevant in practice yet the fix complicates and bloats the code. LLM itself even accepts it when confronted. That is the main problem, in an attempt to overachieve at the task it is given, it can do deceptive or impractical things that can have negative effects. You might try to fix this with a config file but it just turns into a tug-of-war. So I find it more practical and trustable to simply eyeball the bugs it has found, implement (or ask LLM to implement) corrections to those and be amazed at how it found some of those bugs.
Ofcourse if you have asked LLM to write the software from scratch, you don’t stand much chance of vetting the bugs via eyebaling. You have to spend much more time to understand how relevant they are. So your only option really is to defend the position of fully autonomous LLM coders…