Each new iteration of LLMs gets its coding capabilities ‘upgraded’, ’enhanced’ and more with great results from some software engineering scoring system. Everytime I see this I get excited and curious, but it always ends in disappointment for me. I find it a bit difficult to describe why exactly, but I’ll try to get some of my thoughts written up here.