At Meta, Microsoft, Salesforce and other large companies, devs are purposefully burning tokens (and money!) to inflate their AI usage and hit AI usage metrics which they treat as targets.
Most people have learned by now, that “lines of code“ is a terrible metric for evaluating productivity. Why are we doing the exact same thing with AI tokens now?
Because companies have been talking up how their adoption of AI is going to make them faster and more able to capitalise on opportunities in order to prop up their valuations for a while now and it seems to work as far as share price goes.
Being able back up this talk with metrics showing that their employees are all in on AI reinforces this, since the share price is the metric the business optimises for over product development employee reviews will index on this over cost effectiveness, and at most big tech companies engineers are very much making every decision with an eye to performance review optimisation (i.e. how it will affect their next review rather than the product they are building)
There is also some lesser incentives in that meta employees care directly about the meta share price since a lot of their compensation is in the form of RSUs.
I’m not condonig this as a desirable state of affairs, just explaining the incentive curve that the actors are following.
I’m no developer, just so some casual scripting for my job, but lines of code being a performance metric is a hilarious notion. Like, the indicator of good code is that it’s efficiently written in a small number of lines. It’s similarly just as easy to waste tokens on nothing of value.
A division of AppleComputer started having developers report LinesOfCode written as a ProductivityMetric. The guru, BillAtkinson, happened to be refactoring and tuning a graphics library at the time, and ended up with a six-fold speedup and a much smaller library. When asked to fill in the form, he wrote in NegativeLinesOfCode. Management got the point and stopped using those forms soon afterwards.
Most people have learned by now, that “lines of code“ is a terrible metric for evaluating productivity. Why are we doing the exact same thing with AI tokens now?
Before ai, my company’s misguided kpi was the number of merge requests
At least that one worked well for me since I’m generally making many small changes to an existing code base
You would be surprised to know how many managers still rely on this metric, even if it’s not part of their KPIs.
Because middle manglement has a constant compulsive need to justify their existence by finding new ways and metrics to “manage”.
Because companies have been talking up how their adoption of AI is going to make them faster and more able to capitalise on opportunities in order to prop up their valuations for a while now and it seems to work as far as share price goes.
Being able back up this talk with metrics showing that their employees are all in on AI reinforces this, since the share price is the metric the business optimises for over product development employee reviews will index on this over cost effectiveness, and at most big tech companies engineers are very much making every decision with an eye to performance review optimisation (i.e. how it will affect their next review rather than the product they are building)
There is also some lesser incentives in that meta employees care directly about the meta share price since a lot of their compensation is in the form of RSUs.
I’m not condonig this as a desirable state of affairs, just explaining the incentive curve that the actors are following.
I’m no developer, just so some casual scripting for my job, but lines of code being a performance metric is a hilarious notion. Like, the indicator of good code is that it’s efficiently written in a small number of lines. It’s similarly just as easy to waste tokens on nothing of value.
I love this story: