I was always convinced that the only reason we didn’t have this kind of tech is that it would just be prohibitively expensive to use.
Everyone hails transformers and attention heads as this huge breakthrough, but I’m here scratching my head how that’s a breakthrough when we’re just brute forcing through the problem…
Cory Doctorow explains it quite well in his podcast.
The ‘innovation’ is the replacement of the problem domain model (in earlier machine learning systems) with the brute force. This allows the billionaires/ruling-class/fascists to do away with troublesome experts (needed to create the model, but who may have ‘ethics’ and can say ‘no’) and just throw money at it.
I was always convinced that the only reason we didn’t have this kind of tech is that it would just be prohibitively expensive to use.
Everyone hails transformers and attention heads as this huge breakthrough, but I’m here scratching my head how that’s a breakthrough when we’re just brute forcing through the problem…
Cory Doctorow explains it quite well in his podcast. The ‘innovation’ is the replacement of the problem domain model (in earlier machine learning systems) with the brute force. This allows the billionaires/ruling-class/fascists to do away with troublesome experts (needed to create the model, but who may have ‘ethics’ and can say ‘no’) and just throw money at it.