AI can cheaply fake almost every signal we traditionally used to judge quality. I now see this pattern everywhere, in open source, in education, in research, all breaking at the same time. I find signaling theory useful for diagnosing what is happening and why it is so disorienting, and signaling theory and adjacent literature can provide the vocabulary to discuss possible solutions.
Not entirely onboard with how negative the perspective is but agree with the increase of noise making it harder to find real signals. This might sound simplistic but I just see it as the progression of technology and it’s just something you deal with like getting old. There are plenty of waves in technology that created new problems and it always leads to new solutions.
I don’t entirely agree with the doom and gloom of that article. Humans are particularly good at adapting to changes. And humans are particularly good at detecting patterns and anti-patterns. Patterns are signals, and while adaption will take a while and will be painful, I think we’ll work it out. For example, detecting AI slop reliably and intuitively is a skill that more and more people learn. Once the pattern is learnt, it’s a new useful signal. Other new signals emerge for quality detection. In a world of crap software, the few that work well will stand out. Anything that can be “created” by anyone with a few prompts loses its value. What’s valuable will be what stands out from the slop.
One of the things the author provided as a moderate answer is actually scarier than anything else he described. Delegation
Delegation plays right into the hands of people who made AI, because that’s what they want: for you to stop thinking about whether something is good or valid or not and trust other sources. Their sources, to be precise. It goes by a term I’ve seen thrown around that I like: cognitive surrender. You stop critically thinking about the problems and hope somebody else solved them.
Some things must delegate to some degree. As an example: Before, if we didn’t know how to code through something, we threw the question or problem at StackExchange or user forums and hoped a person more talented and experienced than us could answer. Beats bashing your head into a wall all day. And usually somebody eventually came through. Because some humans love helping and it doesn’t hurt if they want to show off either. They’ll even explain how and why it works. You got unblocked from your problem and then carried on doing everything else. The problem and solution are available for all to see. StackExchange doesn’t own the resulting contributions or gatekeep them, it just hosts them. This is healthy delegation.
Now you ask the equivalent of a magic 8ball to just do all of the work for you. That includes the knowledge to do so, and the quality control thereof. But the 8ball doesn’t actually understand your usecase. It’s not interested in helping you. It has no desires. It doesn’t really know what good projects look like. It doesn’t know if it even completed what you asked for unless you tell it that it did. When it’s “done,” you have a product of questionable quality and no idea how or why it made certain decisions. It can be prompted to explain itself, but it doesn’t really know. And it’ll give different people different answers. Your prompts and its decisionmaking and answers are private. Nobody except the AI owner knows what happened and why. Actually even AI corporations don’t have the complete why.
Now, propose that some team of people, let’s say Microsoft, starts to become the arbiter. They detect whether a project is completed and good enough. They give the project ratings based on metrics you can’t see or audit. Are they working in your best interests? Have they ever had your interests in mind? Who gets to audit whether or not their checks and balances are fair and correct? You don’t. You gave up that responsibility and now nobody except the entity that made the AI is even authoritative enough to answer.
The result is you don’t own the problem or the solution. You ask and you’re forced to be happy with what you get back. This is not delegation. Because you don’t have the responsibility anymore. This is surrender.
If we delegate, it has to be to everyone, not walled fortresses of corporate design. That’s what open source was designed to give. That’s what we have to claw back. Designating specific groups as arbiters of truth and quality creates an infinitely corruptible system. We’ve seen it before. People put in that place of power will abuse it. So the power must be distributed and shared by everyone instead.
There is no AI, what we have is precisely cognitive conformity as a service that replaces conventional information search. We all should be fucking scared.
I agree you don’t know if the AI is working in your your best interest. But at the same time you don’t know if the person answering your StackOverflow question, is acting in your best interest either. They could just be racking up SO points to unlock more mod powers.
If you’re saying that AI centralizes power, then I agree with you.
You can’t just live in plato’s cave and hope that the shadows on the wall are well-written code. But there will be a lot of people who try.
One of the best takes I have read in a while. It eloquently describes my main concern and worry. The destruction of curated knowledge by drowning out the signal with noise.
It worked for media to gain political power. First thing they went for is “balanced” news, and then they torpedoed the science/education sections by adding “junk” programming.
Great article! I’m floored.
I think all of the outcomes will happen in different domains and use cases. It all depends on the cost to verify trust and quality. Plenty can do with slop, but certain donations absolutely cannot.
Reminds me of the book Fall or Dodge in Hell where they use PERDAs and editors to verify authorship and build trust networks to combat fake signaling from AI. I think that book (while bleak) is a bit too hopeful. An excellent read though, well worth the time.
When production costs more due to allowing time and training for quality labor, and profit is reigned in for affordability, everyone wins.
