Does ChatGPT harm critical thinking abilities? A new study from researchers at MIT’s Media Lab has returned some concerning results.
The study divided 54 subjects—18 to 39 year-olds from the Boston area—into three groups, and asked them to write several SAT essays using OpenAI’s ChatGPT, Google’s search engine, and nothing at all, respectively. Researchers used an EEG to record the writers’ brain activity across 32 regions, and found that of the three groups, ChatGPT users had the lowest brain engagement and “consistently underperformed at neural, linguistic, and behavioral levels.” Over the course of several months, ChatGPT users got lazier with each subsequent essay, often resorting to copy-and-paste by the end of the study.
The paper suggests that the usage of LLMs could actually harm learning, especially for younger users. The paper has not yet been peer reviewed, and its sample size is relatively small. But its paper’s main author Nataliya Kosmyna felt it was important to release the findings to elevate concerns that as society increasingly relies upon LLMs for immediate convenience, long-term brain development may be sacrificed in the process.
“What really motivated me to put it out now before waiting for a full peer review is that I am afraid in 6-8 months, there will be some policymaker who decides, ‘let’s do GPT kindergarten.’ I think that would be absolutely bad and detrimental,” she says. “Developing brains are at the highest risk.”
OK, let’s slow down for one secound.
- if you re-designed it with complex multiplication problem instead od writtng tasks, and calculators instead of chat gpt - and change nothing else - you would get exactly the same result. If you reduce the challange to minimum, how do you expect brain to respond?
- The “we have this not-reviewed singular study based on 50 people we should raise the alarm NOW” is every sensational.
There are so many problems with AI, and we need so many checks and balances. Sure. But let’s not change it into another “our pop-science vs their pop-science” kind of problem.
And people brewing coffee on a keurig don’t think about the process of making coffee as much as someone who grinds their own beans and brews a pour-over.
The output of the two is different. And while they can serve the same function (providing caffeine in the form of a drink), if someone needs artisan coffee, the keurig will not do.
And if someone has only ever used a keurig, they may not know where to start.
Anyways, I contend that we need to do a better job actually interesting people in artisan essays, so the slop produced by LLMs is more easily identified as such.
Of course that requires a desire to educate people for a reason other than extracting economic value from them, so it’s unlikely to become widespread.
