This article has been updated. It was originally published on November 23rd, 2023.

1. Outsourcing Human Capabilities

In Nicholas Carr’s book, The Shallows, he advances the idea that modern technologies like the internet, hard drives, and other software innovations can negatively impact natural human abilities. To motivate this view to those who haven’t read his book I will sketch out a general form of his argument. Then I will show a particular cases as examples to help align our intuitions.

Carr’s Outsourcing Argument

  1. The Neuroplasticity of the human brain adapts it to learn or shed behaviors which become needed or not needed.
  2. Modern technologies make certain human behaviors less necessary
  3. Therefore, modern technologies change the human brain.

On the first premise, the property of neuroplasticity captures the common sense idea of use it or lose it. If you stop practicing an instrument, you will get worse at it, same if you stop practicing arithmetic. The idea is not that you will forget how to do these things. You won’t forget the steps on how to multiply. You will still be able to compute a multiplication if given the adequate time. What changes is how well you perform this capability. Same if you stop exercising in that case, you lose fitness. In many places in our life, if you stop performing a behavior your ability to do that behavior atrophies.

Moving to the second premise, modern technologies continue to gain capabilities that make our lives easier. One key example from Carr’s work is the invention of the internet. The internet makes information rapidly accessible to our fingertips. If you want to know the names of the three people who first visited the moon you can just pull out your phone and within seconds have the answer. More than just information, if you want to know what 12 * 18 is you can just pull out your phone, open the calculator and have your answer. These new capabilities surely extend the human intellect. We can perform arithmetic at super human levels with this technology. We can learn, access and know the joint collection of all human knowledge with the internet. Carr’s point is that with this ease of access we should be aware that we vulnerable to losing a human capability. You can now be a perfectly effective engineer without ever needing to be good at arithmetic, just lean on your calculator. You can be the greatest trivia expert, just lean on Google. As we use calculators, our innate calculating skills dull. As we use Google, our need to remember all the information we learned becomes less important. In this latter example, we are outsourcing our knowledge and memory to the internet. This demonstrates the idea that as we become reliant on new technological inventions, the worse some of our innate human capabilities become.

To some this concern may be skirted by evolving needs in our society. Some may argue that these new technologies level the playing field for all people. As we said before, if you weren’t good at arithmetic, you can still become a competent engineer, making a field which may have been previously inaccessible to some, accessible to all. Others can push against this by presenting concerns that these innate capabilities are part of what it means to be a human. Are we ok with losing the experience of number crunching? What about the feeling of thinking hard with some friends to remember who was the 7th president? More importantly, there may be side effects to losing certain capabilities we are unaware of. For instance, one of Carr’s implications is that our diminishing need to tap on our memory and recall information may impede our working memory as a side effect. Working memory is one of the key parts of human cognition that allows us to hold multiple ideas in our mind all at once to solve problems. We need to think hard before we allow one of our cognitive capabilities to atrophy. This I hope demonstrates the full thrust of Carr’s concern.

AI’s New Capabilities:

ChatGPT’s creation ushered in a whole bevy of new capabilities to AI systems. GPT-4o, a state of the art language model by OpenAI, can perform all the following at or near human levels.

  1. Natural Language Understanding and Generation
  2. Translation
  3. Text Summarization
  4. Information Retrieval and Research
  5. Code Writing and Debugging
  6. Grammar and Style Correction

While extremely impressive, many of these abilities have existed in some form already, especially considering pre-existing tools found on internet. The new emerging behavior, however, that I think is worth spending time thinking about is reasoning. These new AI models have been shown to have the ability to reason similar to that of a human. For instance, if you propose a math word problem to GPT-4 it can utilize logic to and perform arithmetic to answer it eerily similar to how a human would:

Question: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?

GPT-4’s Answer: The cafeteria had 23 apples originally. They used 20 to make lunch. So they had 23 - 20 = 3. They bought 6 more apples, so they have 3 + 6 = 9. The answer is 9

This ability allowed GPT-4 to score in the top 90th percentile on the notoriously challenging Bar exam, the top 88th percentile on the LSAT, and a 4 on the AP Calculus BC exam. GPT-4 is undoubtably doing something akin to human reasoning to achieve scores like these. It, like a human, can apply its knowledge base and then reason generally about situations to arrive at correct answers. GPT-4 and other AI Language Models still fail notably at other complex reasoning cases. Fundamentally, these models perform all their abilities by predicting the next word in a sentence. This has been shown to be inadequate in handling problems requiring complex reasoning where working memory and future planning is required. However, even these inadequacies have been reduced when you simply tell the model to ‘think it through step-by step’. Adding this simple request to GPT-4 improves its reasoning capabilities substantially. Additionally, some work has gone into providing models like GPT-4 with ‘scratch pads’ that allow them to have a ‘working memory’ to think and reference intermediate steps necessary for reasoning. Even with these enhancements, current AI models still struggle in certain scenarios, but it is important to note that improving the performance of these models is still a field of ongoing research and future improvements are to be expected.

3. The Future of Human Reasoning with AI

If we take Carr’s outsourcing argument seriously, and we suppose that AI’s reasoning capabilities will continue to improve, then it seems like human reasoning may be at risk of atrophying. If AI models become common place in society and are used to solve problems, then it is reasonable to ask how this will shape our innate reasoning. The previous human capabilities lost to AI seem to be importantly different than reasoning. To no longer be able to compute the multiplication of two 3 digit numbers seems to be expendable. It won’t directly change the way you live and experience the world. To become worse at remembering certain facts about the world spelled a great concern in the ways it may impact our working memory. At bottom, however, one could still argue that these new tools allowed us to reason about the stuff that mattered. We could now build bridges easily, write papers to discover new ideas with research on the internet. We could equip ourselves to know more and solve harder problems. These technologies still preserved something unique to be performed by humans and that was our ability to reason about the world.

Reasoning is crucial to the human experience, it is the capability that Aristotle famously used to separate humans from all other animals. Further, to Aristotle what made human life worth living was it’s ability to exercise this function. It seems like now, with new developments in AI this feature of human cognition may no longer be unique. As I said above, if we take Carr’s argument seriously, this implies that as a society we may be moving toward outsourcing our human reasoning to AI. This fact is terrifying on its face. It seems like a transformational and existential threat on what it means to be a human being. This concern, the ethical implications and ways forward for humanity are obviously problems worth considering. These are problems which desperately need to be addressed, and sooner than we may have previously expected. The invention of this sort of Artificial General Intelligence was something which was thought to be far away. New developments have shown that this may not be the case. As for an immediate way forward, we need to be aware of this influence. No one with clear eyes will gladly shut off their cerebral cortex to embrace AI, but these changes happen slowly. No one willingly walked into having a diminished attention span, rather it was the steady increase of consumption in short form content and multitasking enabled by technology. As I understand Carr’s argument it is not that we clearly acknowledge and consent to outsourcing our capabilities. It is that as humans we naturally seek things that make our lives easier. As our lives become more complicated in modern society we seek more complicated tools to deal with these new problems. Before the modern farming machines, we needed the muscle strength to plow lands by hand to eat food. Now, since we can acquire food and eat without needing this muscle, the human body sheds it. Then we create societies which are at scales that the human body alone would be impossible to sustain. So likewise, when AI machines become reasoners at a levels greater than humans, we can expect naturally our human brains may begin to shed the cognitive abilities needed to solve complex problems. It will take hard and optional work to retain these abilities. I hope to expand on possible ways forward, but in the mean time it will require us to resist certain human impulses to want to outsource. AI will enable humanity to solve some of our greatest problems, but we will need to find a path forward which allows us to conserve our most central of traits to retain our humanity as we do so. For what is the value of world with all our problems solve if we were left without the cognitive abilities to even appreciate it?

4. The Frontier of Artificial Reasoning

Since I originally published this essay, OpenAI has developed a series of “reasoning” models. These models are specialized to solve problems which require longer periods of thinking. Instead of just responding immediately, these models spend time generating text which is never shown to the user. Bizarre as that might sound, researchers essentially allow the model to read and write to a scratch pad. The model uses this scratch pad in a sophisticated way which it learned from real human reasoning examples. With these examples it was fine-tuned to break down questions, solve intermediate problems, check its work and give a final response based on its work. These models implement an improvement in capability at inference time, rather than training time, a paradigm shift which has excited many researchers. Many skeptics decry the invention of artifical “reasoning” by supposing the models are still just predicting the next word, but in a way we find more relatable. They contend that the model still lacks a true understanding of any concepts and their semantic meaning. These voices are not wrong, SOTA reasoning models like o1 and o3 still show occasional failures on trivial human reasoning tasks. Importantly they do this while also showing impressive benchmark improvements over every other language model created. For example, o3 is ranked in the top 15 on competitive coding tasks and recieves at 99% score on the AIME, a challenging formal math exam. Most shockingly o3 got a 25% score on the EpochAI’s FrontierMath exam, an exam made up of problems that PhD level mathematicians struggle to solve. How can we make these two idea square in our heads? Language models are both displaying clear signs of some of the highest forms of reasoning, while also failing some of the lowest and most intuitive forms. I believe the takeaway is that these models aren’t engaging in “human reasoning”, but some clearly differentiated artifical version of reasoning. Does the rise of artificial reasoning mean it will diminish human reasoning? As for a practical end point here, replacing human reasoning with an alien form of reasoning feels like it might change how we view the world. The mere loss of comprehension of what AI systems are doing is frightening, but the impact on our own day to day lives with an increased reliance on these artificial systems is likely to be transformative. Like cars and machine made human exercise unnecessary, I think arbitrary cognitive workouts like reading, writing, and solving math problems will become a new requirement for a full healthy life.