By: Emily Wang
A.I Chatbots like ChatGPT are seemingly struggling with math. These chat-bots can summarize texts, write essays, and manipulate words on a whim, but math seems to stump them every time. So what is causing this?
Kristan Hammond, a Computer Science professor and A.I researcher at Northwestern University says “The A.I chatbots have difficulty with math because they were never designed to do it.” A summary of computing, since it became a popular subject, has basically been math on steroids, ever since computers first appeared in the 1940s, as tireless, fast, and remarkably accurate calculating machines, far outperforming humans. Computers were programmed to follow step-by-step rules, making them powerful but brittle, so any efforts of achieving A.I hit a brick wall.
But over a decade ago, a new method called a neural network, loosely modeled off the human brain, broke through making it able to comprehend words, based on data it has, and can predict what word or phrase comes next, just like humans. A.I has stumbled with math that needs multiple steps before a solution at times. Its proficiency is improving, but still unsatisfactory. Recently, Kristen DiCerbo, Chief Learning Officer of Khan Academy, which has been experimenting with A.I chatbot tutors, introduced the subject of math accuracy. Khan Academy also made significant changes to Kanmingo, their A.I powered tutor. By using a calculator program, designed to do math, it can provide correct answers.
OpenAI said in a statement that math is an “important ongoing area of research”, and a field in which scientists have made nearly 64 percent accuracy on public databases, which contain thousands of questions that require visual perception and mathematical reasoning, they said. This version is a whole 58 percent more accurate than the last one.
The erratic performance in math, fueled an already ongoing debate about the best way to go forward in the field of A.I, which is mostly split into two groups. One side believes that the neural networks that run A.I chatbots is a singular path to progress, and eventually to A.G.I, or artificial general intelligence, a computer which is as capable as a human brain, which is the dominant view in silicon valley. There are also skeptics that wonder if adding more data as well as computing firepower to the language models is enough.
Prominent in the latter group: skeptics that wonder if adding more data as well as computing firepower to the language models is enough, is Yann LeCun, Chief A.I Scientist at Meta. Dr. LeCun has said that the large language models lack logic and common sense reasoning. He insists that what’s needed is a wider approach, which he dubs “world modeling”, or systems that can learn like humans about the world, and it may take over a decade to accomplish.
However, Kirk Schnider, who is a high school math teacher in New York, states that he thinks using A.I chatbots in everyday education is inevitable. School administrators can try to ban them, but students are still going to use them. However, he has some doubts, “They’re usually fine, but usually isn’t good enough in math. It’s got to be accurate”. For his students, a life lesson worth remembering long after they have forgotten about other things is: Don’t believe everything A.I tells you. Don’t trust it too much.
Image Credit by AEI.org