Episode 12: How Does AI Learn from Mistakes? Understanding Growth Through the “Loss Function”

How Does AI Notice Its Mistakes?

What does it mean when we say that AI “learns”? For humans, when we make a mistake, we often think, “Next time, I’ll try doing it this way.” For example, if you get lost on your way somewhere, you might decide to check a map app before heading out next time. This process of learning from failure is actually something AI does too. At the heart of this idea is something called the “loss function.”

What Is a Loss Function? The Starting Point of How AI Learns

A loss function is a way for AI to quantify how much it got something “wrong.” For instance, if an AI looks at an image and says, “This is a cat,” but it’s actually a dog, the loss function calculates how far off that answer was. In other words, it’s like a reflection notebook for the AI.

The bigger the gap between the prediction and the correct answer, the more the AI recognizes that it was quite off this time. Then, using that insight, it starts adjusting its internal rules and calculations little by little. Through repeating this process over and over again, its predictions gradually become more accurate. This entire cycle is what we call “learning,” and the loss function is where it all begins.

Understanding Loss Functions Through Making Curry

Let’s look at a more relatable example. Imagine you’re making curry for the first time. You follow the recipe exactly as written, but when you taste it, it’s bland. You might think, “Maybe I should’ve added more salt?” That feeling of “it didn’t taste quite right” represents the kind of gap—or loss—we’re talking about. And when you adjust your seasoning next time to fix that gap, your curry gets closer to being delicious. This trial-and-error process is essentially how learning works.

But even with this idea of “loss,” there are many ways to measure it. For example, in classification tasks—like identifying images or text—the loss function compares how different the prediction is from the correct label. On the other hand, in tasks involving numbers like sales forecasts, it measures how far off the predicted value was from reality. So depending on what kind of task AI is doing, different types of loss functions are used.

Also worth mentioning is a method called “gradient descent,” which uses information from the loss function to update things inside the AI—like weights or rules—in order to improve accuracy. We’ll explore that topic in more detail in another article.

While useful, loss functions do have their challenges. If there’s bias in the data itself—for example if certain groups are underrepresented—even correct judgments can be mistakenly treated as errors. And since everything is processed as numbers, subtle human nuances can sometimes be lost or misinterpreted.

The Fascinating Way AI Grows From Its Mistakes

Still, there’s something almost human about this ability to learn from mistakes. It’s fascinating that machines can reflect on their errors and grow from them—it feels surprisingly lifelike.

In our next article, we’ll take a closer look at how AI not only learns from these gaps but also changes its learning approach itself based on those experiences. Don’t worry if everything doesn’t click right away—just building up those little moments of “Oh, I get it now” will take you far enough. Let’s keep moving forward together at your own pace.