What is overfitting in machine learning?

Overfitting occurs when a machine learning model learns not just the underlying patterns in the training data, but also the noise and outliers present in that data. This means that while the model performs exceptionally well on the training dataset—often achieving high accuracy—it struggles to generalize to new, unseen data. The core issue with overfitting is that the model becomes too complex and tailored to the training set, capturing random fluctuations rather than the actual trends. This results in poor performance when applied to validation or test datasets, as the model fails to make accurate predictions outside of its training context.

Understanding noise versus signal is critical here: the signal is the true underlying pattern, while the noise includes irrelevant, random fluctuations. A well-trained model should focus on learning the signal to make reliable predictions on new data, rather than just memorizing the training examples, which leads to overfitting.