In this article, we discuss the so called 'Curse of Dimensionality' and overfitting as a result, and explain why it is important when designing a classifier