The classification problems aim to associate a set of pattern to one or more classes. With pattern we identify a multidimensional array of data labeled by a pre-determined tag. In this case we talk about supervised learning, i.e the full set of data is already annotated and we have prior knowledge about the association between data and classes.
In machine learning a key rule is played by Bayesian methods, i.e methods which use a Bayesian statistical approach to the analysis of data distributions. It can be proved that, if the underlying distributions are known, i.e a sufficient number of its moments are known with a sufficient precision, the Bayesian approach is the best possible method to face the classification problem (Bayesian error rate).