We propose a new basis for classification. The standard approach is to base classification on the basis of a set of features. In our approach we first compute dissimilarities from the set of features and base our classification on these dissimilarities. This leads to nonlinear classification boundaries in the original feature space. In contrast with machine learning tools, this procedure provides interpretable results in terms of distances to exemplars, whereas others are often seen as black boxes. We study various dissimilarity measures, both binary and multiclass classification problems, variable importance measures, marginal dependence relationships, and extensions to high dimensional problems.