Based on a constructive leing approach, covering algorithms, we investigatethe relationship between support vector sets and kel functions in support vector machines(SVM). An interesting result is obtained. That is, in the linearly non-separable case, any sampleof a given sample set K can become a support vector under a certain kel function. The resultshows that when the sample set K is linearly non-separable, although the chosen kel functionsatisfies Mercer’s condition its corresponding support vector set is not necessarily the subsetof K that plays a crucial role in classifying K. For a given sample set, what is the subsetthat plays the crucial role in classification? In order to explore the problem, a new concept,boundary or boundary points, is defined and its properties are discussed. Given a sample setK, we show that the decision functions for classifying the boundary points of K are the sameas that for classifying the K itself. And the boundary points of K only depend on K and thestructure of the space at which K is located and independent of the chosen approach for findingthe boundary. Therefore, the boundary point set may become the subset of K that plays acrucial role in classification. These results are of importance to understand the principle of thesupport vector machine (SVM) and to develop new leing algorithms.