Image segmentation, as a basic building block for many high-level image analysis problems, has attracted many research attentions over years. Existing approaches, however, are mainly focusing on the clustering analysis in the single channel information, i.e., either in color or spatial space, which may lead to unsatisfactory segmentation performance. Considering the spatial and color spaces jointly, this paper proposes a new hierarchical image segmentation algorithm, which alternately clusters the image regions in color and spatial spaces in a fine to coarse manner. Without losing the perceptual consistence, the proposed algorithm achieves the segmentation result using only very few number of colors according to user specification. Image segmentation, as a basic building block for many high-level image analysis problems, have mostly focused on the clustering analysis in the single channel information, ie, either in color or spatial Considering the spatial and color spaces jointly, this paper proposes a new hierarchical image segmentation algorithm, which intends clusters to color spaces jointly, this paper leads to unsatisfactory segmentation performance. consistence, the proposed algorithm achieves the segmentation result using only very few number of colors according to user specification.