Fast and accurate extraction of vascular structures from medical images is fundamental for many clinical procedures. However, most of the vessel segmentation techniques ignore the existence of the isolated and redundant points in the segmentation results. In this study, we propose a vascular segmentation method based on a prior shape and local statistics. It could efficiently eliminate outliers and accurately segment thick and thin vessels. First, an improved vesselness filter is defined. This quantifies the likelihood of each voxel belonging to a bright tubular-shaped structure. A matching and connection process is then performed to obtain a blood-vessel mask. Finally, the region-growing method based on local statistics is implemented on the vessel mask to obtain the whole vascular tree without outliers. Experiments and comparisons with Frangi’s and Yang’s models on real magnetic-resonance-angiography images demonstrate that the proposed method can remove outliers while preserving the connectivity of vessel branches.