Maneuvering targets tracking is a fundamental task in intelligent vehicle research. This paper focuses on the problem of fusion between radar and image sensors in targets tracking. In order to improve positioning accuracy and narrow down the image working area, a novel method that integrates radar filter with image intensity is proposed to establish an adaptive vision window.A weighted Hausdorff distance is introduced to define the functional relationship between image and model projection, and a modified simulated annealing algorithm is used to find optimum orientation parameter. Furthermore, the global state is estimated, which refers to the distributed data fusion algorithm. Experiment results show that our method is accurate.