Sb2S3 solar cells with substrate structure usually suffer from pretty low short circuit current (JSC) due to the defects and poor carrier transport. The Sb2S3, as a one-dimensional material, exhibits orientation-dependent carrier transport property. In this work, a thin MoSe2 layer is directly synthesized on the Mo substrate followed by depositing the Sb2S3 thin film. The x-ray diffraction (XRD) pattes confirm that a thin MoSe2 layer can improve the crystallization of the Sb2S3 film and induce (hk1) orientations, which can provide more carrier transport channels. Kelvin probe force microscopy (KPFM) results suggest that this modified Sb2S3 film has a benign surface with less defects and dangling bonds. The variation of the surface potential of Sb2S3 indicates a much more efficient carrier separation. Consequently, the power conversion efficiency (PCE) of the substrate structured Sb2S3 thin film solar cell is improved from 1.36% to 1.86% , which is the best efficiency of the substrate structured Sb2S3 thin film solar cell, and JSC significantly increases to 13.6 mA/cm2. According to the exteal quantum efficiency (EQE) and C-V measurements, the modified crystallization and elevated built-in electric field are the main causes.