A built-up areas saliency index (BASI) to extract built-up areas is proposed in this article. The proposed method is designed to be especially effective for dealing with high-resolution remote sensing images and complex scenes. Due to the complexity of built-up areas in high-resolution images, visual attention model based on textural feature is applied for the calculation of BASI. To highlight built-up areas in complex scenes, we present an improved signal processing method to describe the textural feature of built-up areas, which is used as the low-layer feature of the visual attention model. Comparison studies and experimental results demonstrate the accuracy and robustness of BASI for discrimination between built-up and non-built-up areas from satellite and aerial high-resolution remote sensing data.