Outdoor environment exchanges heat with indoor environment, enabling pollutants to infiltrate indoors, affecting buildings’ energy efficiency, comfort, and indoor air quality. Investigating the impact of the outdoor environment on the indoor thermal environment is crucial. Firstly, this paper reviews the coupling method to link the outdoor environment with the indoor environment. Secondly, it examines the impact of the outdoor physical environment, including neighboring buildings, greening, road surface, water body, and sky, on the indoor thermal environment. During the hottest summer, an increase of 17% in trees can reduce indoor temperature by 1.1 °C. Thirdly, the impact of weather conditions, including outdoor temperature, outdoor humidity, external wind, global warming, extreme weather conditions, and solar radiation, on the indoor thermal environment is studied. Due to global warming, cooling energy consumption and heating energy consumption in 2050 could increase by 223% to 1050%, and heating demand could decrease by 36% to 58%. Finally, the impact of outdoor air pollution on indoor environment and energy consumption is analyzed. For every 75 μg/m³ increase in PM_2.5 concentration, average power consumption could increase by 11.2%. Recommendations for future research are provided. This study contributes to the understanding of the outdoor–indoor thermal relationship and offers insights into enhancing indoor thermal comfort and reducing building energy consumption.