Shanxi was the territory of state of Jin during the Spring and Autumn Period (722 BC - 403 BC), which underwent a three-way split into the states of Han, Zhao and Wei in 403 BC, the traditional date taken as the start of the Warring States Period (403 BC - 221 BC). By 221 BC all of these states had fallen to the state of Qin, which established the Qin Dynasty (221 BC - 206 BC).

The Han Dynasty (206 BC - 220 AD) ruled Shanxi as the province (zhou) of Bingzhou. During the invasion of northern nomads during the Sixteen Kingdoms period (304 - 439), what is now Shanxi was controlled contiguously by several regimes, including Later Zhao, Former Yan, Former Qin, and Later Yan. They were followed by Northern Wei (386 - 534), a Xianbei kingdom, which had one of its earlier capitals at present-day Datong in northern Shanxi, and which went on to rule nearly all of northern China.