After successfully overthrowing the Northern Song Dynasty (960-1127) in 1227, the Jin rulers moved the astronomical instruments in the capital at Bianliang (today's Kaifeng, central China’s Henan Province) to Beijing (then called Zhongdu, meaning Central Capital) and installed it in the Jin Chief Astronomer's Observatory. In 1279, when the Yuan Dynasty (1279-1368) defeated the Jin Dynasty (1115-1234), it established its capital in Beijing and built a new observatory just north of the site of the present-day structure. The instruments of the observatory were designed by Wang Xun and Guo Shoujing and built by Nepalese craftsman Arniko. For the last 500 years, they served virtually unchanged as the basis of astronomical work.

In 1368, Zhu Yuanzhang, the first Ming Emperor, moved the capital to Nanjing, and brought these astronomical instruments with him. In 1403, the third Emperor of the Ming Dynasty Zhu Di took power. And he moved the capital back to Beijing later in 1420. He did not dare to move these instruments because the tomb of the first Ming Emperor was in Nanjing. Instead he sent some artisans to the city in 1437 to make wooden copies of the Song armillary sphere and the Yuan guibiao (a type of sundial) and abridged armilla (a simplified form of the armillary sphere). A new set of bronze instruments was then cast in Beijing modeled after these wooden copies.

At the same time, a new observatory was constructed on the site of the water tower to the southeast of the old capital. It was during that period that the Ancient Beijing Observatory took on its present scale and layout and was equipped with such traditional instruments as the armillary sphere, the abridged armilla , and the celestial globe on the observatory platform, as well as the guibiao and the water clock below the platform. 

During the period from 1662 to 1722, Ferdinand Verbient, a Belgian missionary, was put in charge of introducing European astronomical measurements and instrumentation in the Imperial Astronomical Bureau. Between 1669 and 1673, he supervised the construction of a celestial globe, an equatorial theodolite, a zodiac theodolite, an altazimuth, a quadrant, and an ancient sextant. Later another altazimuth and an armilla were built in 1715 and 1744 respectively.

In 1900, when the Allied Forces of Eight Powers invaded Beijing, everything was looted at the observatory. The French troops shipped the equatorial armilla, the ecliptic armilla, the azimuth theodolite, the quadrant and the abridged armilla to the French Embassy to China in Beijing. Two years later in 1902, under the pressure of public opinion, these astronomical instruments were returned to China. The Ming made armillary sphere, and Qing made armillary sphere, and Qing made celestial globe, armilla, azimuth theodolite, and the sextant were taken away by the Germans to Beriin. It was not until 1921 that these instruments were sent back to Beijing after World War I in compliance with the Versailles Peace Treaty.