Shimōsa was originally part of a larger territory known as Fusa Province (総国, occasionally 捄国 Fusa-no-kuni), which was divided into "upper" and "lower" portions (i.e. Kazusa and Shimōsa) during the reign of Emperor Kōtoku (645–654). It was well-known to the Imperial Court in Nara period Japan for its fertile lands, and is mentioned in Nara period records as having supplied hemp to the Court. Shimōsa was divided into 11 (later 12) counties. The exact location of the capital of Shimōsa is not precisely known, but is believed to have been somewhere within the borders of the modern city of Ichikawa, Chiba, near Kōnodai Station where the ruins of the Kokubun-ji have been located. However, the Ichinomiya of Shimōsa Province is the Katori Jingū in what is now the city of Katori, Chiba, on the opposite coast of the province.
During the Heian period, the province was divided into numerous shōen controlled by local samurai clans, primarily the Chiba clan, which sided with Minamoto no Yoritomo in the Genpei War. During the Kamakura period, much of the province was under the control of the Chiba clan. By the early Muromachi period, the area was a highly contested region highly fragmented by various samurai clans. By the Sengoku period, the Later Hōjō clan held sway following the Battle of Kōnodai (1538) against the Ashikaga clan and the Satomi clan.
Following the installation of Tokugawa Ieyasu in Edo, after the Battle of Odawara, he created eleven han within the borders of Shimōsa to reward his followers, with the remaining area retained as tenryō territory owned directly by the shōgun and administered by various hatamoto. The entire province had an assessed revenue of 681,062 koku.
Following the Meiji Restoration, these various domains and tenryō territories were transformed into short-lived prefectures in July 1871 by the abolition of the han system. Most of Shimōsa Province became part of the new Chiba Prefecture on June 15, 1873, with four districts (Yūki, Toyoda, Sashima, Okada) going to the new Ibaraki Prefecture and the portion to the west of the Edogawa River going to the new Saitama Prefecture.