The Moroccan Arabic name, meaning "The Small Castle", can be transcribed l-Qṣər ṣ-Ṣġir or Ksar Sghir. The name distinguishes it from Ksar-el-Kebir ("The Big Castle"), which is farther south. The Spanish name used to translate this as Castillejo but now transliterates it as Alcázar Seguir or Alcázarseguir; its Portuguese equivalent is Alcácer-Ceguer. Under the Almoravids and Almohads, it was known as Qasr al-Majaz, Ksar al-Majar, or Ksar al-Djawaz ("castle en route") because it was an important embarkation port for Moroccan troops on their way to Spain. Other names for the Muslim fortification include the 11th-century geographer al-Bakri's al-Qasr al-Awwal ("The First Castle") and the 13th-century historian Abdelwahid al-Marrakushi's Ksar Masmuda ("Masmuda Castle"), after the local Berber tribe.
In antiquity, it was known by the names Lissa and Exilissa (Greek: Ἐξίλισσα), which Lipiński conjectures represent the survival of the Phoenician settlement's name Ḥiq or Ḥeq-še-Elišša ("Bay of Elissa"). Note, however, that Pliny and Lipiński place the ancient settlement further east, closer to Benzú. The Byzantine Greek name was Exilýssa (Εξιλύσσα).