Abū Yazīd's father Kayrād was a trans-Saharan trader born in Qastiliyya (modern Chott el Djerid); he grew up in Tozeur. After he grew up, he went to Tiaret, the Rustamid dynastic capital and the main center of Ibadi Kharijism in the Maghreb and took up teaching. The Nakkariyyah branch of Sufri Kharijism was named after him.
However, in 909 the Ismaili Shī‘ī Fatimids conquered the Rustamids and soon after the Sufri state of Sijilmasa to the west. Abū Yazīd moved to Tiqyus and began agitating against Fatimid rule in 928. When the Fatimid al-Mahdi died in 934, Abū Yazīd launched a rebellion in the Aurès Mountains and declared himself Shaykh al-Mu'minīn "Elder of the Believers", seeking aid from the Umayyad Caliphate of Córdoba in al-Andalus.
Early in his rebellion, Abū Yazīd was given a gray donkey which he used to ride, for which he received the nickname "Possessor of the donkey". Abū Yazīd also habitually wore a short woollen jubba (cloak) and with his conspicuous frugality, he recalled the Kharijite imams of Tiaret and Sijilmasa.
Abū Yazīd was initially notably successful. He took Baghai, then Tébessa, Medjana, and several Tunisian cities including Béja, where he is said to have massacred the civilian population. The population of Tunis threw out their governor and let Abū Yazīd in. By the end of the year, he had conquered Kairouan itself, dealing several severe defeats to the Fatimid armies.
In 945, as Abū Yazīd besieged Sousse, Caliph al-Qa'im bi-Amr Allah died and was succeeded by his son al-Mansur Billah. Under al-Mansur's leadership, the Fatimid forces recovered their position, first breaking the siege of Sousse and then driving Abū Yazīd's forces out of Kairouan back into the Aurès Mountains. In 947, the Fatimids finally defeated them in the Kiyana Mountains near what later became Beni Hammad Fort.
One scholar argues that the Hausa culture hero Bayajidda represents a folk personification of the supporters of Abū Yazīd who fled North Africa after his defeat.
The various Bayajida legends in Hausa folklore describe how Bayajida, son of the king of Baghdad, came to Bornu and married the ruler's daughter. He later fled and came to Daura, fathering the founders of the seven Hausa states. The legends seem to be describing events which happened during the tenth century A.D. and Bayajida may be identical with the Ibāḍite sectary Abū Yazīd who resisted the Fāṭimids of Tunisia until he was killed by them in 947. The debris of his army may have fled across the Sahara and arrived in Bornu, then north of Lake Chad. After some time a part of this rabble which had remained unassimilated moved south-west and interbred with the indigenous inhabitants round Daura, forming the Hausa aristocracies. Different ingredients of the legends may be folk memories of events near Mecca, Berber myths of origin and perhaps Greek mythology, as well as accounting for the introduction of horses and the sinking of wells in rock by the incoming Berbers.