Sufism emerged early on in Islamic history, partly as a reaction against the worldliness of the early Umayyad Caliphate (661–750) and mainly under the tutelage of Hasan Al-Basri. Although Sufis were opposed to dry legalism, they strictly observed Islamic law and belonged to various schools of Islamic jurisprudence and theology.

This mystic tradition within Islam developed in several stages of growth, emerging first in the form of early asceticism, based on the teachings of Hasan al-Basri, before entering the second stage of more classical mysticism of divine love, as promoted by al-Ghazali and Attar of Nishapur, and finally emerging in the institutionalized form of today's network of fraternal Sufi orders, based on Sufis such as Rumi and Yunus Emre.

However, Sufism as a historical phenomenon emerged in the 7th/8th centuries CE through the preaching of a movement of ascetics, and developed in Baghdad in the 9th/10th centuries around some charismatic figures, the most influential of which is the master Junayd al-Baghdadi (d. 910).

The exact origin of Sufism is disputed. Some sources state that Sufism is the inner dimensions of the teachings of Muhammad whereas others say that Sufism emerged during the Islamic Golden Age from about the 8th to 10th centuries. However Islamic scholars predating the Islamic Golden Age were referred to as Sufis, like Hasan of Basra.