Al-Qaida has recruited an estimated 40,000 fighters since Sept. 11, 2001, when the Osama bin Laden-led extremist group attacked the United States, according to the not-for-profit Council on Foreign Relations.

Despite a United States-led global “war on terror” that has cost US$5.9 trillion, killed an estimated 480,000 to 507,000 people and assassinated bin Laden, al-Qaida has grown and spread since 9/11, expanding from rural Afghanistan into North Africa, East Africa, the Sahel, the Gulf States, the Middle East and Central Asia.

In those places, al-Qaida has developed new political influence – in some areas even supplanting the local government.

So how does a religious extremist group with fewer than a hundred members in September 2001 become a transnational terror organization, even as the world’s biggest military has targeted it for elimination?

According to my dissertation research on the resiliency of al-Qaida and the work of other scholars, the U.S. “war on terror” was the catalyst for al-Qaida’s growth.

Bin Laden and the ‘war on terror’

Al-Qaida was founded in Afghanistan in 1988 in response to the Soviet invasion of that country.

For decades, it was a small, weak and uninspiring movement. Bin Laden sought to raise an Islamic coalition of forces to establish a caliphate – an Islamic state governed with strict Islamic law – across the Muslim world. But as late as 1996 he had just 30 fighters willing to die for the cause.

For years, bin Laden tried to merge with such extremist groups as Egypt’s Ibn al-Khattab and the Libyan Islamic Fighting group, hoping to create a global Islamist movement.

These organizations rejected bin Laden’s overtures. These disparate groups lacked a common enemy that could unite them in al-Qaida’s fight for an Islamic caliphate.

So bin Laden shifted his strategy. He decided to make the United States – a country most Islamic extremist groups see as the enemy of Islam – his main target.

In 1998 al-Qaida waged successful attacks on the U.S. embassies in Tanzania and Kenya. In 2000, it bombed the USS Cole, a military ship refueling in a Yemen harbor, killing 17 sailors.

Bin Laden hoped the U.S. would respond with a military invasion into Muslim majority territory, triggering a holy war that would put al-Qaida at the forefront of the fight against these unholy invaders.

After al-Qaida operatives flew planes into the World Trade Center and the Pentagon on Sept. 11, 2001, killing 2,977 people, bin Laden got his wish. The United States invaded Afghanistan on Oct. 7, 2001. Eighteen months later, it invaded Iraq.

How al-Qaida grew

Islamic groups and individual extremists flocked to bin Laden’s cause after 9/11. Al-Qaida became the nucleus of a global violent Islamist movement, with affiliates across the Middle East and Africa swearing their allegiance.

At the same time, the war in Afghanistan was decimating al-Qaida’s core operations.

Leaders were killed by drone strikes or driven into hiding. The Bush administration claimed killing 75% of al-Qaida leadership. Bin Laden and other al-Qaida leaders sought refuge in places like the Federally Administered Tribal Areas of Pakistan and Yemen – remote areas outside the easy reach of U.S. ground forces.

To evade U.S. detection, al-Qaida had to limit communication between its newly decentralized fronts. That meant the group’s global leadership had to have autonomy to operate relatively independently.

Bin Laden expected al-Qaida affiliates to adhere to certain core values, strategies and, of course, pursue the objective of establishing an Islamic caliphate.

But newly minted regional al-Qaida leaders – people like Abu Musab al-Zarqawi in Iraq, Ahmed Abdi Godane in Somalia and Nasir al-Wuhayshi in Yemen – enjoyed enough autonomy to pursue their own agendas in these unstable places.

Al-Qaida Iraq, al-Shabaab and al-Qaida in the Arabian Peninsula, as their groups came to be known, embedded themselves in the local political scene. They began building credibility, establishing alliances and recruiting fighters.

By 2015, when bin Laden was killed, al-Qaida was a network of regional caliphates. Today its territory spans from Afghanistan and Pakistan to North Africa, the Middle East and beyond.

Manipulation of a sectarian divide

Al-Qaida in the Arabian Peninsula, headquartered in Yemen, is a case study in how the group now wields its power more locally.

Yemen has been in civil war since 2015, when a Houthi Shiite armed group declared war against the country’s Sunni Muslim government.

Although this conflict appears sectarian in nature, the Yemen scholar Marieke Brandt argues it is largely about political power – namely, the Yemeni government’s longstanding neglect of the Houthi minority, who come from northern Yemen.

Nonetheless, al-Qaida – a Sunni terror group – saw political opportunity in Yemen’s civil war.

The group has played up religious divisions in the civil war. Using its Arabic magazine, martyrdom videos, poetry and popular songs, al-Qaida has endeared itself to the local Sunni people and Yemen’s powerful Sunni tribal leaders. It has also ingratiated itself to Yemen’s Saudi Arabia-backed government and fought alongside Sunni tribal militias to battle the Houthi incursion.

The strategy has been remarkably effective for al-Qaida.

Al-Qaida in the Arabian Peninsula had hundreds of fighters at its founding in 2009. It now has about 7,000 fighters in Yemen, most of them Sunnis recruited from territory the Houthis have attempted to take over.

It has planted landmines and bombs across Yemen that have killed hundreds, held journalists hostage and, in 2015, orchestrated the massacre at the offices of the Charlie Hebdo newspaper in Paris.

The U.S. government considers al-Qaida in the Arabian Peninsula to be the most sophisticated and threatening branch of al-Qaida.

Adapt the tactic, keep the mission

In adapting its methods to Yemeni culture, al-Qaida in the Arabian Peninsula has made some missteps.

In 2011, the group attempted to impose extremely strict Islamic rule over two areas it controlled in south Yemen. Al-Qaida instituted rigid punishments of the sort common in Afghanistan, such as cutting off the hands of a thief and banning the chewed stimulant plant called khat.

These extreme rules got al-Qaida run out of town by Sunni tribal militias.

The next time al-Qaida in the Arabian Peninsula asserted its political power over parts of Yemen left ungoverned in the chaos of civil war, in 2015, it did not rule directly over these territories. Rather, it allowed a local council to govern according their own norms and customs. And it kept the khat market open.

Al-Qaida also paid for long-neglected public services like schools, water and electricity – effectively becoming the state.

According to the International Crisis Group, a humanitarian organization, this softer stance helped garner the acceptance of the local population. That, in turn, ensured al-Qaida could keep using Yemen as a regional headquarters.

A similar shift from global to local has occurred in al-Qaida affiliates in Somalia, Iraq and Syria.

Al-Qaida is no longer a hierarchical organization taking orders from its famous, charismatic leader, as it was on 9/11.

But it is stronger and more resilient than it was under bin Laden. And the “war on terror” has helped, not hurt it.

[ You’re smart and curious about the world. So are The Conversation’s authors and editors. You can read us daily by subscribing to our newsletter. ]

Christian Taylor is affiliated with Warzone Initiatives, a nonprofit organization working to address conflicts involving non-state armed groups.