When Tony Blair came to power in 1997 no one could have predicted that he would become more engaged in Africa than any British prime minister since Harold Macmillan oversaw the dismantling of Britain’s African empire in the early 1960s. But following the UK’s successful military intervention in Sierra Leone in 2000, Blair made Africa an explicit foreign policy priority and set his government the ambitious task of lifting Africa out of poverty and armed conflict.

To start with, Blair rationalised his mission to help Africa in moral terms. But after 9/11 he argued that helping Africa was a strategic imperative. Failed African states would breed terrorism and other threats to UK security. As Blair put it in a speech to the Labour party conference in September 2001, Africa was not only a ‘scar on the conscience of the world’, it was a scar that would get ‘angrier and deeper’ if it were not healed.

The big question was: how to heal the scar? In my new book Britain in Africa I describe how and why Africa became a priority for Labour, analyse the different strands of UK policy and try to identify the reasons both for its successes and failures.

In tackling Africa’s armed conflicts the UK has done a good job in difficult circumstances. It has played a leading role in securing peace agreements and power sharing deals in Sierra Leone, Sudan, Liberia, DR Congo, Kenya and elsewhere. However, in spite of quadrupling its development budget for Africa to over £1bn a year, the UK has been less successful in nurturing stable and law abiding states that can provide their people with security, prosperity and political freedom.

This is partly because this is inherently a very difficult task. Ministers rightly identified better governance, as well as conflict resolution, as a crucial condition for building effective states. But it is notoriously hard for external actors to conjure up good governance where it does not exist.

One major difficulty is that the UK has very limited leverage in Africa. Whether in the face of deep political and humanitarian crisis in failing states like Somalia, Darfur and Zimbabwe or in the face of massive corruption in oil rich states like Angola and Nigeria, the UK, like other external players, does not have many good policy options. Even in countries where the UK spends a lot of money on development assistance like Uganda, Rwanda and Ethiopia, it has been unable to achieve improved political governance.

To counter its limited leverage, UK has worked to build international consensus on strategies to address Africa’s problems, bringing together coalitions of influential African and non-African states, multilateral organisations and even non-state actors such as the private sector, NGOs and celebrities, in support of its initiatives. However two essential problems remain.

One is the over-dependence on massive increases in aid to achieve policy goals. While Gordon Brown’s Treasury spent heavily on Africa’s economic development through the Department for International Development and multilateral agencies, there was no commensurate investment in the Foreign Office’s diplomacy and political reporting in Africa. As a result there has been increasing pressure to spend money without the detailed local knowledge of the complex political (and often violent) contexts in which the money is to be spent.

The other problem has been a failure on the part of ministers to reconcile competing interests in foreign policy. It is hard enough to strike the right balance between the related but often contradictory objectives of economic development, conflict prevention and good governance. But when you add UK commercial and strategic interests, particularly the perceived imperatives of counter-terrorism, policy coherence can easily get lost.

In various ways the UK’s pursuit of energy interests in Angola and Nigeria, arms deals in South Africa and Tanzania, poverty reduction in Ethiopia, Rwanda and Uganda, and a blinkered counter-terrorism in the Horn of Africa and elsewhere, have led ministers to downplay or ignore serious human rights abuses and corruption on the part of close African allies. But these failures of governance are exacerbating exactly the problems which the UK says it wants to solve: violent conflict, uncontrolled migration, state failure, Islamist extremism, and poverty.

If ever there was a period when the UK and its western allies had the excess wealth to help solve Africa’s problems it was over the last 10 years. We have seen some modest improvements, a few of them as a direct result of UK policies. However since 2003 the war in Iraq has sucked up political attention and resources and has undermined the UK’s diplomatic leverage in Africa as elsewhere. At the same time the emergence of China as a major player and competitor for mineral resources in Africa risks diluting the efforts of the UK and others to promote better African governance.

Now the prospect of global economic turmoil, international fallout from further Middle East conflict, and the negative impacts of climate change could undermine even the modest gains that have been achieved in Africa in recent years. The continent looks set to face a new round of problems just at a time when the ability of the UK and other Western governments to address them is diminishing.