The next Labour leader should see it as their duty to create the space to encourage real debate

It used to be said of the American Democrats that, in the aftermath of their repeated electoral drubbings in the 1980s, the party’s response was to form a circular firing squad.

This is a trait that Labour has frequently shared. It is an axiom of British politics that divided parties lose elections. The vicious infighting after Labour’s defeat in 1979 sparked by the hard left’s takeover attempt played a big part in keeping the party out of power for the next 18 years. Similarly, battles between supporters of Aneurin Bevan and Hugh Gaitskell in the 1950s helped create an impression of disunity which proved deeply damaging to Labour’s electoral prospects throughout the decade.

The Tories, too, have learned the high price voters exact from parties which turn on themselves. Parliamentary rebellions, leadership contests (both real and threatened) and attacks by rightwing newspapers destabilised John Major’s government and led to the catastrophe which befell it in 1997.

But, while unity may be a prerequisite of victory, alone it is not enough. Moreover, when calls for unity snuff out an honest analysis of the challenges a party faces in the aftermath of defeat, they can prove utterly detrimental to its political recovery.

The last five years provide a case study in these dangers. Labour’s leadership election in 2010 had many flaws but two were particularly damaging.

First, the contest itself failed to debate many of the issues that, after 13 years in power, the party desperately needed to address. In place of a thoroughgoing examination of what Labour got right and wrong in government, superficial and glib explanations for Gordon Brown’s defeat were offered. Decrying Labour’s record on immigration became a proxy for the much wider debate about its record that the party required.

Second, Ed Miliband’s election as leader was seen as bringing to a close the need for any further contest of ideas. But the notion that the party’s response to the new political, social and economic landscape it faced could be resolved in four summer months while much of the country was distracted by the novelty of a coalition government and, more likely, its summer holidays, was farcical.

The issues Labour had to grapple with – how to restore its battered economic credibility in the wake of the financial crisis, and the party’s purpose when it could no longer just switch on the public spending spigots – were never going to be resolved simply by the election of a new leader. By pretending that they had been, and that any attempt to discuss them was disloyal, the party did itself a huge disservice. Instead, it adopted a ‘safety-first’ approach, assuming that the implosion of the Liberal Democrats and the unpopularity of the Tories would hand it victory.

There were, of course, some who attempted to rouse Labour from its intellectual stupor. Whatever our disagreements with them at times, both Compass and those around Maurice Glasman’s blue Labour played such a role. Progress’ Purple Book attempted to fulfil a similar function. So too did a number of members of parliament on the party’s left and right. Sadly, however, those who were determined to shut down any form of debate chose not to engage with their arguments, but to brief against them and question their motives. Their attitude appears to have stemmed from a fear that they might not like the outcome of such a debate. As such, the calls for unity were little more than a guise for censorship.

Moreover, problems which reared their head after 2010 were similarly swept under the carpet: Labour’s defeat in the 2011 Scottish parliamentary election; the rise of the United Kingdom Independence party and how the party was harming Labour, not just the Conservatives; the fallout from the Scottish independence referendum; and the Tories’ ‘English votes for English laws’ proposals. On each occasion, Miliband’s team simply adopted a lowest common denominator, not shared, position.

Successful parties are those which are intellectually vibrant; they unite after, not before, debate and they coalesce around the conclusions. The intellectual ballast which underpinned the great reforming government of 1945 was provided by socialist thinkers such as RH Tawney, Evan Durbin and Michael Young. Giles Radice, Will Hutton and Tony Giddens, alongside groups such as the Labour Coordinating Committee, Renewal and Nexus, and revisionist thinkers around Marxism Today all played a part in Labour’s comeback in the 1990s. Indeed, however destructive its impact, it is indisputable that Margaret Thatcher’s election in 1979 and the longevity of Thatcherism owed a great deal to the intellectual energy of the Tory right in the 1970s.

Facing a crisis which is, as Jon Cruddas put it last month, ‘epic in scale’, Labour must learn from the mistakes of the last parliament. Unity based around nothing more than a dislike of the Tories is – and has been proven to be – not enough. Instead, the party needs an honest reckoning with why it fell so spectacularly short on polling day, to show the public it has heard its message, and a robust but comradely debate about how it can win in 2020.

Labour’s challenge in many ways is so great. The fracturing of the party’s support last month demonstrates that. It must wrestle not only with its parlous position in Scotland, but its loss of support to Ukip in many of its northern citadels and its utter rejection by the south of England.

But to overcomplicate why Labour ended up 100 seats behind David Cameron’s party risks obscuring a simple truth: no party wins when it is behind on leadership and economic competence. Getting ahead on both is Labour’s task. This is no easy assignment and is unlikely to be completed between now and September.

Instead of providing all the answers, the next Labour leader should see it as their duty to create the space to encourage real debate. In so doing, he or she will take a first, huge, step towards avoiding the fate of their predecessor.