History’s most successful political party has a new leader and Labour ignores the significance of this event at its peril. It’s true that the weakened state of the Conservatives over the past 15 years – and the fact that David Cameron is the fifth leader the party has elected during this relatively short period of time – might tempt some into thoughts of complacency.

Cameron is, after all, probably the most inexperienced leader of the Opposition since the time of William Pitt the Younger. He is a massive gamble for the Tory party in a way that two young Labour leaders who had not previously held cabinet office – Neil Kinnock and Tony Blair – were not. Each of them had fought general elections as senior members of their party with high-profile frontbench positions. Whatever Cameron’s qualities – and Labour must recognise that his transformation from rank outsider in the leadership election to comfortable victor indicates that they should not be under-estimated – it is a sign of the Conservatives’ desperation that they should have placed such a high-stakes bet on a man who has sat in the House of Commons for just four years.

We should not forget too that the new Tory leader confronts a Labour government that, whatever its difficulties, has won three general elections. Cameron takes up the reins of a party which has not polled more than a third of the vote since John Major’s re-election in 1992. Even the nadir of Labour’s fortunes in the 1980s does not compare. The Conservatives have wasted their time in opposition in a way that Labour did not. Four years after losing power in 1979, Labour elected a leader who was not only a credible prime minister, but was also strongly committed to, and determined to drive through, the kind of deep and fundamental changes which would bring his party back to government.

Neither William Hague nor Iain Duncan Smith meet either of the ‘Kinnock test’ criteria, while Michael Howard, although conceivable as the occupant of No 10, lacked either the imagination or the will to begin the process of Tory renewal. Much of the advance the Tories made under Howard in the general election was a reflection of Labour’s weakness (principally, the defection of progressive voters to the Lib Dems), not the Conservatives’ strength.

At the next election, it is more than likely that Cameron’s rival to lead the country will be Gordon Brown. However the new Tory leader turns out, Labour’s probable leader will be a formidable opponent: the architect of the economic record which was key to Labour’s re-election in 2001 and 2005, a critical figure in the creation of New Labour and the modernisation of the party, and a master political strategist.

But a glance back at the Tories’ history underlines the dangers for Labour if it fails to recognise the potential threat Cameron poses. The Conservatives’ capacity for reinvention, their pragmatism and their overwhelming desire to acquire power rests at the heart of the phenomenal electoral success they enjoyed during the course of the 20th century. During that time, the party of the aristocracy not only accepted, but successfully exploited, the advent of universal suffrage; the party of laissez-faire accepted, and for a time promised to expand, the welfare state; and the party of Empire accepted, and then presided over, the decolonisation of the 1950s and early 1960s.

The Tory leader’s promise to fashion a ‘new, modern, compassionate conservatism’ needs to be seen in this light. At present, the nature of the process that Cameron intends to undertake is unclear. Two possible scenarios present themselves and both represent a real danger for Labour.

First, Cameron may intend to lead the Tories into the kind of far-reaching and substantive modernisation which Labour engaged in under Neil Kinnock, John Smith and Tony Blair. He may, as the Tories did under Churchill in the late 1940s, accept the new social and economic landscape of Labour’s time in office. Despite their initial opposition to much of it, we can probably assume, for instance, that the Tories will embrace the equalities legislation enacted since 1997, building on the accommodation they have already made with policies such as the minimum wage, international debt relief and devolution.

But what if Cameron goes further? Adapting Labour’s strategy in the run-up to the 1997 election, he could commit a Tory government to honouring Gordon Brown’s spending plans, promising only to go further in terms of reforming public services, delivering efficiency savings (however spurious), and reducing bureaucracy. Loose talk of tax cuts will be snuffed out in the same way that idle Labour chatter of new spending commitments in the 1990s was; the phrase ‘when circumstances allow’ becomes the constant refrain when any discussion of reducing taxation is authorised. The Europhobes may be prevailed upon to cease publicly engaging in their obsession. Cameron may take on the Tory old-guard and truly change the face of the Conservative party in terms of candidate selection. The Tory ‘Clause IV moment’ turns out to be one of substance: a promise – by better means, of course – to deliver on Labour’s pledge to eliminate child poverty.

As a comprehensive strategy this appears possible though not probable. Up to now, Cameron seems to have bought into the notion that the Tories’ main problem is one of image. According to this theory, if only the party were not perceived as out-of-touch, pessimistic, and loathing of the modern world, the widespread appeal of its policies (which, many Tories believe, does indeed exist) would bring it back to government. Cameron’s election is thus seen by some Conservatives as critical to the process of modernisation, simply because his youth, rhetorical moderation, and sunny optimism will transform the image of the party. ‘As soon as he is elected,’ suggested the right-wing commentator Bruce Anderson, ‘the party will appear to have modernised, as he is modernity personified.’

This thinking suggests the Tories may be more inclined to the kind of image-led modernisation epitomised by George Bush’s rebranding of the Republican party during the 2000 US presidential election. It’s worth remembering that Bush promised to end the harsh partisanship of Washington by bringing ‘civility and respect’ to the nation’s political discourse; prioritise education so that ‘no child is left behind’; and ‘extend the promise of prosperity’ with a ‘compassionate conservative’ war on poverty. Sound familiar?

The point, however, is not that, as Bush’s time in office has proved, the moderate veneer was principally a guise in order to ease the pursuit of traditional conservative goals: cutting taxes, slashing anti-poverty programmes, and ramping up defence expenditure (an objective long before 9/11). More crucially, the American experience suggests that a rebranding exercise, devoid of any real substance, can still be electorally successful.

That success owed much, of course, to the nature of the progressive response to it. Faced with a rhetorical challenge for the centre ground, the Democrat presidential candidate, Al Gore, vacated the territory staked out by Bill Clinton and moved sharply to the left. Gore compounded the problem by failing, for much of the campaign, to defend the record of the administration of which he had been a part: one which had presided over record economic growth, plummeting crime and poverty rates, and the creation of millions of new jobs.

These are critical lessons for Labour in the wake of Cameron’s election. Whatever direction he decides to take his party in, British politics is entering uncharted territory.

New Labour proved it could successfully exploit an era of Tory decline, but can it weather a time of Conservative revival? We are about to find out.