The other week in my advice surgery I heard a constituent complain as he looked at the queue, ‘It’s like a bloody immigration office round here.’ When he reached my desk I braced myself as he slapped down his request for help, billed ‘taxpayers’ rights’.

Actually, the taxpayer in question was his Thai wife. She is working in a low-paid job, paying taxes, and he was horrified when he lost his job to discover that, because she has not yet passed the ‘life in the UK’ test and achieved settlement, she is not eligible for benefits.

As I explained the probationary status of newly arrived immigrants, he was clearly disbelieving. ‘But they,’ he declared, pointing back towards the many African and Asian constituents in the waiting room, ‘get benefits as soon as they arrive.’ The fact that a man whose own experience contradicts the red-top newspaper myths about migrants’ access to benefits still believes them, shows how hard it is to shift those fixed attitudes and prejudices.

Labour in government has not even tried. From the Crewe byelection claims that ‘only Labour wants ID cards for foreigners’, to Gordon Brown’s demand for ‘British jobs for British workers’, our language and policy suggests migrants represent a threat to stability, good order, public services and employment. It suggests that prejudice is justified.

Yet we came to government promising a fairer deal in immigration and many of our early reforms delivered it. That is what people expect of us. They know Labour is instinctively the party of racial equality. The most important lesson of the 10p tax fiasco should be that a party will lose public confidence when it tries to do something which goes against all its stated values. People felt furious, because this is not what Labour is supposed to do. We are not the party of taxing the poor to help the prosperous.

In politics what you do must always be based on your values. So trying to look tougher on ethnic minorities than the party which showed its disdain for racial equality by making no effort to discover why Stephen Lawrence’s murderers had remained unpunished was always going to fail. Tough measures which target migrants confirm the prejudices of those who believe that they gain unfair advantages. Such measures do not extinguish the flame of prejudice – they give it oxygen.

So apart from abandoning tough rhetoric how can we build confidence in our policies on race and immigration?

First, get on with it, do a competent job, the inefficiency in the system is one of the things that makes us vulnerable, the new points system may help because it is simpler.

People understand that globalisation has made migration easier. But they don’t want uncontrolled immigration which would advantage the pushy and greedy and threaten fair access to public provision.

So as well as rules about entry which work in practice, we need to build confidence about what happens when people get here. They have to be paid the minimum wage, they must conform to our health and safety standards, they have to wait their turn for services like housing which are in short supply. There should be equal chances to succeed in education and at work.

In theory this is what happens. But the deaths of Chinese cockle pickers brutally revealed a very different practice. And the only way to build confidence is to make practice match the promises.

The benefits of immigration are experienced nationally, with an injection of craft skills and young workers rebalancing our ageing population. But stresses are felt locally. Living close to people may not generate understanding, but could create a sense that ‘their problems are our problems’. And if these problems are tackled fairly the first step towards tolerance can be taken.

There is no short cut to alleviating public concern about immigration. But trying to win by being the nasty party persuades no one, and will make the problem worse.