David Cameron needs to keep his headless chickens in the coop

The coalition will be strained to breaking point if the prime minister fails to face down the angry Tory right
***BESTPIX*** Prime Minster David Cameron Holds Press Conference
David Cameron needs to deal firmly with the To0ry right if the coalition is not to be put under heavy strain. Photograph: Peter Macdiarmid/Getty Images

Rudyard Kipling offers sound advice to political parties about how to respond to midterm election results. In triumph or disaster, the wise man treats those two impostors the same. Since the results began rolling in, climaxing with the dramatic cliffhanger in London that was announced minutes before midnight, two out of three of the main parties have responded in a rational fashion. The exception is the Conservatives. If I were David Cameron, I would worry quite a lot about the hammering suffered by the Tories, but I would be more alarmed that many in his party are responding to a reversal of fortune by losing their heads.

By contrast, Ed Miliband ought to be encouraged that his party has reacted in a sensibly sober way to these elections. Sentiment among Labour people can be summarised as encouraged but not intoxicated, which is the appropriate response to a performance that saw gains that were meaningful without being spectacular. Labour consolidated its dominance in the north of England, made important strides in the Midlands, restored its historic hegemony in Wales, did better than expected in Scotland against the Nationalists while making advances in the south and east of England in some of the places Labour will have to win at a general election to become the next government.

Another fillip is expected to be delivered from across the Channel today where a pointy-headed social democrat, once widely ridiculed as a no-hoper with all the charisma of a blancmange, is likely to supplant a showboating conservative as president of France. All of this makes the idea of prime minister Miliband just a little more plausible to his own party and a little more scary to his opponents, which will be a tonic for his morale and a confidence-booster for Labour. The internal mutters and external sniping about his flaws as a leader will subside, at least for a while. He has gained some momentum and opened up some space. Now the challenge is to show that he can do something with this opportunity.

None of which means that he is on an assured march to Number 10. The important thing is that this seems to be understood by Labour's leadership. They remember that Michael Foot and Neil Kinnock enjoyed midterm triumphs in the 80s against Mrs Thatcher but never went on to become prime minister. Mr Miliband has been rightly careful to stress that Labour still has "work to do" – and a lot of it – to win back public trust, especially on the economy. One of the senior aides who accompanied the Labour leader on his celebratory visit to Birmingham described it to me as "the humility in victory tour". Some humility is certainly called for. Labour's night was marred by Ken Livingstone's failure to recapture the mayoralty of London.

It is fair for Labour apologists to point out that the party did much better in the election for the London Assembly. There is truth, too, in their claim that the personality contest for the capital was something of a one-off. While Boris Johnson over-polled his party, Ken Livingstone under-polled his. But that raises a difficult question for the Labour leadership: why was the Labour candidacy stitched up for a politician who was past his sell-by-date with many Londoners? A related question was raised by Labour's defeat at the hands of George Galloway at the Bradford West byelection. Labour needs to think quickly and deeply about how it refreshes its selection processes to find attractive candidates for important contests.

Taking the elections across the country, Labour's overall share of the vote, at 38%, was a sparkling wine performance rather than a champagne one. This represents a massive improvement on its general election result of two years ago, but Labour didn't get to the psychologically and psephologically important threshold of 40% and fell even further short of the 43% vote share secured by Tony Blair a year before he entered Downing Street. The pitiful turnout – only one in three of those entitled to a vote bothered to cast it – is a further caution against Labour becoming overconfident.

There can be no danger of that with the Lib Dems. Catastrophic is not too strong a word to describe what has just happened to them. They went into these elections thinking that at least they could not possibly suffer as badly as they did last year when fury with Lib Dem "betrayals" was at its most intense. In the 12 months since, Nick Clegg has pursued a strategy of more aggressive differentiation from the Tories in the belief that this would claw back some credit and respect. What has that achieved? If these elections are any guide, the effect of Mr Clegg's switch of strategy has been precisely zero. His party took another severe beating. Over two years, they have lost more than 1,000 councillors, leaving them with fewer local representatives than at any time since the party was created. The one impressive thing about the Lib Dems is how they have responded to their punishment. Whatever profound despair they may be feeling in private, they have largely managed to stay composed in public. Their ejected councillors have displayed grace in defeat. Mr Clegg's parliamentary party, in contrast to Mr Cameron's crew, have not done an impersonation of headless chickens. The one advantage of all those decades that the Lib Dems spent in opposition is that it made them a stoical and resilient lot.

David Cameron must be rather envious of that when the Tory leader contemplates some of the hysteria erupting within his party. The prime minister and his under-strappers have tried to shrug off their reverses as no more than "midterm blues", but significant elements of his party have nevertheless gone into a funk. Lurch right! Stuff the Lib Dems! Make war on Europe! Slash spending harder! Get nasty on immigration! A cacophony of dissident Tory voices are shouting their unsolicited advice at the prime minister.

True enough, these were bad results for the Conservatives. In contrast to last year, when the Tory vote held up because many of the public were still prepared to give the government the benefit of the doubt on the economy, the Conservatives suffered serious losses. The odds on the Tories securing a parliamentary majority at the next election look longer today than they did on Thursday morning. The Cameroon ambition to revive the Conservatives in the great cities of northern England has gone nowhere except backwards. They are still without a single councillor in Newcastle, Manchester or Sheffield. In the Liverpool mayoral election, the Tory candidate came seventh. The one bright spot in the gloom – Boris Johnson's victory in London – also prompts the question: what has Boris got that Dave lacks? In many ways, it is unreasonable to compare a glorified transport commissioner who cleverly put distance between himself and the government with a prime minister presiding over punishing austerity who has no choice but to own everything done in the name of the coalition. But just because a question is unreasonable doesn't prevent many Conservatives from asking it. The answer they give is that Boris – at least in the current incarnation of the grinning blond – is offering a more authentic, undiluted, vibrant version of Conservatism. This argument is dangerous for the prime minister and the coalition if it takes a hold on his party.

The Tory reverses are bringing to a head the tensions that have simmered among Conservatives since the general election; the great, unresolved argument between those Tories who think David Cameron failed to achieve a clean victory in 2010 because he had not modernised enough and those who believe he failed because he was not right wing enough. The academic analysis of how voters behaved supports the first view much more than it does the second. The Tory party tried it the unreconstructed right's way at the general elections of 2001 and 2005 and went down to worse defeats than they did last Thursday. Yet that has not inhibited Tory right-wingers from seizing on these results to noisily press their agenda. Some of them were popping up before a single result had been announced to blame defeat on the desertion of core supporters alienated by the compromises made with the Lib Dems and issues such as gay marriage.

Maybe gay marriage did lose the Tories some votes to Ukip. But they lost many, many more voters over much more core issues. Their economic policy is not producing the results that it promised; living standards are being remorselessly squeezed; the budget was a disaster; and the run-up to these elections was accompanied by a spectacular cluster of blunders. The Tories won't become more competent nor discover a more fruitful economic strategy by lurching off to the right. All that is likely to do is further alienate centrist voters and strain relations with the Lib Dems to breaking point. The next few days will be an important test of David Cameron's character. We will see whether he seeks to appease the restless right of his party or faces them down. That decision may determine the fate of the coalition.