Shadow home secretary Yvette Cooper said Mr Cleverly's comments proved he never personally believed in the plan – a day after a Supreme Court ruling left the scheme to send migrants to the African country in tatters.
And former Conservative attorney general Dominic Grieve said it showed the government’s approach on Rwanda was “not coherent”.
It came as former Supreme Court Justice Lord Sumption accused ministers of attempting to use parliament to say black was white and warned the planned new treaty with Rwanda would face opposition in the House of Lords.
He hit out at Mr Sunak’s announcement that he would bring in emergency laws to deem Rwanda a safe country for asylum seekers, saying he had never heard of a government “trying to change the facts, by law”.
It came as:
Jeremy Hunt admitted the government can't guarantee that flights to Rwanda will take off next year
The Bar Council, which represents the legal profession in England and Wales, said it had “grave concerns” about Mr Sunak’s planned law change
The new treaty faces significant opposition in the House of Lords
Mr Cleverly failed to deny he had used the phrase in an excruciating round of interviews on the policy.
He was repeatedly questioned about his use of language after Ms Cooper told MPs in the House of Commons: “I don’t believe the new home secretary ever believed in the Rwanda plan. He distanced himself from it and his predecessor’s language on it. He may even on occasion have privately called it ‘batshit’”.
Asked by the BBC whether he had described the policy as “batshit”, Mr Cleverly said: “I certainly don’t remember saying anything like that.”
Separately he told Sky News: “I don’t recognise that phrase ... the Rwanda scheme is an important part, but only a part, of the range of responses we have to illegal migration.”
In a third interview, on BBC Radio 4’s Today programme, he again said he could “not remember” having used the phrase.
The home secretary hit out at the Labour Party, accusing it of setting a “trap” and claiming the official opposition “would love us to discuss this particular issue rather than the gaping vacuum in its immigration policy”.
Mr Grieve said Mr Cleverly’s comments illustrated that the government’s approach on Rwanda was “not coherent”.
And, addressing Mr Sunak’s plans to save the Rwanda scheme, Mr Grieve told The Independent: “It is a wheeze and I think like all wheezes it is in danger of coming seriously unstuck.”
Earlier Lord Sumption said: “I have never heard of them [the government] trying to change the facts, by law.
“For as long as black isn’t white, the business of passing acts of parliament to say that it is, is profoundly discreditable.”
He added that the plan “won’t work internationally. It will still be a breach of the government’s international law obligations” and risked not getting through the House of Lords. He told The Independent: “I would have thought it was obvious that the Lords is going to be the main focus of opposition and they may throw it out altogether.”
It is understood a Labour MP, not Ms Cooper, says they heard Mr Cleverly call the Rwanda policy “batshit”.
The revelation of an apparently private discussion is reminiscent of John Major’s “b*****ds” scandal, when in 1993 he was caught on camera denouncing three of his own cabinet members over Europe.
Former Labour minister Ben Bradshaw told The Independent that “batshit” was “too mild a term” for the Rwanda scheme because the policy is “dishonest, damaging and hugely expensive”.
Liberal Democrat home affairs spokesperson Alistair Carmichael said “even the home secretary knows how unworkable, expensive and flawed this ridiculous scheme is”.
The charity Refugee Action said Mr Cleverly could have used “plenty” of other words to describe the Rwanda deal, including “cruel, inhumane, racist, hostile, unworkable or expensive”.
Mr Cleverly’s apparent comments, made before he became home secretary, were revealed as the government scrambles to rescue its plans to send asylum seekers to Rwanda.
The prime minister has vowed to bring in emergency legislation and to strike a new treaty with the African country, saying he would “not allow a foreign court to block these flights”.
But he has resisted pressure from those on the right of his party to immediately pull the UK out of the European Convention on Human Rights to push the plans through.
Mr Sunak has said he hopes the first removal flights will leave in the spring, but Mr Hunt said the government “can’t guarantee“ planes deporting asylum seekers to Rwanda would take off next year.
On Thursday morning, Mr Cleverly said the new treaty would be published “within days, not weeks” – but he refused to say when the first migrants would be deported.
After the Supreme Court issued its damning verdict, the PM faced a furious backlash from Tory MPs who urged him to ignore the law and “get the planes in the air” while warning that his job was on the line.
Other notable political gaffes include when Labour prime minister Gordon Brown was humiliated in 2010 when a private conversation with an aide was caught on microphone, revealing he had called a Labour supporter who challenged him over the economy and immigration a “bigoted woman”.
Mr Brown was later seen with his head in his hands as his remarks were played back to him in a radio studio.
The new foreign secretary David Cameron once had to apologise to the Queen for having claimed she “purred down the line” during a private phone call in which he told her Scotland had voted against becoming independent.
In May 1991 the then chancellor Norman Lamont drew criticism for claiming that unemployment and a recession were “a price worth paying” to bring down rampant inflation.
And former education secretary David Blunkett once clashed with then Downing Street spin doctor Alastair Campbell after the latter described comprehensive schools as “bog standard”.