The new Environment Secretary insisted the money did not amount to a "partisan deal", despite Theresa May needing the support of the DUP's 10 MPs after losing the Tories' Commons majority in a disastrous general election.
It came as the Conservatives confirmed a Mail on Sunday report that the party paid £20,000 to fly DUP leader Arlene Foster to Belfast on an RAF plane after talks in Downing Street, despite there being 38 commercial flights that run between the two countries and tickets starting at £41.
Tory backbencher Heidi Allen, who has already expressed her "distaste" at her party using public money "to garner political control", warned that her party has become "too inward looking" and has "forgotten the purpose".
But Mr Gove insisted a deal was needed to ensure a "secure and stable" Government and highlighted Northern Ireland's unique problems, dealing with the Troubles and needing investment in mental health and infrastructure.
Told the confidence and supply deal was a bung on BBC One's Andrew Marr Show, Mr Gove said: "I think that's unfair, I think that's unfair to the people of Northern Ireland and I think it's also unfair to the way in which during this negotiation decisions were taken in the interests of everyone in the United Kingdom.
"Bung is, the implication is... it implies this money is somehow going to the DUP on their own as if it were a partisan deal. It's not. It's about strengthening the whole United Kingdom by helping people in one of its most vulnerable areas."
Amid DUP opposition, the Tories have now scrapped manifesto pledges to end the triple-lock and to means-test the winter fuel allowance for pensioners.
"It's absolutely right that we should, after a general election in which we didn't secure a majority, that we should have an opportunity to review how we help the most vulnerable in our society," Mr Gove said.
Ms Allen denied she was on the edge of leaving the party, telling the programme: "I hope I'm on the edge of seeing the Conservative Party come back to the one I know can and should be there.
"We have got to change our tone and our language.
"I think we've become too inward looking, we've forgotten the purpose, why we're there, and I'm reminded constantly when talk to my constituents what they're looking for."
On Ms Foster's flight, a Conservative spokesman said: "Arlene Foster was in London to conclude the signing of the Confidence and Supply agreement between the Conservative Party and the Democratic Unionist Party which will provide the UK with the stable Government it needs at this critical time.
"The Conservative Party paid for a flight back to Belfast to allow her swiftly to resume talks on re-establishing a powersharing Executive in Northern Ireland."