James Alexander warned it was not enough to argue for control over powers and money to be moved out of Whitehall and Westminster.
Speaking at a conference on the future of local public services,the York Council leader said more clarity was needed if the devolution argument is to be won.
Coun Alexander was speaking after Scottish Secretary Alistair Carmichael admitted English devolution is “unfinished business” at the weekend.
The York Council leader said: “I’m going to be speaking to a number of council leaders in the region to try and use those comments to further that cause.
“Some of my colleagues have already said the genie is out of the bottle when it comes to devolution.
“I think that’s right, but what does it mean and what is it exactly that we are asking for? Because there has to be an articulated case over what devolution means and whether that means to the North of England or whether that means to Yorkshire and the Humber or is it the combined authorities and so on.”
Coun Alexander said it was important devolution was accompanied by fair funding for services in the North.
Mr Carmichael had acknowledged the shift of power to Belfast, Cardiff and Edinburgh had created anomolies that mean, for example, Scottish MPs can vote on matters that only affect England.
But he said the English needed to decide whether they wanted an English parliament, regional assemblies or some other mechanism to address the current imbalance.