Mark Carney said the EU referendum has had a noticeable impact on the UK economy, having lowered growth by "up to 2%" against what the Bank had expected in 2016 if the UK had voted to remain in the EU.
"That's a reasonable difference," he said.
"Real household incomes are about Â£900 lower than we forecast in 2016.
"The question is why and what drove that difference. Some of it is ascribed to Brexit."
Speaking to the Treasury Select Committee, he said business investment was still being held back, but there was a chance there could be a "sharp pick-up" when the Brexit agreement is finalised.
"It's understandable why businesses are holding back - there's some big decisions that are about to be made - why wouldn't they want to wait until the path becomes clearer?" he told MPs.
It comes after the bank downgraded its 2018 growth forecast to 1.4%, from 1.8% predicted in February, though forecasts remained unchanged at 1.7% for 2019 and 2020.
The Bank's quarterly forecasts revealed some doubt around the underlying economic picture in the UK, clouded by the recent weather impact, with the Bank noting "greater-than-usual uncertainty" over consumer spending.
The inflation forecasts indicated that any lost economic output in the first quarter would not be recovered throughout the remainder of the year, despite expectations for the second quarter to be "commensurately stronger" at around 0.4%.
Speaking to MPs about his reappointment to the Bank of England's interest rate-setting Monetary Policy Committee (MPC), Gertjan Vlieghe said the fact that the UK's economic growth has fallen behind the rest of the G7 was "circumstantial evidence for the effect of Brexit".
"It's not proof that Brexit is having that effect but (is) combined with all the other things we know about it.
"So we know that UK investment growth has dipped below the investment growth of the other countries and has done that at a time when the global economy has strengthened.
"Where we go from here will depend very much on what sort of decisions are taken in relation to our future trading relationship with the EU."