His comments follow the decision in July to scrap plans to open a new state-of-the-art manufacturing and weaving facility at the Grade I listed Temple Works building in Leeds' South Bank.
The luxury fashion brand has said it is considering whether to develop the 10 acres of land it owns next to the building or whether to maintain the sites it currently owns at Castleford and Cross Hills in Yorkshire. Another option would be to build a new green field site.
The Temple Works building would have involved an initial investment of over £50m in the heart of Yorkshire and was expected to provide permanent employment for more than 1,000 people. The plans were put on hold after the Brexit referendum last year.
Mr Bailey told the BBC: "Since we made that decision and bought that land a lot of things have changed in the world and as any responsible organisation, when you have these big shifts you need to reflect.
"We are absolutely committed to keeping our manufacturing in this country with our factories in Yorkshire, but with the new site we're just taking a moment to make sure we understand the ramifications."
Asked on the BBC's Today programme what the potential is for post-Brexit Britain, Mr Bailey said it was "enormous".
"I think it's a much smaller world today than it's ever been in terms of being able to trade," he said.
"The possibilities and the potential for growth, both locally here in the UK and overseas, is enormous."