Baxter joined Hull on a season-long loan from Chelsea in June but has only forced his way into Grant McCann's starting 11 this month.
The 23-year-old's emergence into the side has coincided with an upturn in form for the Tigers, who have won their last four games and conceded just one goal with Baxter between the posts.
The goalkeeper is open to staying in East Yorkshire beyond his loan deal but still has ambitions of making it in the Premier League with Chelsea.
Baxter has enjoyed playing in front of the City fans and paid tribute to the travelling Hull support after last week's midweek win at Cardiff.
“I still have ambitions to play for Chelsea, that’s been my dream since the age of seven when I joined the club,” he told BBC Radio Humberside.
“I’ve loved playing in front of the fans and have received a brilliant reception – I thought the support on Wednesday night (at Cardiff) was unbelievable.
“A late away night at Cardiff with all the Covid pass stuff, the turnout was incredible and I thought they out sung the Cardiff fans.
“The support we have received all year has been great.
“If and when that decision is made, obviously it’s something that I’m open to but with football, it’s just a case of taking it game by game."