Boro have booked their play-off place with a game to spare, although the identity of their opponents will not be clear until they have completed their campaign at Ipswich next weekend.
Millwall still have a statistical chance of claiming a play-off place, although in reality their hopes are gone as they would need a nine-goal swing to overhaul Derby on goal difference even if they were to win their final game against Aston Villa and both the Rams and Preston were to lose.
They were up against it from the moment Assombalonga opened the scoring, and their fate was effectively sealed when Jordan Archer's goalkeeping error enabled Howson to double Middlesbrough's lead midway through the second half.
Boro went into the game knowing only a victory would guarantee their place in the play-offs, and Tony Pulis' side made an ideal start as they claimed the lead after just 12 minutes.
Pulis opted to keep Assombalonga in the starting line-up despite the availability of a fit-again Patrick Bamford, and his faith in his club's record signing was rewarded as he claimed his 15th goal of the season with the first opportunity that came his way.
Controlling Daniel Ayala's long ball out of defence with a delicate first touch, Assombalonga spun inside Mahlon Romeo before drilling home a low finish via the inside of the left-hand post.
The goal was a fitting reward for Boro's bright, industrious start, and the hosts went on to dominate the majority of the first half.
Stewart Downing twice went close with efforts that were blocked, first by Ryan Tunnicliffe and then by Shaun Hutchinson, who produced an excellent last-ditch challenge in the heart of his own penalty area.
Both chances were set up by Mo Besic, and for the second game in a row the Everton loanee was the driving heart of the Middlesbrough midfield, surging forward with the ball at his feet and constantly looking to ask questions of the Millwall defence.
The visitors have been one of the surprise packages of the Championship season, but while they enjoyed plenty of possession as they looked to get back into the game, they struggled to carve out chances in the final third.
Steve Morison headed Shaun Williams' corner over the crossbar in first-half stoppage time, but that was a rare moment of concern for a Boro defence that has improved markedly since Pulis replaced Garry Monk.
The hosts needed a second goal to provide a degree of comfort, though, and it should have arrived four minutes after the break. Adama Traore ghosted behind the Millwall defence to reach Besic's through-ball, but while he brought the ball down neatly, his shot was skewed wastefully wide.
The Teessiders finally doubled their lead in the 66th minute, and their second goal owed much to a glaring error from Archer. The Millwall goalkeeper allowed Downing's left-wing cross to slip through his grasp, leaving Howson with the simple task of slotting home from inside the six-yard box.