James Collins’ volley in the 70th minute was the difference between the two sides at the AESSEAL New York Stadium as Town moved up to 9th position.
Manager Nathan Jones made six changes to the side that lost 3-0 to Brentford on Saturday, including a first start of the season for George Moncur and a return to the starting line-up for Harry Cornick, who missed the last four matches due to injury.
On a cold night in Yorkshire, Simon Sluga was on hand to provide two good saves during the first-half to deny the hosts an opener.
The first was particularly good, as he clawed away Jamie Lindsay’s effort from 20-yards, with a shot that was destined for the top corner if it wasn’t for his quick reaction.
Sluga denied the midfielder once more eight minutes later, at the near post this time as he turned the ball behind for a corner.
Town grew into the game as the half went on as James Collins found space in the box to deliver a cross towards Luke Berry, who left it for Moncur to hit first-time, but his low shot was saved by Jamal Blackman.
Further to that chance, Matty Pearson went close with a surging run forward and a well-hit low shot, but it went just wide of the target in added time.
The Hatters couldn’t quite believe they didn’t take the lead before the hour mark as a period of madness ensued.
Moncur crossed brilliantly for Collins but somehow the striker couldn’t turn the ball in from close range as it ricocheted behind for a corner.
Seconds later, Cornick stabbed the ball goalwards with Blackman stranded but Michael Ihiekwe managed to get back for Rotherham and cleared off the line.
Berry then tried his luck from the edge of the box, but the goalkeeper was equal to it and turned it behind.
Fortunately though, it wasn’t long before Town did break the deadlock and earned a well-deserved opener.
Berry produced an inch-perfect free-kick into the box and Collins volleyed the Hatters in front for his fifth goal of the season.
Town had several attempts to try and extend their lead, first as Moncur broke through towards goal with a lung-busting run forward but his tame chipped effort went straight into Blackman’s hands.
Moments later, Berry produced some great work down the right-hand-side, cutting into the box and then back onto his right foot, before hitting the post.
The Hatters held out and won in Yorkshire for the third time this season, following victories against Barnsley and Sheffield Wednesday prior to this evening's fixture.
Town are back on the road on Saturday when they travel to Huddersfield for the last game before the international break.
Town: Sluga, Cranie, Pearson, Bradley (C); Rea, Tunnicliffe, Berry, Morrell (Mpanzu 45), Moncur (Lockyer 89); Cornick (LuaLua 64), Collins.
Subs not used: Shea, Lee, Bree, Nombe.
Goals: Collins (70)
Yellows: Berry
Rotherham: Blackman, Harding, Ihiekwe, A. MacDonald, Mattock (Crooks 11); Jozefzoon (Ladapo 80), Wiles, Lindsay, Barlaser, Miller, Smith (Hirst 59).
Subs not used: Johansson, Wood, Sadlier, Vassell.
Yellows: Miller
Referee: Andy Madley