A Cai Hockey own goal in the first half and a strike from Josh Keyes in the second period earned the Robins a victory and a place in the fourth round.
Alex Lawless’ side began strongly, forcing Swindon keeper Redman Evans into a string of saves, the best coming when he denied Claude Kayibanda twice in quick succession following two corners from Jack Lorentzen-Jones.
However, the visitors rode that early storm and the Town were indebted to goalkeeper Hockey when he saved Fletcher Hubbard’s penalty on 32 minutes after Christian Chigozie had fouled Botan Ameen.
Hockey then saved well to keep out Harry Chard’s effort three minutes before the interval but then in first half stoppage time the Town number one diverted the ball into his own goal from a corner to give the Robins the lead.
Six minutes after the break Hockey pulled off a vital save to keep it 1-0 but, shortly after the hour, Swindon doubled their lead when striker Josh Keyes poked home from 12 yards.
Half-chances then came and went for either side. Lorentzen-Jones headed over on 66 minutes and Jamie Odegah drove a shot wide of the target four minutes later as the Town searched for a way back into the tie.
Swindon came close to putting the game beyond the reach of the Town when Ameen’s effort crashed against the crossbar and Miles Obodo somehow scooped the ball over the bar.
Time was against Lawless’ side and it was the Town who then were denied by the woodwork when Benedict Benagr saw his drive clip the bar with two minutes left.
But despite seven minutes additional time, the visitors saw out remaining minutes to book their spot in the next round at the Hatters’ expense and a meeting with either Derby or Manchester United.
Town: Hockey, Giwa, Johnson, Benagr, Chigozie, Kayibanda, Lorentzen-Jones, Trustram (sub Martins 74), Odegah (sub Takawira 90+4), Anderson, Xavier-Jones (sub Houghton 74)
Subs not used: Thomas, Ioannides, Harvey.
Attendance: 463