The U's now need just one point from their final game at Yeovil Town next weekend to be assured of automatic promotion alongside Essex neighbours Southend.
Colchester began brightly but their opening goal was bizarre in the extreme.
Rotherham defender Shaun Barker tried to head a cross from Neil Danns back to his goalkeeper, only for the ball to loop beyond a stranded Neil Cutler and roll into an empty net for a soft own goal.
That put the U's 1-0 up after 29 minutes which was no more than they deserved for a dominant opening.
In fact, they had the ball in the back of the net after only two minutes although Scott Vernon had clearly handled the ball when diverting back home Greg Halford's long throw.
The Essex hosts were also denied by the woodwork when star man Mark Yeates, who has been on loan from Tottenham all season, let fly with a rising shot that struck the underside of the bar. Vernon screwed his shot wide from the rebound.
Rotherham had their chances in the first half although young keeper Dean Gerken, who was starting his first home match since last August, was equal to the efforts of Lee Williamson and striker Paul Shaw.
Colchester settled their nerves by scoring the all-important second goal early in the second half. Vernon helped on a flick from Chris Iwelumo into the path of Yeates and the Irishman advanced a couple of yards before beating Cutler with a marvellous right-footed shot from just inside the penalty area.
It was Yeates' sixth goal of the season and it ensured that Colchester ended their home campaign on a winning note.
They were content to sit back for the final half-hour and Rotherham had just one good chance to give themselves hope when striker Martin Butler fired a half-volley agonisingly wide of the far post.
Rotherham now play host to fellow relegation strugglers MK Dons on the final weekend of the season.