Rangers had to work hard for it, but the Third Division leaders saw off 10-man Peterhead thanks to Fran Sandaza's goal on the half-hour.
Even after the hosts saw Rory McAllister sent off midway through the second half, the visitors could not add to their advantage in what was an evenly-matched affair.
Missing captain Lee McCulloch through an ankle injury, the visitors made just one change to the side that had beaten Berwick Rangers handsomely last weekend, with Ian Black returning to the side in place of Dean Shiels, having completed a one-match suspension.
Spaniard Sandaza led the line, with support coming from Andy Little and David Templeton on the flanks.
Peterhead are very much in the mix for the play-off places at the moment, and had drawn 0-0 with Queen's Park at Hampden last weekend. They made two changes from the side that had lined up at the National Stadium, with David Cox and McAllister coming in for Robbie Winters and Martin Bavidge.
And it was Peterhead who were first to threaten. Dean Cowie won a free kick in the midfield and Neil Alexander in the Rangers goal did well to turn away a Scott Ross header from Ryan Strachan's delivery.
The goalkeeper looked less comfortable as he tried to claim Cox's delivery at the near post, the former Scotland international relieved to hold the ball at the second attempt.
Rangers' first effort on goal saw Black seize upon Scott Ross' slack pass, to fire a snapshot from 25-yards, but his effort was always curling away from the target.
It had been a stuffy opening period, with both teams still settling into the game when Peterhead's Ryan McCann became the first player to go into the book for a cynical foul on David Templeton.
The goal then came the half-hour. Peterhead's Graeme Smith, himself a former Rangers goalkeeper, did well to save a Wallace shot, but could not hold on and Sandaza reacted first to fire the ball high into the net.
Peterhead hit back, and some fine hold-up play by McAllister created an opening for Cox, but he dragged his effort wide of the post.
And the Blue Toon started the second half brightly, too, as Cox squeezed out Black on the touchline and cut inside, before flashing a left-footed effort over the crossbar.
There was more of an edge to proceedings early in the second period, with Cowie putting in a couple of typically robust challenges which seemed to unsettle the Rangers players slightly. Certainly, referee Mike Tumilty had a quiet word to encourage the local lad to calm things down.
Tumilty did have his yellow card out again shortly before the hour mark, as McAllister went into the book for a foul on Wallace, and Strachan swiftly became the third player booked.
The visitors were starting to gain a territorial advantage, and came close when Lewis McLeod's cross found the head of Chris Hegarty at the back post, but his header failed to trouble Smith.
Peterhead's task was made much harder after 65 minutes when McAllister was adjudged to have handled in the area, and received a second booking, reducing his side to 10 men.
The home side had certainly been more physical in the early stages of the second half but they will have been disappointed to lose their main goal threat so softly.
Rangers were dealt a blow themselves as Black had to be withdrawn through injury shortly after, Robbie Crawford replacing him in the midfield.
But with a man advantage and Peterhead's discipline starting to go - Ross was booked within minutes of the red card - the visitors began to take a firm grasp of the game.
Indeed, Templeton worked his way into the box with 20 minutes remaining, only to drag his low shot wide of Smith's far post.
Kyle Hutton became the first Rangers player to be booked, after a foul on Cowie, with 15 minutes to go, and that came as the home side began to find their way back into the game.
Rangers could have sealed the points with five minutes remaining. McLeod 's clever reverse pass found Templeton in acres of space on the 18-yard line, but the former Hearts man sent his effort wide of the upright.
Peterhead were determined to battle for everything though, and that was evidenced as Ross Smith became the fifth Blue Toon player to pick a up a booking - the central defender throwing himself into a challenge midway inside the Rangers half.
In injury time, Alexander was forced to turn a Cox corner over his own crossbar, but Peterhead could not find an equaliser that few would have begrudged them.