The Glasgow giants were twice forced to fight back from behind as goals from Jon Daly and David Templeton cancelled out strikes from David Banjo and Bobby Linn.
McCulloch's late spot-kick then proved to be decisive as Rangers managed to hold on for the win.
Goalkeeper Cammy Bell reclaimed the gloves - despite an impressive display from stand-in Steve Simonsen at Forfar on Monday - after the regular number one missed the last match due to the birth of his daughter.
Ian Black was also back in the starting line-up after returning from suspension but Arnold Peralta had to settle for a place on the bench after serving his own ban.
Arbroath had shipped eight goals - with just one in reply - in two previous meetings with Rangers this season.
But they had managed to open the scoring in a 5-1 defeat on their last visit to Ibrox and it was deja vu as the Red Lichties surged into the lead again on this occasion.
The match was just two minutes old when Alan Cook's cross from the left found Banjo in far too much space and he was able to dispatch into the back of the net at the back post.
Rangers attempted to hit back immediately and a corner cleared only as far as Lee Wallace saw the left-back launch a thunderous effort that flashed just past the post.
Black then sent a free-kick curling just over the bar, before the home side levelled on 21 minutes.
It was Black who started the move when he released Wallace down the left flank and he cut back for Daly to stroke past goalkeeper Scott Morrison.
Rangers were threatening again when Dean Shiels cut inside from the right only to smash his shot into the side-netting.
The home side were denied again just before the break when Bilel Mohsni's header was cleared off the line by an alert Alex Keddie.
Rangers still looked dangerous after the restart and an over-head kick from Shiels released Nicky Law through on goal but his effort was blocked by Morrison.
Incredibly, it was Arbroath who had the net bulging again after 50 minutes when a McCulloch mistake allowed Linn to break free of the Gers defence and unleash a shot that came off the inside of the far post and trundled over the line.
Jeers rang around the stadium as furious fans made their feelings at the shock scoreline known.
Lewis Macleod hobbled out of the action and was replaced by Templeton a minute after the goal, before Shiels was swapped for Andy Little three minutes after the first substitution.
It was one of those replacements who hauled Rangers back into the game with the second equaliser of the day after 72 minutes when Templeton forced his way into the box before drilling low past Morrison.
Seconds later, Mohsni also had the ball in the net, when he pounced on the rebound after Templeton's effort struck the post, but the close-range shot was ruled offside.
Rangers did manage to edge in front with 12 minutes to go when Daly was felled in the box and McCulloch stepped up to convert the resultant spot-kick with confidence to secure the points as the hosts survived a last-gasp scramble in their box.