The Gers are overwhelming favourites to win the match at Ibrox next month and book a place in the semi-final for the first time in five years.
But McCoist recalls how the part-time minnows dumped SPFL Premiership high flyers Motherwell out of the competition in the fourth round.
He will ensure that his men do not take the threat posed by the Coatbridge club lightly.
He said: "To be fair to Albion Rovers, they've been unbelievable in the Scottish Cup so far this season. Obviously they put out Motherwell with what is arguably the result of the competition so far, but then they had a very comfortable 2-0 win over Stenhousemuir on Saturday.
"We have, of course, played them already this season in the Ramsdens Cup at Livingston and we had a good performance that day, with Nicky Law getting his first goals for the club.
"This is a different game, though, and just because we were quite comfortable winners back then, it doesn't mean we will be now. That's far from the case."
Speaking to the official Rangers website, McCoist went on: "They'll be saying it's a brilliant draw for them. They're in the quarters for the first time in 80 years and there will be no pressure on them.
"They will be able to come to Ibrox, enjoy the game and they'll have nothing to lose because everyone will expect us to win the tie.
"We'll have to be guarded and there will be no complacency from us. The fact we've the chance to play for a semi-final at Ibrox is a huge incentive for us.
"That's what we are playing for here, effectively a home tie in which we'd have a chance to get to the final and that's brilliant, but we'll treat Albion with all the respect they deserve.
"I wanted a home draw and that's all I asked for after the game on Friday night. We've got that so we've got to be pleased."