Ally McCoist's side were far from their best as they turned in an abject performance to miss out on their second piece of silverware of the season.
The League One champions must now pick themselves up for their William Hill Scottish Cup semi-final clash with Dundee United next Saturday after they were booed off at the end of a low-quality 120 minutes.
Despite being played in front of a near 20,000 crowd in the Capital, there was little to shout about for either set of fans as both sides struggled to create clear scoring chances.
Rangers saw plenty of the ball in the first half but there was little spark and nous from a midfield containing Ian Black, Kyle Hutton and Stevie Smith.
The left-back was the surprise inclusion of the afternoon for the champions but he did come closest to breaking the deadlock in the opening 45 minutes, keeper Lee Robinson making a good save from Smith's curling free-kick.
Things did pick up after the break and it was Rangers who would again pass up the best opportunities, Bilel Mohsni heading over from just a couple of yards out.
McCoist had already lost key men Lee Wallace by that point and Nicky Clark, on for Hutton, came close a couple of times in the dying minutes.
Midway through the first period of extra time, Nicky Law saw a well-struck effort come back off Robinson's left-hand post as Rangers pushed in search of a breakthrough.
But it was John Baird who would finally find the net with just three minutes remaining, the former St Mirren and Partick Thistle striker converting after Cammy Bell had saved well from Greig Spence.
And it proved enough to give Raith the cup as Rangers fans made their feelings clear as McCoist's side trudged off the pitch.