Yet even in victory, the Glens boss was left nursing a bitter sense of injustice over a controversial display from officials that could easily have cost his club their first quarter-final spot since 1992. They will find out their opponents tomorrow.
Undoubtedly their biggest calamity ended his participation in proceedings only 15 minutes in as he was sent from the dugout for protesting that whistler James Campbell had blundered in ruling that Pumpherston winger David Swan's shot off the underside of the crossbar had crossed the line.
Harvey said: "The linesman was far better placed than the referee to make the call yet never signalled the ball was in so I was simply asking him to raise his flag and give his take on things.
"He did, but only to complain about my actions which I find a bit of a get-out and very disappointing given all I wanted was for him to fulfil his role in the game.
"This is an emotional game and if debatable calls are made then you have to be allowed to vent your feelings as long as it's not abusive."
The absence through injury of experienced quartet Andy McLay (injured in a midweek car smash), Ross Ballantyne, Dean Muir and Willie Galloway saw a rejigged Glens line-up struggle to assert any cohesive authority in the opening phases.
However, they still forged ahead in 12 minutes with a score of simplistic beauty. Joe McAlpine rifled a quickly-taken free-kick down the left flank to meet the run of striker Denis McLaughlin, whose first-time volleyed cross was met and powerfully headed home by Brett Agnew from around six yards.
Just three minutes later, visiting winger Swan capitalised on the space afforded him by makeshift Glens full-back Sean McGuire in ghosting clear to controversially level matters at 1-1.
Not long afterwards, assistant referee Ivan Stankovic compounded his earlier lack of involvement by totally ignoring a blatant trip on McAlpine and the upshot was him taking exception to a blast from Glens coach Joe Pryce, who was duly sent to join gaffer Harvey on the terracing.
There was still time for a decisive second goal for the Ru'glonians on the stroke of half-time as Agnew made room for a shot that was parried by Pumphy goalkeeper Mark Reid, but only to the bustling frontman, who reacted quickest to rifle home a low shot for 2-1.
The home team's pressure was incessant in the second half and another score looked on the cards, however, totally against the run of play, Pumpherston broke upfield in 74 minutes to equalise for a second time through dangerman Swan.
The beleaguered East Region side looked content to kill time and hope for a replay this week, but that man Agnew had other thoughts.
His surging run into the penalty box with three minutes of regulation time left led to a melee that saw McCarry bang the ball home and spark wild scenes of celebration.
Harvey added: "We made it a lot harder for ourselves than it should have been because of missing chances to put the game out of sight at 2-1.
"But, that said, it's hard to be over-critical considering almost half a team of regulars were unavailable and we had guys playing out of position.
"Brian Welsh deserves special mention for getting through just over an hour in midfield seeing as last Wednesday night's training session was his first workout in the past three months."
Plenty of backslaps greeted the Glens players coming off the pitch, with most reserved for hard-working frontman Agnew.
He said: "The manager is right to say we were sloppy in front of goal and I was as guilty as anyone in that respect, so it was a massive relief to see Mark grab that late goal."
Glencairn's success was the one ray of Emirates Cup sunshine on an otherwise disappointing day for West Region sides after both Kilsyth Rangers (2-0 to Kelty Hearts) and Rob Roy (3-2 to Bo'ness United) crashed out against East Region opponents.