Celtic target Jordan Rhodes netted a double as Scotland came from behind to look like they would secure the win thanks to the Huddersfield Town star's last-minute goal.
However, Bulgaria equalised with the last kick of the game to sicken the Scots.
Scotland are still top of Group 10, but have now played a game more than Holland. Stark's boys end the campaign with a double-header against Austria and Luxembourg in September.
And Rangers kid Perry believes Scotland are capable of getting six points from their last two games.
The Ibrox starlet said: "We need to recover from this and we still have a chance to get through.
"We have to win our last two matches and we will be looking to take six points from the double-header in September. There are plenty of teams who have came here and found it tough and had a hard time.
"Holland only got a late winner here and Austria got beaten, so we knew it was going to be a hard place to come.
"We have always had confidence in our ability though, and in any game we go into, especially after the game in Holland, we think we can win.
"But if you look back at the game, maybe a point is not a bad result and we are still confident."
Scotland fell behind in the 33rd minute after a tough half hour.
The home side were finding plenty of joy with quick, sharp passing with Scotland looking sluggish and Georgi Milanov played a one-two with Momchil Tsvetanov outside the box before the Litex Lovech man struck a low shot that crept into Mark Ridgers' goal, with the Tynecastle keeper possibly unsighted.
The Scots then blew a glorious chance to take the lead before that when Dundee United's Johnny Russell missed an open goal after delaying his shot when he had the gaping net at his mercy.
Rhode's double – his eighth of a remarkable campaign – was the perfect end to a perfect season for the striker, who was on a high after helping Huddersfield Town to promotion to the npower Championship last week after beating Sheffield United at Wembley.
He equalised in the 69th minute when a short corner was played to Kilmarnock ace Liam Kelly and his chipped cross was nodded in by the striker who had found space in the box.
The Scots had started to dominate the game at this stage with Bulgaria struggling to contain Stark's men.
And Rhodes thought he had won the game at the death for the Scots with a typical predator's finish after substitute Stuart Armstrong had cut the ball back to him and the former Ipswich Town star turned and volleyed the ball into the net for a dramatic goal.
It caused scenes of joy on the Scots' bench as a vital three points seemed within reach.
However, Bulgaria had other ideas and Georgi Kostadinov scrambled the ball into the net with the very last kick of the game to deliver a hammer blow to the Scots.
