Georgios Samaras opened the scoring in the 32nd minute with a header from an Emilio Izaguirre corner but Jags striker Kris Doolan levelled in the 65th minute from close range.
However, 10 minutes later Balde, on for Beram Kayal in the 66th minute, finished off a solo run by sliding the ball past Jags keeper Scott Fox to take the three points back to Glasgow and leave the Hoops unbeaten in the league this season.
Thistle showed commendable spirit all afternoon but are still looking for their first home league win of the season although their fans gave them a round of applause at the end in appreciation of their efforts in what was a decent game.
Celtic skipper Scott Brown returned from suspension to take over from Charlie Mulgrew, out with a heel injury, while Joe Ledley came in for James Forrest who was ruled out with an ankle complaint.
Thistle boss Alan Archibald kept the same starting X1 who beat Inverness 2-1 last week with Doolan, who scored both goals, leading the line.
As the cascading rain and strong wind showed little sign of abating, it was the visitors who had the first effort on goal in the fifth minute when Ledley fired over the bar from 20 yards after being set up by Anthony Stokes.
In the 13th minute Jags midfielder Isaac Osbourne was booked by referee Willie Collum for a foul on Brown as the Hoops midfielder drove towards the box but Stokes curled the free-kick wide of the target.
Thistle were not short on enthusiasm and 10 minutes later defender Aaron Taylor-Sinclair went past Hoops right-back Mikael Lustig with ease but his angled shot from 12 yards was easily dealt with by Celtic keeper Fraser Forster.
On the half-hour mark, with the game going nowhere in particular, Kayal put in Stokes with a clever pass but his left-footed was blocked to safety by Fox.
The league leaders started to drive forward with more purpose and determination but Thistle were left pointing at each other when the unmarked Samaras walked on to Izaguirre's in-swinging corner, which had been too easily conceded, to power a header over the line.
Twice in two minutes Teemu Pukki had chances to double the champions' lead but both times Fox saved at the expense of a corner, the second from which defender Virgil van Dijk headed wide.
In between, Partick forward Kallum Higginbotham fired wide of the target from the edge of the box.
There was a Celtic penalty claim just after the break when Brown, attempting to reach an Izaguirre cross, went to ground inside the box after a challenge by Jags defender Conrad Balatoni but the flag was up for offside.
The Maryhill men were then gifted an opportunity when Celtic defender Efe Ambrose passed the ball straight to Stuart Bannigan 30 yards from goal but the Scotland Under-21 midfielder curled a weak shot straight to Forster.
Doolan's equaliser came seconds after he had hit the bar with a flick from close range, after Balatoni had headed on Higginbotham's corner.
When the ball found its way back out to Higginbotham, he crossed it back into the middle where Doolan controlled it before poking it past Forster from a few yards out.
The home side harboured hopes of a famous win but they were to be outdone again.
Balde replaced Kayal and Dylan McGeouch came on for Pukki but in between the two substitutions, Izaguirre should have had Celtic ahead again when he worked his way into the box but the left-back fired over the bar from 12 yards.
At the other end, Forster parried Higginbotham's angled-drive and then Celtic raced up the other end with Balde showing strength and composure as he held off Jags stopper Aaron Muirhead to slip the ball past Fox.
Izaguirre was then carried off on a stretcher to be replaced by Nir Biton and when Samaras limped off with five minutes remaining the Hoops were down to 10 men.
However, Celtic saw out the five minutes of added time to take the points.