Kallum Higganbotham's spectacular overhead finish inside the opening three minutes looked like it had handed the Jags their first league win in Maryhill since beating promotion rivals Morton during last season's First Division title run in.
But Aaron-Taylor Sinclair's last-man foul on WIlliam Gros a minute from time saw him sent off and the visitors handed a penalty.
Stand-in Thistle keeper Paul Gallacher saved Kris Boyd's penalty but teenager Muirhead was the quickest to react as he slammed home the rebound to secure a 1-1 draw.
The opener was the least Thistle deserved from a dominant first-half display but they failed to build on Higganbotham acrobatic effort just 150 seconds in.
Visiting winger Chris Johnstone was forced to concede a thrown near the right-hand corner flag but the danger was not over there and from Stephen O'Donnell's throw, Killie skipper Manuel Pascali failed to put enough distance on his clearing header, leaving Higganbotham perfectly placed to net with the bicycle kick.
It was a fine effort but away keeper Craig Samson should have shown a stronger wrist as he was beaten despite getting down in time to save.
Thistle really should have made more of their possession but at that point Killie were struggling to threaten.
The Rugby Park outfit were obviously frustrated by their lack of penetration and it got the better of Muirhead who tried to engineer a shooting chance using suspect tactics on 32 minutes.
The 17-year-old went to ground easily as Jags debutant Lee Mair slid in but referee Steven McLean was not fooled by the obvious dive as he dished out a booking to the Kilmarnock player.
Kilmarnock then claimed for a penalty three minutes into the second half when Boyd claimed he had been felled by Aaron Taylor-Sinclair's flailing arm as the pair jumped together at a corner but McLean was unmoved.
Instead it was Pascali who was next to be penalised as he was booked for blocking Gary Fraser's shot with a hand five yards outside his own box. From the free-kick that followed, Taylor-Sinclair shot low but Samson did better this time as he pushed the strike away to his left.
O'Donnell was next to be booked after fouling Killie sub Greg Kiltie, while Boyd did well to turn Conrad Balatoni on the hour mark but sent a fierce dig wide of the target.
A second Thistle goal should have arrived on 65 minutes when Higganbotham did brilliantly to reach the byline but Stevie Lawless failed to read his cut-back and the chance was gone.
Samson then saved comfortably from Lawless with 20 minutes left after Stuart Bannigan robbed Jackson Irvine before teeing the midfielder up.
Partick threw-away a two-goal lead last week against Ross County as they blew the chance to record that long-awaited home win and once again a sense of nervousness descended on Firhill.
Alan Archibald had warned his men not to retreat too far towards their own box but that warning was not heeded as the Jags back four began to edge deeper, inviting Killie pressure.
With eight minutes left Jags keeper Gallacher - filling in for injured number one Scott Fox - had to block bravely from Boyd while Balatoni darted back to his line to head clear as Gros latched onto the rebound.
But Taylor-SInclair found himself the wrong side of Gros inside the box in the final minute and paid the price for a clumsy foul when he was shown a straight red by whistler McLean. Boyd's low effort was saved by Gallacher as he dived to his left but Muirhead was first in to slam home the leveller.