Despite Kevin Watt giving the Bully Wee the lead at Shielfield, Ian Little's side fought back to claim all three points.
Next up for Clyde is the visit of league leaders Rangers this weekend.
The clash against Ally McCoist's team is their final fixture in the opening quarter of the campaign.
Rangers slipped to defeat on their last away outing in the Irn-Bru Third Division, falling to Stirling Albion before the international break.
Now Duffy is targeting a swift recovery from his side's Berwick disappointment this Sunday.
The manager said: "The players put in plenty of effort but we lacked spark in the danger area.
"I can't complain because the guys had absolutely no sparkle left.
"Now we really need to pick ourselves up to face Rangers.
"Just to underline our situation, we seemed to be denied a goal in the last minute. That could have earned us a point."
Saturday's Shielfield defeat leaves the Bully Wee eighth in the Third Division standings, four points above basement boys Stirling as their winless run stretched to six matches.
Watt had them ahead just before the break but it wasn't to last, Dean Hoskins levelling for the Borderers just minutes later.
After Iain Gray was sent off after the hour mark, Steven Notman netted the decisive goal for Berwick.
John Neil thought he had snatched a point for the Bully Wee but his effort was flagged offside to leave the Cumbernauld club frustrated.