The Archbishop of Glasgow – who was last week named O'Brien's temporary replacement – was giving a pre-scheduled Lent Station Mass last night at St Andrew's Cathedral.
He said that the church cannot "throw in the towel" and that the sadness will pass.
He said: "This is a sad moment for the church in our country. The events around Cardinal O'Brien, his resignation, his statement, have left us all very sad for everyone involved and for the church.
"Many reproaches have been aimed at the church and at individuals over this matter.
"The most stinging charge which has been levelled against us in this matter is hypocrisy, and for obvious reasons. I think there is little doubt that the credibility and moral authority of the Catholic Church in Scotland has been dealt a serious blow, and we will need to come to terms with that."
The Archbishop looked forward to the election of a new Pope in the coming weeks to offer hope to worshippers.
He added: "And this sad time for the Catholic Church in Scotland will also pass.
"We will not forget for a long time, but we will heal and we will carry on.
"As for the church's mission in our country, yes our credibility and moral authority have been undermined.
"It will take time, perhaps a long time to recover these intangible but important realities. We cannot be defeatist. The answer to this sad episode is not to throw in the towel."
The mass came at the end of a day in which it was revealed, in later edition's of yesterday's Evening Times, that Cardinal O'Brien will face a Vatican inquiry after admitting that his sexual conduct over the past 50 years "had fallen beneath the standards" expected.
The cardinal shocked the Roman Catholic community when he indicated that he would not contest claims against him and intended to retire permanently from the public life of the church.
The admission came a week after three priests and a former priest accused Britain's most senior Catholic cleric of inappropriate behaviour dating back to the 1980s.