The award was bestowed on Madiba in 1981 while he was imprisoned on Robben Island.
The city continued to mark its support for the campaign to free him in 1986 when it renamed St George's Place - the site of the South African consulate - as Nelson Mandela Place in his honour.
Thousands gathered in George Square when he finally visited the city to receive the award in October 1993, the year before he became president. Mr Mandela thanked the citizens of Glasgow and said the award was a "tremendous" personal honour.
During his speech in Glasgow, he said: "While we were physically denied our freedom in the country of our birth, a city 6000 miles away, and as renowned as Glasgow, refused to accept the legitimacy of the apartheid system and declared us to be free."
And the day after his death, famous Scots paid tribute to Mr Mandela.
First Minister Alex Salmond praised the anti-apartheid icon as an "inspiration to countless millions".
Former prime minister Gordon Brown said the 95-year-old had been "the greatest leader of our generation".
Glasgow's Lord Provost, Sadie Docherty, described Mr Mandela as a political and moral icon. She said: "Nelson Mandela dedicated his life to bringing freedom, justice and equality to the people of South Africa.
"His beliefs cost him years of his own freedom but his vision for peace and democracy prevailed. His legacy will live on and inspire generations to come.
"Glasgow was proud to be the first city in the world to honour him with a Freedom of the City award and he will be sadly missed by a city which had the greatest of respect for him."
Holyrood's Presiding Officer, Tricia Marwick MSP, has written to the South African high commissioner to offer condolences on behalf of the Parliament.
She said: "Nelson Mandela was an inspirational figure, not just in South Africa but throughout the world and particularly here in Scotland.
"On behalf of all members of the Scottish Parliament, I extend our sincere sympathies to Mr Mandela's family and to the people of South Africa."
Scottish Labour leader Johann Lamont said: "He fought injustice when there seemed to be no chance of victory. But he fought, he endured and he won.
"Nelson Mandela was the best of Africa. He was the best of humanity. He was the best of us all."
Scottish Conservative leader Ruth Davidson said Mr Mandela was "a giant of our age".
She said: "He healed his nation and, in doing so, inspired millions. He showed the world that reconciliation could be a more powerful force than retribution."
Mr Mandela had been patron of the Scottish Refugee Council since 1995.
Judith Robertson, chairwoman of the board of directors, said the charity is "deeply grateful" for his support and will continue to take "hope, inspiration and dignity" from his life.
Professor Alan Miller, chairman of the Scottish Human Rights Commission, said Mr Mandela transcended race, class and nationality to "epitomise the human spirit" and "bring out the best in all of us".
Figures from the Scottish arts scene also paid tribute to Mr Mandela.
Glasgow band Belle & Sebastian tweeted an image of Mr Mandela's visit to Glasgow in 1993.
And Stuart Braithwaite, of Glasgow rock band Mogwai, also mourned his passing with a number of tweets.